1
|
Mitchell EC, Ott M, Ross D, Grant A. Development of a Tool to Assess Surgical Resident Competence On-Call: The Western University Call Assessment Tool (WUCAT). JOURNAL OF SURGICAL EDUCATION 2024; 81:106-114. [PMID: 38008642 DOI: 10.1016/j.jsurg.2023.10.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 09/13/2023] [Accepted: 10/02/2023] [Indexed: 11/28/2023]
Abstract
BACKGROUND A central tenet of competency-based medical education is the formative assessment of trainees. There are currently no assessments designed to examine resident competence on-call, despite the on-call period being a significant component of residency, characterized by less direct supervision compared to daytime. The purpose of this study was to design a formative on-call assessment tool and collect valid evidence on its application. METHODS Nominal group technique was used to identify critical elements of surgical resident competence on-call to inform tool development. The tool was piloted over six months in the Division of Plastic & Reconstructive Surgery at our institution. Quantitative and qualitative evidence was collected to examine tool validity. RESULTS A ten-item tool was developed based on the consensus group results. Sixty-three assessments were completed by seven staff members on ten residents during the pilot. The tool had a reliability coefficient of 0.67 based on a generalizability study and internal item consistency was 0.92. Scores were significantly associated with years of training. We found the tool improved the quantity and structure of feedback given and that the tool was considered feasible and acceptable by both residents and staff members. CONCLUSIONS The Western University Call Assessment Tool (WUCAT) has multiple sources of evidence supporting its use in assessing resident competence on-call.
Collapse
Affiliation(s)
- Eric C Mitchell
- Department of Surgery, Western University, London, Ontario, Canada
| | - Michael Ott
- Department of Surgery, Western University, London, Ontario, Canada
| | - Douglas Ross
- Department of Surgery, Western University, London, Ontario, Canada
| | - Aaron Grant
- Department of Surgery, Western University, London, Ontario, Canada.
| |
Collapse
|
2
|
Liao KC, Ajjawi R, Peng CH, Jenq CC, Monrouxe LV. Striving to thrive or striving to survive: Professional identity constructions of medical trainees in clinical assessment activities. MEDICAL EDUCATION 2023; 57:1102-1116. [PMID: 37394612 DOI: 10.1111/medu.15152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 05/15/2023] [Accepted: 05/30/2023] [Indexed: 07/04/2023]
Abstract
CONTEXT Assessment plays a key role in competence development and the shaping of future professionals. Despite its presumed positive impacts on learning, unintended consequences of assessment have drawn increasing attention in the literature. Considering professional identities and how these can be dynamically constructed through social interactions, as in assessment contexts, our study sought to understand how assessment influences the construction of professional identities in medical trainees. METHODS Within social constructionism, we adopted a discursive, narrative approach to investigate the different positions trainees narrate for themselves and their assessors in clinical assessment contexts and the impact of these positions on their constructed identities. We purposively recruited 28 medical trainees (23 students and five postgraduate trainees), who took part in entry, follow-up and exit interviews of this study and submitted longitudinal audio/written diaries across nine-months of their training programs. Thematic framework and positioning analyses (focusing on how characters are linguistically positioned in narratives) were applied using an interdisciplinary teamwork approach. RESULTS We identified two key narrative plotlines, striving to thrive and striving to survive, across trainees' assessment narratives from 60 interviews and 133 diaries. Elements of growth, development, and improvement were identified as trainees narrated striving to thrive in assessment. Neglect, oppression and perfunctory narratives were elaborated as trainees narrated striving to survive from assessment. Nine main character tropes adopted by trainees with six key assessor character tropes were identified. Bringing these together we present our analysis of two exemplary narratives with elaboration of their wider social implications. CONCLUSION Adopting a discursive approach enabled us to better understand not only what identities are constructed by trainees in assessment contexts but also how they are constructed in relation to broader medical education discourses. The findings are informative for educators to reflect on, rectify and reconstruct assessment practices for better facilitating trainee identity construction.
Collapse
Affiliation(s)
- Kuo-Chen Liao
- Division of Geriatrics and General Internal Medicine, Department of Internal Medicine, Chang Gung Memorial Hospital (CGMH), Linkou, Taiwan (ROC)
- Chang Gung Medical Education Research Centre, CGMH, Linkou, Taiwan (ROC)
- School of Medicine, College of Medicine, Chang Gung University, Taoyuan City, Taiwan (ROC)
| | - Rola Ajjawi
- Centre for Research in Assessment and Digital Learning, Deakin University, Melbourne, Victoria, Australia
| | - Chang-Hsuan Peng
- Chang Gung Medical Education Research Centre, CGMH, Linkou, Taiwan (ROC)
- School of Medicine, College of Medicine, Chang Gung University, Taoyuan City, Taiwan (ROC)
| | - Chang-Chyi Jenq
- Chang Gung Medical Education Research Centre, CGMH, Linkou, Taiwan (ROC)
- Department of Nephrology, CGMH, Linkou, Taiwan (ROC)
- Medical Humanities Center, CGMH, Linkou, Taiwan (ROC)
- Department of Medical Humanities and Social Sciences, School of Medicine, College of Medicine, Chang Gung University, Taoyuan City, Taiwan (ROC)
| | - Lynn V Monrouxe
- Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
| |
Collapse
|
3
|
Kleijer EFW, Schuurmans MJ, Ten Cate O, Pool IA. Preceptors' considerations when entrusting professional activities to postgraduate nursing students: A qualitative study. NURSE EDUCATION TODAY 2023; 125:105799. [PMID: 36989637 DOI: 10.1016/j.nedt.2023.105799] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/04/2022] [Revised: 02/28/2023] [Accepted: 03/09/2023] [Indexed: 06/19/2023]
Abstract
BACKGROUND Nurse-preceptors regularly struggle to evaluate students' readiness to take care of patients unsupervised, even with sophisticated workplace-based assessment tools. Preceptors' gut feelings are not always captured well, but are critical for judgement of readiness for learner entrustment with care tasks. Studies in medical education report features that clinicians consider important when trusting students with clinical responsibilities that might also apply in nursing. OBJECTIVES To unravel preceptors' considerations when entrusting professional activities to postgraduate nursing students. The findings may contribute to the improvement of workplace-based assessments and the training of preceptors. METHODS Thematic analysis of semi-structured interviews with sixteen nurse-preceptors from three postgraduate nursing specialisations in Dutch hospitals. RESULTS Three themes emerged: CONCLUSIONS: For preceptors of postgraduate nursing students, entrustment requires more than merely insight into objectively measurable competencies. Entrusting is accompanied by subjectivity related to what preceptors expected of students. These expectations are in line with suggested factors in the literature-capability, integrity, reliability, agency, and humility-considered before entrusting students with clinical responsibilities identified in medical training. Entrusting is also accompanied by what preceptors realise about their own role in entrustment decisions. Combining different information sources made assessment more transparent and the implicit more explicit.
Collapse
Affiliation(s)
- E F Wilma Kleijer
- Academy, University Medical Centre Utrecht, Utrecht, The Netherlands.
| | - Marieke J Schuurmans
- Educational Centre, University Medical Centre Utrecht, Utrecht, The Netherlands.
| | - Olle Ten Cate
- Utrecht Centre for Research and Development of Health Professions Education, University Medical Centre Utrecht, Utrecht, The Netherlands.
| | | |
Collapse
|
4
|
Smith C, Patel M. 'Ticked off'? Can a new outcomes-based postgraduate curriculum utilising programmatic assessment reduce assessment burden in Intensive Care Medicine? J Intensive Care Soc 2023; 24:170-177. [PMID: 37260422 PMCID: PMC10227897 DOI: 10.1177/17511437211061642] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/20/2023] Open
Abstract
Context Increasing dissatisfaction with existing methods of assessment in the workplace alongside a national drive towards outcomes-based postgraduate curricula led to a recent overhaul of the way Intensive Care Medicine (ICM) trainees are assessed in the United Kingdom. Programmatic assessment methodology was utilised; the existing 'tick-box' approach using workplace-based assessment to demonstrate competencies was de-emphasised and the expertise of trainers used to assess capability relating to fewer, high-level outcomes related to distinct areas of specialist practice. Methods A thematic analysis was undertaken investigating attitudes from 125 key stakeholders, including trainees and trainers, towards the new assessment strategy in relation to impact on assessment burden and acceptability. Results This qualitative study suggests increased satisfaction with the transition to an outcomes-based model with capability judged by educational supervisors. However, reflecting frustration relating to current assessment in the workplace, participants felt assessment burden has been significantly reduced. The approach taken was felt to be an improved method for assessing professional practice; there was enthusiasm for this change. However, this research highlights trainee and trainer anxiety regarding how to 'pass' these expert judgement decisions of capability in the real world. Additionally, concerns relating to the impact on subgroups of trainees due to the potential influence of implicit biases on the resultant fewer but 'higher stakes' interrogative judgements became apparent. Conclusion The move further towards a constructivist paradigm in workplace assessment in ICM reduces assessment burden yet can provoke anxiety amongst trainees and trainers requiring considered implementation. Furthermore, the perception of potential for bias in global judgements of performance requires further exploration.
Collapse
Affiliation(s)
- Christopher Smith
- Intensive Care Medicine Trainee ST6, North West School of ICM, Mersey, UK
| | - Mumtaz Patel
- North West School of ICM, Health Education England, UK
| |
Collapse
|
5
|
Stephan A, Cheung G, van der Vleuten C. Entrustable Professional Activities and Learning: The Postgraduate Trainee Perspective. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2023; 47:134-142. [PMID: 36224504 PMCID: PMC10060374 DOI: 10.1007/s40596-022-01712-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Accepted: 09/16/2022] [Indexed: 05/25/2023]
Abstract
OBJECTIVE Entrustable professional activities (EPAs) are used as clinical activities in postgraduate psychiatry training in Australasia. This study aimed to explore psychiatry trainees' perceptions of the impact of EPAs on their motivation and learning. METHODS A constructivist grounded theory approach was used to conceptualize the impact of EPAs on trainees' motivation and learning. A purposive sample of trainees was recruited from across New Zealand. Semi-structured individual interviews were used for data collection and continued until theoretical saturation was reached. RESULTS The impact of EPAs on learning was mediated by the trainee's appraisals of subjective control, value, and the costs of engaging with EPAs. When appraisals were positive, EPAs encouraged a focus on particular learning needs and structured learning with the supervisor. However, when appraisals were negative, EPAs encouraged a superficial approach to learning. Trainee appraisals and their subsequent impact on motivation and learning were most affected by EPA granularity, alignment of EPAs with clinical practice, and the supervisor's conscientiousness in their approach to EPAs. CONCLUSIONS To stimulate learning, EPAs must be valued by both trainees and supervisors as constituting a coherent work-based curriculum that encompasses the key fellowship competencies. If EPAs are to be effective as clinical tasks for learning, ongoing faculty development must be the leading priority.
Collapse
Affiliation(s)
- Alice Stephan
- Mental Health and Addictions Service, Waikato District Health Board, Hamilton, New Zealand
| | - Gary Cheung
- School of Medicine, Faculty of Medical and Health Sciences, The University of Auckland, Auckland, New Zealand.
| | - Cees van der Vleuten
- School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, University of Maastricht, Maastricht, Netherlands
| |
Collapse
|
6
|
Stefan P, Pfandler M, Kullmann A, Eck U, Koch A, Mehren C, von der Heide A, Weidert S, Fürmetz J, Euler E, Lazarovici M, Navab N, Weigl M. Computer-assisted simulated workplace-based assessment in surgery: application of the universal framework of intraoperative performance within a mixed-reality simulation. BMJ SURGERY, INTERVENTIONS, & HEALTH TECHNOLOGIES 2023; 5:e000135. [PMID: 36687799 PMCID: PMC9853221 DOI: 10.1136/bmjsit-2022-000135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2022] [Accepted: 08/24/2022] [Indexed: 01/20/2023] Open
Abstract
Objectives Workplace-based assessment (WBA) is a key requirement of competency-based medical education in postgraduate surgical education. Although simulated workplace-based assessment (SWBA) has been proposed to complement WBA, it is insufficiently adopted in surgical education. In particular, approaches to criterion-referenced and automated assessment of intraoperative surgical competency in contextualized SWBA settings are missing.Main objectives were (1) application of the universal framework of intraoperative performance and exemplary adaptation to spine surgery (vertebroplasty); (2) development of computer-assisted assessment based on criterion-referenced metrics; and (3) implementation in contextualized, team-based operating room (OR) simulation, and evaluation of validity. Design Multistage development and assessment study: (1) expert-based definition of performance indicators based on framework's performance domains; (2) development of respective assessment metrics based on preoperative planning and intraoperative performance data; (3) implementation in mixed-reality OR simulation and assessment of surgeons operating in a confederate team. Statistical analyses included internal consistency and interdomain associations, correlations with experience, and technical and non-technical performances. Setting Surgical simulation center. Full surgical team set-up within mixed-reality OR simulation. Participants Eleven surgeons were recruited from two teaching hospitals. Eligibility criteria included surgical specialists in orthopedic, trauma, or neurosurgery with prior VP or kyphoplasty experience. Main outcome measures Computer-assisted assessment of surgeons' intraoperative performance. Results Performance scores were associated with surgeons' experience, observational assessment (Objective Structured Assessment of Technical Skill) scores and overall pass/fail ratings. Results provide strong evidence for validity of our computer-assisted SWBA approach. Diverse indicators of surgeons' technical and non-technical performances could be quantified and captured. Conclusions This study is the first to investigate computer-assisted assessment based on a competency framework in authentic, contextualized team-based OR simulation. Our approach discriminates surgical competency across the domains of intraoperative performance. It advances previous automated assessment based on the use of current surgical simulators in decontextualized settings. Our findings inform future use of computer-assisted multidomain competency assessments of surgeons using SWBA approaches.
Collapse
Affiliation(s)
- Philipp Stefan
- Chair for Computer Aided Medical Procedures and Augmented Reality, Department of Informatics, Technical University of Munich, München, Germany
| | - Michael Pfandler
- Institute and Outpatient Clinic for Occupational, Social, and Environmental Medicine, University Hospital, Ludwig Maximilians University Munich, München, Germany
| | - Aljoscha Kullmann
- Chair for Computer Aided Medical Procedures and Augmented Reality, Department of Informatics, Technical University of Munich, München, Germany
| | - Ulrich Eck
- Chair for Computer Aided Medical Procedures and Augmented Reality, Department of Informatics, Technical University of Munich, München, Germany
| | - Amelie Koch
- Institute and Outpatient Clinic for Occupational, Social, and Environmental Medicine, University Hospital, Ludwig Maximilians University Munich, München, Germany
| | - Christoph Mehren
- Spine Center, Schön Klinik München Harlaching, München, Germany,Academic Teaching Hospital and Spine Research Institute, Paracelsus Medical University, Salzburg, Austria
| | - Anna von der Heide
- Department of General, Trauma and Reconstructive Surgery, University Hospital, Campus Grosshadern, Ludwig Maximilians University Munich, München, Germany
| | - Simon Weidert
- Department of General, Trauma and Reconstructive Surgery, University Hospital, Campus Grosshadern, Ludwig Maximilians University Munich, München, Germany
| | - Julian Fürmetz
- Department of General, Trauma and Reconstructive Surgery, University Hospital, Campus Innenstadt, Ludwig Maximilians University Munich, München, Germany
| | - Ekkehard Euler
- Department of General, Trauma and Reconstructive Surgery, University Hospital, Campus Innenstadt, Ludwig Maximilians University Munich, München, Germany
| | - Marc Lazarovici
- Institute for Emergency Medicine and Management in Medicine (INM), University Hospital, Ludwig Maximilians University Munich, München, Germany
| | - Nassir Navab
- Chair for Computer Aided Medical Procedures and Augmented Reality, Department of Informatics, Technical University of Munich, München, Germany
| | - Matthias Weigl
- Institute and Outpatient Clinic for Occupational, Social, and Environmental Medicine, University Hospital, Ludwig Maximilians University Munich, München, Germany,Institute for Patient Safety, University of Bonn, Bonn, Germany
| |
Collapse
|
7
|
Branfield Day L, Rassos J, Billick M, Ginsburg S. 'Next steps are…': An exploration of coaching and feedback language in EPA assessment comments. MEDICAL TEACHER 2022; 44:1368-1375. [PMID: 35944554 DOI: 10.1080/0142159x.2022.2098098] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
PURPOSE Entrustable Professional Activities (EPA) assessments are intended to facilitate meaningful, low-stakes coaching and feedback, partly through the provision of written comments. We sought to explore EPA assessment comments provided to internal medicine (IM) residents for evidence of feedback and coaching language as well as politeness. METHODS We collected all written comments from EPA assessments of communication from a first-year IM resident cohort at the University of Toronto. Sensitized by politeness theory, we analyzed data using principles of constructivist grounded theory. RESULTS Nearly all EPA assessments (94%) contained written feedback based on focused clinical encounters. The majority of comments demonstrated coaching language, including phrases like 'don't forget to,' and 'next steps are,' followed by specific suggestions for improvement. A variety of words, including 'autonomy' and 'independence' denoted entrustment decisions. Linguistic politeness strategies such as hedging were pervasive, seemingly to minimize harm to the supervisor-trainee relationship. CONCLUSION Evidence of written coaching feedback suggests that EPA assessment comments are being used as intended as a means of formative feedback to promote learning. Yet, the frequent use of polite language suggests that EPAs may be higher-stakes than expected, highlighting a need for changes to the assessment culture and improved feedback literacy.
Collapse
Affiliation(s)
- Leora Branfield Day
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
| | - James Rassos
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
| | - Maxime Billick
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
| | - Shiphra Ginsburg
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
- Wilson Centre for Research in Education, Toronto, Canada
| |
Collapse
|
8
|
Schumacher DJ, Teunissen PW, Kinnear B, Driessen EW. Assessing trainee performance: ensuring learner control, supporting development, and maximizing assessment moments. Eur J Pediatr 2022; 181:435-439. [PMID: 34286373 DOI: 10.1007/s00431-021-04182-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/01/2021] [Revised: 06/22/2021] [Accepted: 06/23/2021] [Indexed: 11/28/2022]
Abstract
In this article, the authors provide practical guidance for frontline supervisors' efforts to assess trainee performance. They focus on three areas. First, they argue the importance of promoting learner control in the assessment process, noting that providing learners agency and control can shift the stakes of assessment from high to low and promote a safe environment that facilitates learning. Second, they posit that assessment should be used to support continued development by promoting a relational partnership between trainees and supervisors. This partnership allows supervisors to reinforce desirable aspects of performance, provide real-time support for deficient areas of performance, and sequence learning with the appropriate amount of scaffolding to push trainees from competence (what they can do alone) to capability (what they are able to do with support). Finally, they advocate the importance of optimizing the use of written comments and direct observation while also recognizing that performance is interdependent in efforts to maximize assessment moments.Conclusion: Using best practices in trainee assessment can help trainees take next steps in their development in a learner-centered partnership with clinical supervisors. What is Known: • Many pediatricians are asked to assess the performance of medical students and residents they work with but few have received formal training in assessment. What is New: • This article presents evidence-based best practices for assessing trainees, including giving trainees agency in the assessment process and focusing on helping trainees take next steps in their development.
Collapse
Affiliation(s)
- Daniel J Schumacher
- Division of Emergency Medicine, Cincinnati Children's Hospital Medical Center, and Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA.
| | - Pim W Teunissen
- School of Health Professions Education (SHE), Faculty of Health Medicine and Life Sciences and Gynecologist, Department of Obstetrics and Gynecology, Maastricht University Medical Center, Maastricht, the Netherlands
| | - Benjamin Kinnear
- Internal Medicine and Pediatrics, Division of Hospital Medicine, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Erik W Driessen
- School of Health Professions Education (SHE), Faculty of Health Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| |
Collapse
|
9
|
Anderson HL, Kurtz J, West DC. Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S164-S174. [PMID: 34406132 DOI: 10.1097/acm.0000000000004366] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
PURPOSE Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions. METHOD The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders. RESULTS In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain. CONCLUSIONS Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.
Collapse
Affiliation(s)
- Hannah L Anderson
- H.L. Anderson is research associate, Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania; ORCID: http://orcid.org/0000-0002-9435-1535
| | - Joshua Kurtz
- J. Kurtz is a first-year resident, Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | - Daniel C West
- D.C. West is professor of pediatrics, The Perelman School of Medicine at the University of Pennsylvania, and associate chair for education and senior director of medical education, Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania; ORCID: http://orcid.org/0000-0002-0909-4213
| |
Collapse
|
10
|
Rietmeijer CBT, Blankenstein AH, Huisman D, van der Horst HE, Kramer AWM, de Vries H, Scheele F, Teunissen PW. What happens under the flag of direct observation, and how that matters: A qualitative study in general practice residency. MEDICAL TEACHER 2021; 43:937-944. [PMID: 33765396 DOI: 10.1080/0142159x.2021.1898572] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
INTRODUCTION In competency-based medical education, direct observation (DO) of residents' skills is scarce, notwithstanding its undisputed importance for credible feedback and assessment. A growing body of research is investigating this discrepancy. Strikingly, in this research, DO as a concrete educational activity tends to remain vague. In this study, we concretised DO of technical skills in postgraduate longitudinal training relationships. METHODS Informed by constructivist grounded theory, we performed a focus group study among general practice residents. We asked residents about their experiences with different manifestations of DO of technical skills. A framework describing different DO patterns with their varied impact on learning and the training relationship was constructed and refined until theoretical sufficiency was reached. RESULTS The dominant DO pattern was ad hoc, one-way DO. Importantly, in this pattern, various unpredictable, and sometimes unwanted, scenarios could occur. Residents hesitated to discuss unwanted scenarios with their supervisors, sometimes instead refraining from future requests for DO or even for help. Planned bi-directional DO sessions, though seldom practiced, contributed much to collaborative learning in a psychologically safe training relationship. DISCUSSION AND CONCLUSION Patterns matter in DO. Residents and supervisors should be made aware of this and educated in maintaining an open dialogue on how to use DO for the benefit of learning and the training relationship.
Collapse
Affiliation(s)
- Chris B T Rietmeijer
- Department of General Practice, Location VUmc, Amsterdam University Medical Center, Amsterdam, The Netherlands
| | - Annette H Blankenstein
- Department of General Practice, Location VUmc, Amsterdam University Medical Center, Amsterdam, The Netherlands
| | - Daniëlle Huisman
- Department of General Practice, Location VUmc, Amsterdam University Medical Center, Amsterdam, The Netherlands
| | - Henriëtte E van der Horst
- Department of General Practice, Location VUmc, Amsterdam University Medical Center, Amsterdam, The Netherlands
| | - Anneke W M Kramer
- Department of Public Health and Primary Care, Leiden University, Leiden, The Netherlands
| | - Henk de Vries
- Department of General Practice, Location VUmc, Amsterdam University Medical Center, Amsterdam, The Netherlands
| | - Fedde Scheele
- School of Medical Sciences, Amsterdam University Medical Center, Location VUmc, Athena Institute for Transdisciplinary Research, VU University, Amsterdam, The Netherlands
| | - Pim W Teunissen
- School of Health Professions Education, Maastricht University, Universiteitssingel 60, Maastricht, The Netherlands
| |
Collapse
|
11
|
Holm EA, Al-Bayati SJL, Barfod TS, Lembeck MA, Pedersen H, Ramberg E, Klemmensen ÅK, Sorensen JL. Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study. BMJ Open 2021; 11:e047019. [PMID: 34321296 PMCID: PMC8319975 DOI: 10.1136/bmjopen-2020-047019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
Abstract
OBJECTIVES To examine a narrative multisource feedback (MSF) instrument concerning feasibility, quality of narrative comments, perceptions of users (face validity), consequential validity, discriminating capacity and number of assessors needed. DESIGN Qualitative text analysis supplemented by quantitative descriptive analysis. SETTING Internal Medicine Departments in Zealand, Denmark. PARTICIPANTS 48 postgraduate trainees in internal medicine specialties, 1 clinical supervisor for each trainee and 376 feedback givers (respondents). INTERVENTION This study examines the use of an electronic, purely narrative MSF instrument. After the MSF process, the trainee and the supervisor answered a postquestionnaire concerning their perception of the process. The authors coded the comments in the MSF reports for valence (positive or negative), specificity, relation to behaviour and whether the comment suggested a strategy for improvement. Four of the authors independently classified the MSF reports as either 'no reasons for concern' or 'possibly some concern', thereby examining discriminating capacity. Through iterative readings, the authors furthermore tried to identify how many respondents were needed in order to get a reliable impression of a trainee. RESULTS Out of all comments coded for valence (n=1935), 89% were positive and 11% negative. Out of all coded comments (n=4684), 3.8% were suggesting ways to improve. 92% of trainees and supervisors preferred a narrative MSF to a numerical MSF, and 82% of the trainees discovered performance in need of development, but only 53% had made a specific plan for development. Kappa coefficients for inter-rater correlations between four authors were 0.7-1. There was a significant association (p<0.001) between the number of negative comments and the qualitative judgement by the four authors. It was not possible to define a specific number of respondents needed. CONCLUSIONS A purely narrative MSF contributes with educational value and experienced supervisors can discriminate between trainees' performances based on the MSF reports.
Collapse
Affiliation(s)
- Ellen Astrid Holm
- Department of Internal Medicine, Zealand University Hospital Koge, Koge, Denmark
- Institute of Clinical Medicine, University of Copenhagen, Kobenhavns, Denmark
| | | | - Toke Seierøe Barfod
- Department of Internal Medicine, Zealand University Hospital Roskilde, Roskilde, Denmark
| | - Maurice A Lembeck
- Department of Internal Medicine, Nykobing F Sygehus, Nykobing Falster, Denmark
| | - Hanne Pedersen
- Department of Internal Medicine, Glostrup, Rigshospitalet, Kobenhavn, Denmark
| | - Emilie Ramberg
- Department of Internal Medicine, Nykobing F Sygehus, Nykobing Falster, Denmark
| | | | - Jette Led Sorensen
- Juliane Marie Centre for Children, Women and Reproduction Section 4074, Rigshospitalet, Kobenhavn, Denmark
- Children Hospital Copenhagen, Rigshospitalet, Kobenhavn, Denmark
| |
Collapse
|
12
|
Violato C, Cullen MJ, Englander R, Murray KE, Hobday PM, Borman-Shoap E, Ersan O. Validity Evidence for Assessing Entrustable Professional Activities During Undergraduate Medical Education. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S70-S75. [PMID: 34183605 DOI: 10.1097/acm.0000000000004090] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
PURPOSE To explore validity evidence for the use of entrustable professional activities (EPAs) as an assessment framework in medical education. METHOD Formative assessments on the 13 Core EPAs for entering residency were collected for 4 cohorts of students over a 9- to 12-month longitudinal integrated clerkship as part of the Education in Pediatrics Across the Continuum pilot at the University of Minnesota Medical School. The students requested assessments from clinical supervisors based on direct observation while engaging in patient care together. Based on each observation, the faculty member rated the student on a 9-point scale corresponding to levels of supervision required. Six EPAs were included in the present analyses. Student ratings were depicted as curves describing their performance over time; regression models were employed to fit the curves. The unit of analyses for the learning curves was observations rather than individual students. RESULTS (1) Frequent assessments on EPAs provided a developmental picture of competence consistent with the negative exponential learning curve theory; (2) This finding was true across a variety of EPAs and across students; and (3) The time to attain the threshold level of performance on the EPA for entrustment varied by student and EPA. CONCLUSIONS The results provide validity evidence for an EPA-based program of assessment. Students assessed using multiple observations performing the Core EPAs for entering residency demonstrate classic developmental progression toward the desired level of competence resulting in entrustment decisions. Future work with larger data samples will allow further psychometric analyses of assessment of EPAs.
Collapse
Affiliation(s)
- Claudio Violato
- C. Violato is assistant dean for assessment and evaluation and professor of internal medicine, University of Minnesota Medical School, Minneapolis, Minnesota
| | - Michael J Cullen
- M.J. Cullen is director of evaluation, Graduate Medical Education, University of Minnesota Medical School, Minneapolis, Minnesota
| | - Robert Englander
- R. Englander is associate dean for undergraduate medical education and professor of pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota
| | - Katherine E Murray
- K.E. Murray is assistant dean for curriculum and assistant professor of pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota
| | - Patricia M Hobday
- P.M. Hobday is director, Education in Pediatrics Across the Continuum pilot and assistant professor of pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota
| | - Emily Borman-Shoap
- E. Borman-Shoap is vice chair of education, director, pediatric residency program, and associate professor of pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota
| | - Ozge Ersan
- O. Ersan is a PhD candidate, Department of Educational Psychology, University of Minnesota, Minneapolis, Minnesota
| |
Collapse
|
13
|
Johansen RF, Nielsen RB, Malling BV, Storm H. Can case-based discussions in a group setting be used to assess residents' clinical skills? INTERNATIONAL JOURNAL OF MEDICAL EDUCATION 2021; 12:64-73. [PMID: 33840646 PMCID: PMC8411343 DOI: 10.5116/ijme.606a.eb39] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Accepted: 04/05/2021] [Indexed: 06/12/2023]
Abstract
OBJECTIVES The purpose of this study was to explore residents' and assessors' perception of a new group assessment concept. METHODS This qualitative study consists of observations of four group assessment sessions, followed by semi-structured interviews with six residents and four assessors (specialists in internal medicine), who all volunteered to be interviewed. All residents at a medical department (eleven to fifteen each time) and four assessors participated in four group assessments, where the residents' clinical skills were assessed through case-based discussions. An external consultant (an anthropologist) performed the observations and the interviews. Notes from the observations and the interviews were analyzed using an inductive approach. RESULTS Eight of the ten interviewed participants preferred group assessment to individual assessment. Results from the interviews suggested that the group assessments were more consistent and that the level of discussion was perceived to be higher in the group discussions compared to the one-to-one discussions. All residents indicated that they had acquired new knowledge during their assessment and reported having learned from listening to the assessment of their peers. Assessors similarly reported gaining new knowledge. CONCLUSIONS The residents and assessors expressed very favourable attitudes toward the new group assessment concept. The assessment process was perceived to be higher in quality and more consistent, contributing to learning for all participating doctors in the department. Group assessment is feasible and acceptable, and provides a promising tool for assessment of clinical skills in the future.
Collapse
Affiliation(s)
| | | | - Bente V. Malling
- Department of Clinical Medicine, Health, Aarhus University, Denmark
| | - Hanne Storm
- Diagnostic Center, Regional Hospital Silkeborg, Regional Hospital Central, Jutland, Denmark
| |
Collapse
|
14
|
Pagram H, Bilszta JLC, Szabo RA. Defining competency for Royal Australian and New Zealand College of Obstetricians and Gynaecologists training: An exploratory study of Victorian Integrated Training Program coordinators' understanding of competency. Aust N Z J Obstet Gynaecol 2021; 61:454-462. [PMID: 33772747 DOI: 10.1111/ajo.13324] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Revised: 01/24/2021] [Accepted: 01/26/2021] [Indexed: 11/29/2022]
Abstract
BACKGROUND Competency-based medical education (CBME) is increasingly employed by postgraduate training programs worldwide, including obstetrics and gynaecology. Focusing on assessment of outcomes rather than time-in-training, and utilising a well-defined curricular framework, CBME aims to train doctors capable of meeting the needs of modern society. When this study was undertaken, in 2019, the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) had a time-based curriculum and was due to undergo a curriculum review starting in 2020. AIMS To explore Victorian RANZCOG Integrated Training Program (ITP) coordinators' understanding of the concept of competency and how it is taught and assessed within RANZCOG training. MATERIALS AND METHODS A qualitative, grounded theory design using semi-structured interviews was employed. Victorian RANZCOG ITP coordinators from inner and outer metropolitan, and regional sites, were approached to participate. Transcripts were coded and analysed using thematic analysis. RESULTS Themes identified were: Competence, Vision and Innovation, Structures, ITP Coordinator Role and Teaching and Learning. Competence was defined as a combination of independent practice and understanding of ones' own limits, in addition to required clinical skills and knowledge. Enablers and barriers to achieving competency were identified and associated with structures, human and logistical factors. Victorian ITP coordinators believed the current training program has positive elements but could be further improved. CONCLUSIONS Several areas for future research were identified regarding understanding of competency, relevant if RANZCOG is to introduce a CBME framework. Replicating this research across all RANZCOG jurisdictions in Australia and New Zealand would be prudent to determine if the themes are universal.
Collapse
Affiliation(s)
| | - Justin L C Bilszta
- Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Victoria, Australia
| | - Rebecca A Szabo
- Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Victoria, Australia.,Department of Obstetrics and Gynaecology, Melbourne Medical School, University of Melbourne, Melbourne, Victoria, Australia.,Gandel Simulation Service, The Royal Women's Hospital, Melbourne, Victoria, Australia
| |
Collapse
|
15
|
Lillevang G, Ibsen H, Prins SH, Kjaer NK. How to enhance and assess reflection in specialist training: a mixed method validation study of a new tool for global assessment of reflection ability. BMC MEDICAL EDUCATION 2020; 20:352. [PMID: 33032573 PMCID: PMC7545892 DOI: 10.1186/s12909-020-02256-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/19/2020] [Accepted: 09/24/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND In Danish GP training we had the ambition to enhance and assess global reflection ability, but since we found no appropriate validated method in the literature, we decided to develop a new assessment tool. This tool is based on individual trainee developed mind maps and structured trainer-trainee discussions related to specific complex competencies. We named the tool Global Assessment of Reflection ability (GAR) and conducted a mixed method validation study. Our goal was to investigate whether it is possible to enhance and assess reflection ability using the tool. METHODS In order to investigate acceptability, feasibility, face validity, and construct validity of the tool we conducted a mixed method validation study that combined 1) qualitative data obtained from 750 GP trainers participating in train-the-trainer courses, 2) a questionnaire survey sent to 349 GP trainers and 214 GP trainees and 3) a thorough analysis of eight trainer-trainee discussions. RESULTS Our study showed an immediate high acceptance of the GAR tool. Both trainers and trainees found the tool feasible, useful, and relevant with acceptable face validity. Rating of eight audio recordings showed that the tool can demonstrate reflection during assessment of complex competencies. CONCLUSIONS We have developed an assessment tool (GAR) to enhance and assess reflection. GAR was found to be acceptable, feasible, relevant and with good face- and construct validity. GAR seems to be able to enhance the trainees' ability to reflect and provide a good basis for assessment in relation to complex competencies.
Collapse
Affiliation(s)
- Gunver Lillevang
- Department of Clinical Medicine, University of Copenhagen, Blegdamsvej 3B, 2200, Copenhagen N, Denmark
| | - Helle Ibsen
- The Research Unit of General Practice, Department of Public Health, Faculty of Health Sciences, University of Southern Denmark, Campusvej 55, 5230, Odense M, Denmark.
| | - Søren Hast Prins
- Centre for Health Sciences Education, Health, Aarhus University, Palle Juul-Jensens Boulevard 82, 8200, Aarhus N, Denmark
| | - Niels Kristian Kjaer
- The Research Unit of General Practice, Department of Public Health, Faculty of Health Sciences, University of Southern Denmark, Campusvej 55, 5230, Odense M, Denmark
| |
Collapse
|
16
|
Schumacher DJ, Holmboe E, Carraccio C, Martini A, van der Vleuten C, Busari J, Sobolewski B, Byczkowski TL. Resident-Sensitive Quality Measures in the Pediatric Emergency Department: Exploring Relationships With Supervisor Entrustment and Patient Acuity and Complexity. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:1256-1264. [PMID: 32101934 DOI: 10.1097/acm.0000000000003242] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
PURPOSE This study explores the associations between resident-sensitive quality measures (RSQMs) and supervisor entrustment as well as between RSQMs and patient acuity and complexity for encounters in the pediatric emergency department (PED) in which residents are caring for patients. METHOD Pediatric residents rotating through Cincinnati Children's Hospital Medical Center PED as well as supervising pediatric emergency medicine faculty and fellows were recruited during the 2017-2018 academic year for the purpose of collecting the following data from the residents' patient encounters for 3 illnesses (acute asthma exacerbation, bronchiolitis, and closed head injury [CHI]): supervisor entrustment decision rating, RSQMs relevant to the care provided, and supervisor patient acuity and complexity ratings. To measure the association of RSQM composite scores with the other variables of interest, mixed models were used. RESULTS A total of 83 residents cared for 110 patients with asthma, 112 with bronchiolitis, and 77 with CHI. Entrustment decision ratings were positively associated with asthma RSQM composite scores (beta coefficient = 0.03; P < .001). There was no significant association between RSQM composite scores and entrustment decision ratings for bronchiolitis or CHI. RSQM composite scores were significantly higher when acuity was also higher and significantly lower when acuity was also lower for both asthma (P < .001) and bronchiolitis (P = .01). However, RSQM composite scores were almost identical between levels of acuity for CHI (P = .94). There were no significant differences in RSQM composite scores when complexity varied. CONCLUSION This study found limited associations between RSQM composite scores and entrustment decision ratings but offers insight into how RSQMs could be used for the purposes of resident assessment and feedback.
Collapse
Affiliation(s)
- Daniel J Schumacher
- D.J. Schumacher is associate professor of pediatrics, Cincinnati Children's Hospital Medical Center and University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - Eric Holmboe
- E. Holmboe is senior vice president for milestones development and evaluation, Accreditation Council for Graduate Medical Education, Chicago, Illinois
| | - Carol Carraccio
- C. Carraccio is vice president of competency-based assessment, American Board of Pediatrics, Chapel Hill, North Carolina
| | - Abigail Martini
- A. Martini is a clinical research coordinator, Division of Emergency Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
| | - Cees van der Vleuten
- C. van der Vleuten is professor of education, Department of Educational Development and Research, Faculty of Health, Medicine, and Life Sciences, and scientific director, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands
| | - Jamiu Busari
- J. Busari is associate professor of medical education, Maastricht University, Maastricht, The Netherlands
| | - Brad Sobolewski
- B. Sobolewski is associate professor of pediatrics, Cincinnati Children's Hospital Medical Center and University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - Terri L Byczkowski
- T.L. Byczkowski is associate professor of pediatrics, Cincinnati Children's Hospital Medical Center and University of Cincinnati College of Medicine, Cincinnati, Ohio
| |
Collapse
|
17
|
Burm S, Sebok-Syer SS, Van Koughnett JA, Watling CJ. Are we generating more assessments without added value? Surgical trainees' perceptions of and receptiveness to cross-specialty assessment. PERSPECTIVES ON MEDICAL EDUCATION 2020; 9:201-209. [PMID: 32504448 PMCID: PMC7459015 DOI: 10.1007/s40037-020-00594-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
INTRODUCTION Competency-based medical education (CBME) hinges on robust assessment. However, integrating regular workplace-based assessment within demanding and sometimes chaotic clinical environments remains challenging. Many faculty lack assessment expertise, and some programs lack the infrastructure and faculty numbers to fulfill CBME's mandate. Recognizing this, we designed and implemented an assessment innovation that trains and deploys a cadre of faculty to assess in specialties outside their own. Specifically, we explored trainees' perceptions of and receptiveness to this novel assessment approach. METHODS Within Western University's Surgical Foundations program, 27 PGY‑1 trainees were formatively assessed by trained non-surgeons on a basic laparoscopic surgical skill. These assessments did not impact trainees' progression. Four focus groups were conducted to gauge residents' sentiments about the experience of cross-specialty assessment. Data were then analyzed using a thematic analysis approach. RESULTS While a few trainees found the experience motivating, more often trainees questioned the feedback they received and the practicality of this assessment approach to advance their procedural skill acquisition. What trainees wanted were strategies for improvement, not merely an assessment of performance. DISCUSSION Trainees' trepidation at the idea of using outside assessors to meet increased assessment demands appeared grounded in their expectations for assessment. What trainees appeared to desire was a coach-someone who could break their performance into its critical individual components-as opposed to an assessor whose role was limited to scoring their performance. Understanding trainees' receptivity to new assessment approaches is crucial; otherwise training programs run the risk of generating more assessments without added value.
Collapse
Affiliation(s)
- Sarah Burm
- Division of Medical Education, Dalhousie University, Halifax, Nova Scotia, Canada.
| | | | - Julie Ann Van Koughnett
- Divisions of General Surgery and Surgical Oncology, Western University, London, Ontario, Canada
| | - Christopher J Watling
- Departments of Oncology and Clinical Neurological Sciences, Centre for Education Research and Innovation, Western University, London, Ontario, Canada
| |
Collapse
|
18
|
Holbrook AI, Kasales C. Advancing Competency-Based Medical Education Through Assessment and Feedback in Breast Imaging. Acad Radiol 2020; 27:442-446. [PMID: 31147238 DOI: 10.1016/j.acra.2019.04.017] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2019] [Revised: 03/26/2019] [Accepted: 04/02/2019] [Indexed: 10/26/2022]
Abstract
Competency-based medical education (CBME) is a method of educating and assessing trainees that focuses on outcomes, rather than process. In this review, we inform radiologists involved in breast imaging training on the tenets of CBME and its relationship to the milestones, feedback and assessment. We also describe multiple methods for assessment specific to the breast imaging curriculum, and techniques for improving feedback to trainees in breast imaging.
Collapse
|
19
|
Tonni I, Gadbury-Amyot CC, Govaerts M, Ten Cate O, Davis J, Garcia LT, Valachovic RW. ADEA-ADEE Shaping the Future of Dental Education III: Assessment in competency-based dental education: Ways forward. J Dent Educ 2020; 84:97-104. [PMID: 31977092 DOI: 10.1002/jdd.12024] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Accepted: 12/04/2019] [Indexed: 11/10/2022]
Abstract
Assessment in competency-based dental education continues to be a recognized area for growth and development within dental programs around the world. At the joint American Dental Education Association (ADEA) and the Association for Dental Education in Europe (ADEE) 2019 conference, Shaping the Future of Dental Education III, the workshop on assessment was designed to continue the discussion started in 2017 at the ADEA-ADEE Shaping the Future of Dental Education II.1 The focus of the 2019 conference involved examining the potential of entrustable professional activities (EPAs) and current thinking about workplace-based assessment (WBA) within competency-based education in the 21st century. Approximately 30 years ago, George Miller wrote about the assessment of competence in medical education and challenged faculty to reach for higher levels of assessment than knowledge or skill.2 Acknowledging that no one assessment method can result in a valid assessment of competence, Miller proposed a four-level framework for assessment. The lowest level involves measuring what students know ("knows"), followed by assessment of the skill with which knowledge is applied in relevant tasks or problems ("knows how"). Next is an assessment of task performance in standardized settings ("shows how"), and finally, the highest level assesses the student's performance in the unstandardized clinical workplace ("does"). The 2019 assessment workshop focused on advances in the assessment of learners in the unstandardized workplace-the highest level of Miller's assessment pyramid ("does"). Research has shown that dental education has struggled to implement assessment strategies that meet this level.3 The workshop brought together individuals from around the world, with an interest in assessment in dental education, to consider how assessment in the "does" level, specifically EPAs and WBA, factors into competence assessment in dentistry/dental education.
Collapse
Affiliation(s)
| | | | - Marjan Govaerts
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences (FHML), Maastricht University
| | | | - Joan Davis
- Special Projects & Initiatives, A.T. Still Missouri School of Dentistry & Oral Health
| | - Lily T Garcia
- University of Nevada, Las Vegas, School of Dental Medicine
| | | |
Collapse
|
20
|
Thiessen N, Fischer MR, Huwendiek S. Assessment methods in medical specialist assessments in the DACH region - overview, critical examination and recommendations for further development. GMS JOURNAL FOR MEDICAL EDUCATION 2019; 36:Doc78. [PMID: 31844650 PMCID: PMC6905366 DOI: 10.3205/zma001286] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 07/29/2018] [Revised: 07/29/2019] [Accepted: 09/04/2019] [Indexed: 06/01/2023]
Abstract
Introduction: Specialist medical assessments fulfil the task of ensuring that physicians have the clinical competence to independently represent their field and provide the best possible care to patients, taking into account the current state of knowledge. To date, there are no comprehensive reports on the status of specialist assessments in the German-speaking countries (DACH). For that reason, the assessment methods used in the DACH region are compiled and critically evaluated in this article, and recommendations for further development are described. Methods: The websites of the following institutions were searched for information regarding testing methods used and the organisation of specialist examinations: Homepage of the Swiss Institute for Medical Continuing Education (SIWF), Homepage of the Academy of Physicians (Austria) and Homepage of the German Federal Medical Association (BAEK). Further links were considered and the results were presented in tabular form. The assessment methods used in the specialist assessments are critically examined with regard to established quality criteria and recommendations for the further development of the specialist assessments are derived from these. Results: The following assessment methods are already used in Switzerland and Austria: written examinations with multiple choice and short answer questions, structured oral examinations, the Script Concordance Test (SCT) and the Objective Structured Clinical Examination (OSCE). In some cases, these assessment methods are combined (triangulation). In Germany, on the other hand, the oral examination has so far been conducted in an unstructured manner in the form of a 'collegial content discussion'. In order to test knowledge, practical and communicative competences equally, it is recommended to implement a triangulation of methods and follow the further recommendations described in this article. Conclusion: While there are already accepted approaches for quality-assured and competence-based specialist assessments in Switzerland and Austria at present, there is still a long way to go in Germany. Following the recommendations presented in this article, a contribution could be made to improving the specialist assessments in the DACH region according to the specialist assessments objectives.
Collapse
Affiliation(s)
- Nils Thiessen
- EDU - a degree smarter, Digital Education Holdings Ltd., Kalkara, Republic of Malta
| | - Martin R. Fischer
- LMU München, Klinikum der Universität München, Institut für Didaktik und Ausbildungsforschung in der Medizin, München, Germany
| | - Sören Huwendiek
- Universität Bern, Institut für Medizinische Lehre, Abteilung für Assessment und Evaluation, Bern, Switzerland
| |
Collapse
|
21
|
van Bockel EAP, Walstock PA, van Mook WNKA, Arbous MS, Tepaske R, van Hemel TJD, Müller MCA, Delwig H, Tulleken JE. Entrustable professional activities (EPAs) for postgraduate competency based intensive care medicine training in the Netherlands: The next step towards excellence in intensive care medicine training. J Crit Care 2019; 54:261-267. [PMID: 31733630 DOI: 10.1016/j.jcrc.2019.09.012] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2018] [Revised: 07/30/2019] [Accepted: 09/11/2019] [Indexed: 11/15/2022]
Abstract
INTRODUCTION The Competency Based Training in Intensive Care Education (CoBaTrICE) programme developed common standards of ICM training by describing competencies of an intensivist. Entrustable Professional Activities (EPAs) of Intensive Care Medicine (ICM) (EPAsICM) are presented as a new workplace-based assessment tool in competency-based training of intensivists. EPAs are activities to be entrusted to a trainee once he (or she) has attained competence. EPAs emphasise the role of trust between trainees and supervisors. EPAs bridge the gap between competencies and competence. METHODS An expert panel of ICM (vice)programme directors and intensivists in The Netherlands integrated the CoBaTrICE and CanMEDS competencies into EPAsICM. Comment and feedback was sought from other ICM programme directors and educational experts and processed in the final version of EPAsICM before implementation in the Dutch ICM training programme. RESULTS A list of 15 EPAsICM are considered to reflect the spectrum of clinical practice while incorporating the competencies of CoBaTrICE and CanMEDS. The grading system is designed as a 5-point entrustment scale based on the amount of supervision a trainee needs, aligning with daily judgement of trainees by intensivists. CONCLUSION EPAsICM is an assessment tool that formalises entrustment decisions and can be a valuable addition in international ICM training.
Collapse
Affiliation(s)
- Esther A P van Bockel
- Department of Critical Care, University Medical Center Groningen, University of Groningen, Hanzeplein 1, 9700, RB, Groningen, the Netherlands.
| | - Pieter A Walstock
- Department of Critical Care, University Medical Center Groningen, University of Groningen, Hanzeplein 1, 9700, RB, Groningen, the Netherlands
| | - Walther N K A van Mook
- Department of Intensive Care Medicine, Maastricht University Medical Center, P. Debyelaan 25, 6202, AZ, Maastricht, the Netherlands; School of Health Professions Education, Maastricht University, the Netherlands
| | - M Sesmu Arbous
- Department of Intensive Care Medicine, Leiden University Medical Center, Albinusdreef 2, 2333, ZA, Leiden, the Netherlands; Department of Clinical Epidemiology, Leiden University Medical Center, Albinusdreef 2, 2333, ZA, Leiden, the Netherlands
| | - Robert Tepaske
- Amsterdam UMC, University of Amsterdam, Department of Intensive Care Medicine, Meibergdreef 9, 1105, AZ, Amsterdam, the Netherlands
| | - Tina J D van Hemel
- Department of Intensive Care Medicine, Leiden University Medical Center, Albinusdreef 2, 2333, ZA, Leiden, the Netherlands
| | - Marcella C A Müller
- Amsterdam UMC, University of Amsterdam, Department of Intensive Care Medicine, Meibergdreef 9, 1105, AZ, Amsterdam, the Netherlands
| | - Hans Delwig
- Department of Critical Care, University Medical Center Groningen, University of Groningen, Hanzeplein 1, 9700, RB, Groningen, the Netherlands
| | - Jaap E Tulleken
- Department of Critical Care, University Medical Center Groningen, University of Groningen, Hanzeplein 1, 9700, RB, Groningen, the Netherlands
| |
Collapse
|
22
|
Meeuwissen SNE, Stalmeijer RE, Govaerts M. Multiple-role mentoring: mentors' conceptualisations, enactments and role conflicts. MEDICAL EDUCATION 2019; 53:605-615. [PMID: 30723949 PMCID: PMC6590242 DOI: 10.1111/medu.13811] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/19/2018] [Revised: 09/20/2018] [Accepted: 12/21/2018] [Indexed: 05/17/2023]
Abstract
INTRODUCTION Outcome-based approaches to education and the inherent emphasis on programmatic assessment in particular, require models of mentoring in which mentors fulfil dual roles: coach and assessor. Fulfilling multiple roles could result in role confusion or even role conflicts, both of which may affect mentoring processes and outcomes. In this study, we explored how mentors conceptualise and enact their role in a multiple-role mentoring system and to what extent they experience role conflicts. METHODS We conducted a constructivist grounded theory study at one undergraduate medical school. A purposive sample of 12 physician-mentors active in a programmatic assessment system was interviewed. Data analysis followed stages of open, axial and selective coding through which themes were constructed. RESULTS Three predominant mentoring approaches were constructed: (i) empowering (a reflective and holistic approach to student development); (ii) checking (an observant approach to check whether formal requirements are met), and (iii) directing (an authoritative approach to guide students' professional development). Each approach encompassed a corresponding type of mentor-mentee relationship: (i) partnership; (ii) instrumental, and (iii) faculty-centred. Furthermore, mentors' strategies, focus, agency provided to students and perception of the assessment system characterised mentoring approaches and relationships. Role conflicts were mainly experienced by mentors with a directing mentoring approach. They used various coping mechanisms, including deviation from assessment guidelines. CONCLUSIONS In multiple-role mentoring in the context of programmatic assessment, mentors adopted certain predominant mentoring approaches, which were characterised by different strategies for mentoring and resulted in different mentor-mentee relationships. Multiple-role mentoring does not necessarily result in role conflict. Mentors who do experience role conflict seem to favour the directing approach, which is most at odds with key principles of competency-based education and programmatic assessment. These findings build upon existing mentoring literature and offer practical suggestions for faculty development regarding approaches to mentoring in programmatic assessment systems.
Collapse
Affiliation(s)
- Stephanie N E Meeuwissen
- Department of Educational Development and ResearchFaculty of HealthMedicine and Life SciencesMaastricht UniversityMaastrichtthe Netherlands
| | - Renée E Stalmeijer
- Department of Educational Development and ResearchFaculty of HealthMedicine and Life SciencesMaastricht UniversityMaastrichtthe Netherlands
| | - Marjan Govaerts
- Department of Educational Development and ResearchFaculty of HealthMedicine and Life SciencesMaastricht UniversityMaastrichtthe Netherlands
| |
Collapse
|
23
|
Woolf K, Page M, Viney R. Assessing professional competence: a critical review of the Annual Review of Competence Progression. J R Soc Med 2019; 112:236-244. [PMID: 31124405 DOI: 10.1177/0141076819848113] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
The Annual Review of Competence Progression is used to determine whether trainee doctors in the United Kingdom are safe and competent to progress to the next training stage. In this article we provide evidence to inform recommendations to enhance the validity of the summative and formative elements of the Annual Review of Competency Progression. The work was commissioned as part of a Health Education England review. We systematic searched the peer reviewed and grey literature, synthesising findings with information from national, local and specialty-specific Annual Review of Competence Progression guidance, critically evaluating the findings in the context of literature on assessing competence in medical education. National guidance lacked detail resulting in variability across locations and specialties, threatening validity and reliability. Trainees and trainers were concerned that the Annual Review of Competence Progression only reliably identifies the most poorly performing trainees. Feedback is not routinely provided, which can leave those with performance difficulties unsupported and high performers demotivated. Variability in the provision and quality of feedback can negatively affect learning. The Annual Review of Competence Progression functions as a high-stakes assessment, likely to have a significant impact on patient care. It should be subject to the same rigorous evaluation as other high-stakes assessments; there should be consistency in procedures across locations, specialties and grades; and all trainees should receive high-quality feedback.
Collapse
Affiliation(s)
- Katherine Woolf
- Research Department of Medical Education, UCL Medical School, Royal Free Hospital, London NW3 2PF, UK
| | - Michael Page
- Research Department of Medical Education, UCL Medical School, Royal Free Hospital, London NW3 2PF, UK
| | - Rowena Viney
- Research Department of Medical Education, UCL Medical School, Royal Free Hospital, London NW3 2PF, UK
| |
Collapse
|
24
|
Armson H, Lockyer JM, Zetkulic M, Könings KD, Sargeant J. Identifying coaching skills to improve feedback use in postgraduate medical education. MEDICAL EDUCATION 2019; 53:477-493. [PMID: 30779210 DOI: 10.1111/medu.13818] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/26/2018] [Revised: 09/26/2018] [Accepted: 01/09/2019] [Indexed: 06/09/2023]
Abstract
OBJECTIVES Coaching in medical education has recently gained prominence, but minimal attention has been given to key skills and determining how they work to effectively ensure residents are progressing and developing self-assessment skills. This study examined process-oriented and content-oriented coaching skills used in coaching sessions, with particular attention to how supervisors use them to enhance resident acceptance of feedback to enhance learning. METHODS This qualitative study analysed secondary audiotaped data from 15 supervisors: resident dyads during two feedback sessions, 4 months apart. The R2C2 model was used to engage the resident, build a relationship, explore reactions to feedback, explore resident perceptions of content, and coach for change. Framework analysis was used, including familiarisation with the data, identifying the thematic framework, indexing and charting the data and mapping and interpretation. RESULTS Process skills included preparation, relationship development, using micro communication skills and techniques to promote reflection and self-assessment by the resident and supervisor flexibility. Content skills related to the specific feedback content included engaging the resident in discussion, ensuring the discussion was collaborative and focused on goal setting, co-developing a Learning Change Plan, ensuring resident commitment and following up on the plan. Together, these skills foster agency in the resident learner. Three overarching themes emerged from the analysis: the interconnectedness of process and content; tensions between encouraging self-direction and ensuring progress and competence; and balancing a coaching dialogue and a teaching monologue. CONCLUSIONS Effective coaching by supervisors requires a combination of specific process and content skills that are chosen depending on the needs of the individual resident. Mastering these skills helps residents engage and develop agency in their own professional development. These outcomes depend on faculty maintaining a balance between coaching and teaching, encouraging resident self-direction and ensuring progression to competence.
Collapse
Affiliation(s)
- Heather Armson
- Department of Family Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Jocelyn M Lockyer
- Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
| | - Marygrace Zetkulic
- Department of Medicine, Hackensack University Hospital, Hackensack, New Jersey, USA
| | - Karen D Könings
- Department of Educational Development & Research and School of Health Professions Education, Maastricht University, Maastricht, the Netherlands
| | - Joan Sargeant
- Continuing Medical Education and Division of Medical Education, Dalhousie University, Halifax, Nova Scotia, Canada
| |
Collapse
|
25
|
Prins SH, Brøndt SG, Malling B. Implementation of workplace-based assessment in general practice. EDUCATION FOR PRIMARY CARE 2019; 30:133-144. [PMID: 31018801 DOI: 10.1080/14739879.2019.1588788] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
Background: Workplace-based assessment (WPBA) is widely accepted, but few studies have investigated implementation issues during general practice (GP) placements. This study explored possible barriers and identified key elements for successful implementation of a WPBA-programme in Danish GP specialist training. Methods: Supervisors had attended a one-day course in WPBA and trainees had received a short introduction. Questionnaires on experiences with implementation of WPBA were distributed to 106 GP supervisors and 110 trainees after the rotation was finished. Results: The response rate was 61/96 (64%) for trainees and 67/94 (71%) for supervisors. Supervisors were generally more positive towards WPBA and saw fewer barriers than trainees. Lack of planning was most often reported as an impediment to WPBA. Supervisors did not identify trainees' uneasiness of being observed as a problem as often as trainees. A total of 34% of trainees reported uneasiness as an obstacle to WPBA. Conclusions: It seems that the education of supervisors positively influenced supervisors' perception and use of WPBA. Adequate planning of WPBA may be just as big a problem as assigning the time. Further investigations on the impact of education on trainees' perception of WPBA are needed.
Collapse
Affiliation(s)
- Søren Hast Prins
- a Centre for Health Sciences Education, Health, Aarhus University , Aarhus , Denmark
| | | | - Bente Malling
- a Centre for Health Sciences Education, Health, Aarhus University , Aarhus , Denmark
| |
Collapse
|
26
|
Lee V, Brain K, Martin J. From opening the 'black box' to looking behind the curtain: cognition and context in assessor-based judgements. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2019; 24:85-102. [PMID: 30302670 DOI: 10.1007/s10459-018-9851-0] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/25/2018] [Accepted: 09/06/2018] [Indexed: 06/08/2023]
Abstract
The increasing use of direct observation tools to assess routine performance has resulted in the growing reliance on assessor-based judgements in the workplace. However, we have a limited understanding of how assessors make judgements and formulate ratings in real world contexts. The current research on assessor cognition has largely focused on the cognitive domain but the contextual factors are equally important, and both are closely interconnected. This study aimed to explore the perceived cognitive and contextual factors influencing Mini-CEX assessor judgements in the Emergency Department setting. We used a conceptual framework of assessor-based judgement to develop a sequential mixed methods study. We analysed and integrated survey and focus group results to illustrate self-reported cognitive and contextual factors influencing assessor judgements. We used situated cognition theory as a sensitizing lens to explore the interactions between people and their environment. The major factors highlighted through our mixed methods study were: clarity of the assessment, reliance on and variable approach to overall impression (gestalt), role tension especially when giving constructive feedback, prior knowledge of the trainee and case complexity. We identified prevailing tensions between participants (assessors and trainees), interactions (assessment and feedback) and setting. The two practical implications of our research are the need to broaden assessor training to incorporate both cognitive and contextual domains, and the need to develop a more holistic understanding of assessor-based judgements in real world contexts to better inform future research and development in workplace-based assessments.
Collapse
Affiliation(s)
- Victor Lee
- Department of Emergency Medicine, Austin Health, P.O. Box 5555, Heidelberg, VIC, 3084, Australia.
| | | | - Jenepher Martin
- Eastern Health Clinical School, Monash University and Deakin University, Box Hill, VIC, Australia
| |
Collapse
|
27
|
Schumacher DJ, Martini A, Bartlett KW, King B, Calaman S, Garfunkel LC, Elliott SP, Frohna JG, Schwartz A, Michelson CD. Key Factors in Clinical Competency Committee Members' Decisions Regarding Residents' Readiness to Serve as Supervisors: A National Study. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2019; 94:251-258. [PMID: 30256253 DOI: 10.1097/acm.0000000000002469] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
PURPOSE Entrustment has become a popular assessment framework in recent years. Most research in this area has focused on how frontline assessors determine when a learner can be entrusted. However, less work has focused on how these entrustment decisions are made. The authors sought to understand the key factors that pediatric residency program clinical competency committee (CCC) members consider when recommending residents to a supervisory role. METHOD CCC members at 14 pediatric residency programs recommended residents to one of five progressive supervisory roles (from not serving as a supervisory resident to serving as a supervisory resident in all settings). They then responded to a free-text prompt, describing the key factors that led them to that decision. The authors analyzed these responses, by role recommendation, using a thematic analysis. RESULTS Of the 155 CCC members at the participating programs, 84 completed 769 supervisory role recommendations during the 2015-2016 academic year. Four themes emerged from the thematic analysis: (1) Determining supervisory ability follows from demonstrated trustworthiness; (2) demonstrated performance matters, but so does experience; (3) ability to lead a team is considered; and (4) contextual considerations external to the resident are at play. CONCLUSIONS CCC members considered resident and environmental factors in their summative entrustment decision making. The interplay between these factors should be considered as CCC processes are optimized and studied further.
Collapse
Affiliation(s)
- Daniel J Schumacher
- D.J. Schumacher is associate professor, Department of Pediatrics, Cincinnati Children's Hospital Medical Center and University of Cincinnati College of Medicine, Cincinnati, Ohio. A. Martini is clinical research coordinator, Division of Emergency Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio. K.W. Bartlett is associate professor and associate program director, Department of Pediatrics, Duke University School of Medicine, Durham, North Carolina. B. King is research project manager, Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network, McLean, Virginia. S. Calaman is associate professor and pediatric program director, Department of Pediatrics, Drexel University College of Medicine and St. Christopher's Hospital for Children, Philadelphia, Pennsylvania. L.C. Garfunkel is professor and associate program director, Department of Pediatrics, University of Rochester School of Medicine and Dentistry, Rochester, New York. S.P. Elliott is professor, associate chair, and program director, Department of Pediatrics, and interim associate dean, University of Arizona College of Medicine, Tucson, Arizona. J.G. Frohna is professor, Departments of Pediatrics and Internal Medicine, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin. A. Schwartz is Michael Reese Endowed Professor of Medical Education and associate head, Department of Medical Education, and research professor, Department of Pediatrics, University of Illinois College of Medicine, Chicago, Illinois, and director, Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network, McLean, Virginia. C.D. Michelson is assistant professor and pediatric program director, Department of Pediatrics, Boston University School of Medicine, Boston, Massachusetts
| | | | | | | | | | | | | | | | | | | |
Collapse
|
28
|
Melvin L, Rassos J, Panisko D, Driessen E, Kulasegaram KM, Kuper A. Overshadowed by Assessment: Understanding Trainee and Supervisor Perspectives on the Oral Case Presentation in Internal Medicine Workplace-Based Assessment. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2019; 94:244-250. [PMID: 30211754 DOI: 10.1097/acm.0000000000002451] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
PURPOSE The oral case presentation (OCP) is an essential part of daily clinical practice in internal medicine (IM) and a key competency in medical education. It is not known how supervisors and trainees perceive OCPs in workplace-based learning and assessment. METHOD Using a constructivist grounded theory approach, 26 semistructured interviews were held with trainees and supervisors (18 clinical clerks and first- through third-year postgraduate trainees, and 8 supervisors) on the IM clinical teaching unit at the University of Toronto, 2015-2016. Interviews focused on how the OCP was viewed by both trainees and supervisors in clinical practice as a tool for patient care, learning, and assessment. Iterative, constant comparative techniques were used to analyze the interviews and develop a framework to understand trainee and supervisor perspectives. RESULTS Supervisors and trainees viewed the OCP as an important part of informal trainee assessment in IM. Supervisors used OCPs to understand the patient through trainee-demonstrated skills including the use of narratives, information synthesis, and management of uncertainty. However, because of awareness of assessment, trainees sought to control the OCP, viewing it as a performance demonstrating their competence, mediated by senior residents and tailored to supervisor preferences. CONCLUSIONS Preoccupied with assessment around OCPs, trainees often lost sight of the valuable learning taking place. Use of OCPs in assessment necessitates optimization of the educational activity for trainees. Providing explicit direction to both trainees and supervisors, defining expectations, and clarifying the assessment activity of the OCP can optimize the encounter for best educational practice.
Collapse
Affiliation(s)
- Lindsay Melvin
- L. Melvin is assistant professor, Department of Medicine, University of Toronto, and staff physician, Division of General Internal Medicine, University Health Network, Toronto, Ontario, Canada. J. Rassos is a fifth-year resident, general internal medicine subspecialty training program, Department of Medicine, University of Toronto, Toronto, Ontario, Canada. D. Panisko is professor, Department of Medicine, University of Toronto, and staff physician, Division of General Internal Medicine, University Health Network, Toronto, Ontario, Canada. E. Driessen is professor of medical education and chair, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands. K.M. Kulasegaram is scientist and assistant professor, Department of Family and Community Medicine, Wilson Centre, University Health Network and Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada. A. Kuper is associate professor, Department of Medicine, scientist and associate director, Wilson Centre for Research in Education, University Health Network, University of Toronto, and staff physician, Division of General Internal Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| | | | | | | | | | | |
Collapse
|
29
|
Harrison C. Can we redesign the MRCGP assessment to support lifelong learning? EDUCATION FOR PRIMARY CARE 2019; 30:9-12. [DOI: 10.1080/14739879.2018.1563507] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
30
|
Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. MEDICAL EDUCATION 2019; 53:76-85. [PMID: 30073692 DOI: 10.1111/medu.13645] [Citation(s) in RCA: 175] [Impact Index Per Article: 35.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/10/2018] [Revised: 04/04/2018] [Accepted: 05/24/2018] [Indexed: 05/25/2023]
Abstract
CONTEXT Models of sound assessment practices increasingly emphasise assessment's formative role. As a result, assessment must not only support sound judgements about learner competence, but also generate meaningful feedback to guide learning. Reconciling the tension between assessment's focus on judgement and decision making and feedback's focus on growth and development represents a critical challenge for researchers and educators. METHODS We synthesise the literature related to this tension, framed around four trends in education research: (i) shifting perspectives on assessment; (ii) shifting perspectives on feedback; (iii) increasing attention on learners' perceptions of assessment and feedback, and (iv) increasing attention on the influence of culture on assessment and feedback. We describe factors that produce and sustain this tension. RESULTS The lines between assessment and feedback frequently blur in medical education. Models of programmatic assessment deliberately use the same data for both purposes: low-stakes individual data points are used formatively, but then are added together to support summative judgements. However, the translation of theory to practice is not straightforward. Efforts to embed meaningful feedback in programmes of learning face a multitude of threats. Learners may perceive assessment with formative intent as summative, restricting their engagement with it as feedback, and thus diminishing its learning value. A learning culture focused on assessment may limit learners' sense of safety to explore, to experiment, and sometimes to fail. CONCLUSIONS Successfully blending assessment and feedback demands clarity of purpose, support for learners, and a system and organisational commitment to a culture of improvement rather than a culture of performance.
Collapse
Affiliation(s)
- Christopher J Watling
- Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Shiphra Ginsburg
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
31
|
Govaerts MJB, van der Vleuten CPM, Holmboe ES. Managing tensions in assessment: moving beyond either-or thinking. MEDICAL EDUCATION 2019; 53:64-75. [PMID: 30289171 PMCID: PMC6586064 DOI: 10.1111/medu.13656] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/13/2018] [Revised: 04/16/2018] [Accepted: 06/08/2018] [Indexed: 05/09/2023]
Abstract
CONTEXT In health professions education, assessment systems are bound to be rife with tensions as they must fulfil formative and summative assessment purposes, be efficient and effective, and meet the needs of learners and education institutes, as well as those of patients and health care organisations. The way we respond to these tensions determines the fate of assessment practices and reform. In this study, we argue that traditional 'fix-the-problem' approaches (i.e. either-or solutions) are generally inadequate and that we need alternative strategies to help us further understand, accept and actually engage with the multiple recurring tensions in assessment programmes. METHODS Drawing from research in organisation science and health care, we outline how the Polarity Thinking™ model and its 'both-and' approach offer ways to systematically leverage assessment tensions as opportunities to drive improvement, rather than as intractable problems. In reviewing the assessment literature, we highlight and discuss exemplars of specific assessment polarities and tensions in educational settings. Using key concepts and principles of the Polarity Thinking™ model, and two examples of common tensions in assessment design, we describe how the model can be applied in a stepwise approach to the management of key polarities in assessment. DISCUSSION Assessment polarities and tensions are likely to surface with the continued rise of complexity and change in education and health care organisations. With increasing pressures of accountability in times of stretched resources, assessment tensions and dilemmas will become more pronounced. We propose to add to our repertoire of strategies for managing key dilemmas in education and assessment design through the adoption of the polarity framework. Its 'both-and' approach may advance our efforts to transform assessment systems to meet complex 21st century education, health and health care needs.
Collapse
Affiliation(s)
- Marjan J B Govaerts
- Department of Educational Development and ResearchFaculty of Health, Medicine and Life SciencesMaastricht UniversityMaastrichtthe Netherlands
| | - Cees P M van der Vleuten
- Department of Educational Development and ResearchFaculty of Health, Medicine and Life SciencesMaastricht UniversityMaastrichtthe Netherlands
| | - Eric S Holmboe
- Accreditation Council for Graduate Medical EducationChicagoIllinoisUSA
| |
Collapse
|
32
|
Lefebvre C, Hiestand B, Glass C, Masneri D, Hosmer K, Hunt M, Hartman N. Examining the Effects of Narrative Commentary on Evaluators’ Summative Assessments of Resident Performance. Eval Health Prof 2018; 43:159-161. [DOI: 10.1177/0163278718820415] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
Anchor-based, end-of-shift ratings are commonly used to conduct performance assessments of resident physicians. These performance evaluations often include narrative assessments, such as solicited or “free-text” commentary. Although narrative commentary can help to create a more detailed and specific assessment of performance, there are limited data describing the effects of narrative commentary on the global assessment process. This single-group, observational study examined the effect of narrative comments on global performance assessments. A subgroup of the clinical competency committee, blinded to resident identity, assigned a single, consensus-based performance score (1–6) to each resident based solely on end-of-shift milestone scores. De-identified narrative comments from end-of-shift evaluations were then included and the process was repeated. We compared milestone-only scores to milestone plus narrative commentary scores using a nonparametric sign test. During the study period, 953 end-of-shift evaluations were submitted on 41 residents. Of these, 535 evaluations included free-text narrative comments. In 17 of the 41 observations, performance scores changed after the addition of narrative comments. In two cases, scores decreased with the addition of free-text commentary. In 15 cases, scores increased. The frequency of net positive change was significant ( p = .0023). The addition of narrative commentary to anchor-based ratings significantly influenced the global performance assessment of Emergency Medicine residents by a committee of educators. Descriptive commentary collected at the end of shift may inform more meaningful appraisal of a resident’s progress in a milestone-based paradigm. The authors recommend clinical training programs collect unstructured narrative impressions of residents’ performance from supervising faculty.
Collapse
Affiliation(s)
- Cedric Lefebvre
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Brian Hiestand
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Casey Glass
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - David Masneri
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Kathleen Hosmer
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Meagan Hunt
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Nicholas Hartman
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| |
Collapse
|
33
|
Hicks PJ, Margolis MJ, Carraccio CL, Clauser BE, Donnelly K, Fromme HB, Gifford KA, Poynter SE, Schumacher DJ, Schwartz A. A novel workplace-based assessment for competency-based decisions and learner feedback. MEDICAL TEACHER 2018; 40:1143-1150. [PMID: 29688108 DOI: 10.1080/0142159x.2018.1461204] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
BACKGROUND Increased recognition of the importance of competency-based education and assessment has led to the need for practical and reliable methods to assess relevant skills in the workplace. METHODS A novel milestone-based workplace assessment system was implemented in 15 pediatrics residency programs. The system provided: (1) web-based multisource feedback (MSF) and structured clinical observation (SCO) instruments that could be completed on any computer or mobile device; and (2) monthly feedback reports that included competency-level scores and recommendations for improvement. RESULTS For the final instruments, an average of five MSF and 3.7 SCO assessment instruments were completed for each of 292 interns; instruments required an average of 4-8 min to complete. Generalizability coefficients >0.80 were attainable with six MSF observations. Users indicated that the new system added value to their existing assessment program; the need to complete the local assessments in addition to the new assessments was identified as a burden of the overall process. CONCLUSIONS Outcomes - including high participation rates and high reliability compared to what has traditionally been found with workplace-based assessment - provide evidence for the validity of scores resulting from this novel competency-based assessment system. The development of this assessment model is generalizable to other specialties.
Collapse
Affiliation(s)
- Patricia J Hicks
- a Children's Hospital of Philadelphia and University of Pennsylvania , Philadelphia , PA , USA
| | | | | | - Brian E Clauser
- b National Board of Medical Examiners , Philadelphia , PA , USA
| | | | - H Barrett Fromme
- e Pritzker School of Medicine , University of Chicago , Chicago , IL , USA
| | | | - Sue E Poynter
- g Pediatrics , Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine , Cincinnati , OH , USA
| | - Daniel J Schumacher
- g Pediatrics , Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine , Cincinnati , OH , USA
| | - Alan Schwartz
- h Medical Education and Pediatrics , University of Illinois at Chicago and Association of Pediatric Program Directors , Chicago , IL , USA
| |
Collapse
|
34
|
Patel M, Baker P. Supervision for entrustable professional activities. MEDICAL EDUCATION 2018; 52:998-1000. [PMID: 30255530 DOI: 10.1111/medu.13685] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2023]
Affiliation(s)
- Mumtaz Patel
- Research and Development Department, Health Education England North West, Manchester, UK
| | - Paul Baker
- Research and Development Department, Health Education England North West, Manchester, UK
| |
Collapse
|
35
|
Dijkhuizen K, Bustraan J, de Beaufort AJ, Velthuis SI, Driessen EW, van Lith JMM. Encouraging residents' professional development and career planning: the role of a development-oriented performance assessment. BMC MEDICAL EDUCATION 2018; 18:207. [PMID: 30185174 PMCID: PMC6125996 DOI: 10.1186/s12909-018-1317-9] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/22/2017] [Accepted: 08/28/2018] [Indexed: 06/08/2023]
Abstract
BACKGROUND Current postgraduate medical training programmes fall short regarding residents' development of generic competencies (communication, collaboration, leadership, professionalism) and reflective and deliberate practice. Paying attention to these non-technical skills in a structural manner during postgraduate training could result in a workforce better prepared for practice. A development-oriented performance assessment (PA), which assists residents with assessment of performance and deliberately planned learning activities, could potentially contribute to filling this gap. This study aims to explore residents experiences with the PA. METHODS We conducted a qualitative interview study with 16 residents from four different medical specialties who participated in the PA, scheduled halfway postgraduate training. The PA was conducted by an external facilitator, a psychologist, and focused specifically on professional development and career planning. Residents were interviewed 6 months after the PA. Data were analysed using the framework method for qualitative analysis. RESULTS Residents found the PA to be of additional value for their training. The overarching merit was the opportunity to evaluate competencies not usually addressed in workplace-based assessments and progress conversations. In addition, the PA proved a valuable tool for assisting residents with reflecting upon their work and formulating their learning objectives and activities. Residents reported increased awareness of capacity, self-confidence and enhanced feelings of career-ownership. An important factor contributing to these outcomes was the relationship of trust with the facilitator and programme director. CONCLUSION The PA is a promising tool in fostering the development of generic competencies and reflective and deliberate practice. The participating residents, facilitator and programme directors were able to contribute to a safe learning environment away from the busy workplace. The facilitator plays an important role by providing credible and informative feedback. Commitment of the programme director is important for the implementation of developmental plans and learning activities.
Collapse
Affiliation(s)
- Kirsten Dijkhuizen
- Centre for Innovation in Medical Education, Leiden University Medical Centre, PO Box 9600, Zone V7-P, 2300 RC Leiden, the Netherlands
- Department of Obstetrics, Leiden University Medical Centre, PO Box 9600, Zone K6-P, 2300 RC Leiden, the Netherlands
| | - Jacqueline Bustraan
- Centre for Innovation in Medical Education, Leiden University Medical Centre, PO Box 9600, Zone V7-P, 2300 RC Leiden, the Netherlands
| | - Arnout J. de Beaufort
- Centre for Innovation in Medical Education, Leiden University Medical Centre, PO Box 9600, Zone V7-P, 2300 RC Leiden, the Netherlands
| | - Sophie I. Velthuis
- Centre for Innovation in Medical Education, Leiden University Medical Centre, PO Box 9600, Zone V7-P, 2300 RC Leiden, the Netherlands
| | - Erik W. Driessen
- Department of Educational Development & Research Maastricht University, Universiteitssingel 60, 6229 Maastricht, the Netherlands
| | - Jan M. M. van Lith
- Department of Obstetrics, Leiden University Medical Centre, PO Box 9600, Zone K6-P, 2300 RC Leiden, the Netherlands
| |
Collapse
|
36
|
Davies RM, Hadfield-Law L, Turner PG. Development and Evaluation of a New Formative Assessment of Surgical Performance. JOURNAL OF SURGICAL EDUCATION 2018; 75:1309-1316. [PMID: 29580814 DOI: 10.1016/j.jsurg.2018.02.004] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/17/2017] [Revised: 01/08/2018] [Accepted: 02/04/2018] [Indexed: 06/08/2023]
Abstract
BACKGROUND Formative assessment of operative performance is a mandatory part of surgical training. Engagement with assessment is limited in part by the time-consuming nature and the high perceived stakes of current assessment tools. OBJECTIVES Our aims were to develop and collect validity evidence for a new operative assessment tool that addresses barriers to assessment that current trainers and trainees experience. METHODS We developed the Generic Operative Supervised Learning Event (GOSLE). Orthopedic trainees were invited to complete GOSLEs with their trainers after surgical procedures. Experienced consultants assessed videotaped operations performed by trainees using the GOSLE. Validity evidence for content, relationships to other scores, internal structure, response process, and consequences of testing were evaluated. RESULTS A total of 250 GOSLEs were completed. A strong correlation was found between the GOSLE scores and the Procedure-Based Assessment ratings (r = 0.87, p < 0.001). Rasch analysis confirmed satisfactory internal structure of the rating scale, with sequential increases in rating as performance improved. The reproducibility coefficient was 0.88, with 10 assessments of the same trainee who has to achieve a reliability coefficient of 0.8. Over 90% of users found the GOSLE easy to use, with most preferring it to other assessment methods. Feedback quality was higher using the GOSLE than with current assessments. CONCLUSION We have collected validity evidence across multiple domains in support of the GOSLE. Its psychometric performance is comparable to that of current assessments. It is preferred by trainers and trainees over existing assessments. It stimulates high-quality, actionable feedback which better supports formative assessment. By addressing issues experienced with existing assessments, we expect engagement among users to be high.
Collapse
Affiliation(s)
- Ronnie M Davies
- Department of Orthopaedics, Central Manchester University Hospitals NHS Foundation Trust, Manchester, United Kingdom.
| | - Lisa Hadfield-Law
- British Orthopaedic Association, Royal College of Surgeons, London, United Kingdom
| | - Philip G Turner
- Department of Orthopaedics, Stepping Hill Hospital, Stockport, United Kingdom
| |
Collapse
|
37
|
Rietmeijer CBT, Huisman D, Blankenstein AH, de Vries H, Scheele F, Kramer AWM, Teunissen PW. Patterns of direct observation and their impact during residency: general practice supervisors' views. MEDICAL EDUCATION 2018; 52:981-991. [PMID: 30043397 PMCID: PMC6120450 DOI: 10.1111/medu.13631] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2018] [Revised: 03/06/2018] [Accepted: 04/27/2018] [Indexed: 05/17/2023]
Abstract
CONTEXT Direct observation (DO) of residents' performance, despite the importance that is ascribed to it, does not readily fit in with the practice of postgraduate medical education (PGME); it is infrequent and the quality of observation may be poor in spite of ongoing efforts towards improvement. In recent literature, DO is mostly portrayed as a means to gather information on the performance of residents for purposes of feedback and assessment. The role of DO in PGME is likely to be more complex and poorly understood in the era of outcome-based education. By exploring the possible complexity of DO in workplace learning, our research aims to contribute to a better use of DO in the practice of PGME. METHODS Constructivist grounded theory informed our data collection and analysis. Data collection involved focus group sessions with supervisors in Dutch general practice who were invited to discuss the manifestations, meanings and effects of DO of technical skills. Theoretical sufficiency was achieved after four focus groups, with a total of 28 participants being included. RESULTS We found four patterns of DO of technical skills: initial planned DO sessions; resident-initiated ad hoc DO; supervisor-initiated ad hoc DO, and continued planned DO sessions. Different patterns of DO related to varying meanings, such as checking or trusting, and effects, such as learning a new skill or experiencing emotional discomfort, all of them concerning the training relationship, patient safety or residents' learning. CONCLUSIONS Direct observation, to supervisors, means much more than gathering information for purposes of feedback and assessment. Planned DO sessions are an important routine during the initiation phase of a training relationship. Continued planned bidirectional DO sessions, although infrequently practised, potentially combine most benefits with least side-effects of DO. Ad hoc DO, although much relied upon, is often hampered by internal tensions in supervisors, residents or both.
Collapse
Affiliation(s)
- Chris B T Rietmeijer
- Department of General Practice and Elderly Care MedicineVU University Medical CentreAmsterdamThe Netherlands
| | - Daniëlle Huisman
- Department of General Practice and Elderly Care MedicineVU University Medical CentreAmsterdamThe Netherlands
| | - Annette H Blankenstein
- Department of General Practice and Elderly Care MedicineVU University Medical CentreAmsterdamThe Netherlands
| | - Henk de Vries
- Department of General Practice and Elderly Care MedicineVU University Medical CentreAmsterdamThe Netherlands
| | - Fedde Scheele
- School of Medical SciencesVU University Medical CentreAmsterdamThe Netherlands
- Athena Institute for Transdisciplinary ResearchVU UniversityAmsterdamThe Netherlands
| | - Anneke W M Kramer
- Department of Public Health and Primary CareLeiden University Medical CentreLeidenThe Netherlands
| | - Pim W Teunissen
- School of Health Professions EducationMaastricht UniversityMaastrichtThe Netherlands
| |
Collapse
|
38
|
Hemming P, Levine RB, Gallo JJ. "Conversational Advice": A mixed-methods analysis of medical residents' experiences co-managing primary care patients with behavioral health providers. PATIENT EDUCATION AND COUNSELING 2018; 101:85-91. [PMID: 28734557 DOI: 10.1016/j.pec.2017.07.014] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/13/2017] [Revised: 07/11/2017] [Accepted: 07/12/2017] [Indexed: 06/07/2023]
Abstract
OBJECTIVE When integrated behavioral health clinicians (IBHCs) and residents co-manage patients, residents may learn new approaches. We aimed to understand the effect of co-management on residents' behavioral health (BH) management learning. METHODS Residents completed a web-based survey enquiring: whether co-management included a shared visit and/or face-to-face meeting with an IBHC, whether residents received feedback from the IBHC, and what they learned. Qualitative responses were coded thematically using a constant comparative method. RESULTS Among 117 respondents (overall response rate 72%, 117/163), from five residencies recruited from 40 residencies with BH integration, residents were significantly more likely to receive feedback if they had a shared visit with the patient and an IBHC (yes 69% vs. no 33%; adjusted OR 3.0, 95% CI 1.2-7.6). Residents reported three major learning themes: interpersonal communication skills awareness, BH skills awareness, and newly adopted attitudes toward BH. Residents who received feedback were more likely to report themes of interpersonal communication skills awareness (yes 26.6% vs. no 9.4%). CONCLUSION BH integration promotes increased feedback for residents practicing face-to-face co-management with IBHCs, and a positive influence regarding residents' attitudes and perceived skills. PRACTICAL IMPLICATIONS Residency programs can meaningfully improve residents' learning by promoting face-to-face co-management with IBHCs.
Collapse
Affiliation(s)
- Patrick Hemming
- Department of Medicine, Division of General Internal Medicine, Duke University School of Medicine, Durham, NC, USA.
| | - R B Levine
- Division of General Internal Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Joseph J Gallo
- Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| |
Collapse
|
39
|
Entrustables and Entrustment: Through the Looking Glass at the Clinical Making of a Nurse Practitioner. J Nurse Pract 2017. [DOI: 10.1016/j.nurpra.2017.05.018] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
40
|
Favreau MA, Tewksbury L, Lupi C, Cutrer WB, Jokela JA, Yarris LM. Constructing a Shared Mental Model for Faculty Development for the Core Entrustable Professional Activities for Entering Residency. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2017; 92:759-764. [PMID: 28557935 DOI: 10.1097/acm.0000000000001511] [Citation(s) in RCA: 37] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
In 2014, the Association of American Medical Colleges identified 13 Core Entrustable Professional Activities for Entering Residency (Core EPAs), which are activities that entering residents might be expected to perform without direct supervision. This work included the creation of an interinstitutional concept group focused on faculty development efforts, as the processes and tools for teaching and assessing entrustability in undergraduate medical education (UME) are still evolving. In this article, the authors describe a conceptual framework for entrustment that they developed to better prepare all educators involved in entrustment decision making in UME. This framework applies to faculty with limited or longitudinal contact with medical students and to those who contribute to entrustment development or render summative entrustment decisions.The authors describe a shared mental model for entrustment that they developed, based on a critical synthesis of the EPA literature, to serve as a guide for UME faculty development efforts. This model includes four dimensions for Core EPA faculty development: (1) observation skills in authentic settings (workplace-based assessments), (2) coaching and feedback skills, (3) self-assessment and reflection skills, and (4) peer guidance skills developed through a community of practice. These dimensions form a conceptual foundation for meaningful faculty participation in entrustment decision making.The authors also differentiate between the UME learning environment and the graduate medical education learning environment to highlight distinct challenges and opportunities for faculty development in UME settings. They conclude with recommendations and research questions for future Core EPA faculty development efforts.
Collapse
Affiliation(s)
- Michele A Favreau
- M.A. Favreau is associate professor of pediatrics, and adjunct associate professor, Division of Management, Oregon Health and Science University School of Medicine, Portland, Oregon. She was also associate dean for professional development and lifelong learning, Oregon Health and Science University School of Medicine, Portland, Oregon, at the time this work was done. L. Tewksbury is associate dean for student affairs and associate professor of pediatrics, New York University School of Medicine, New York, New York. C. Lupi is assistant dean for learning and teaching and professor of obstetrics and gynecology, Florida International University Herbert Wertheim College of Medicine, Miami, Florida. W.B. Cutrer is assistant professor of pediatrics, Vanderbilt University School of Medicine, Nashville, Tennessee. J.A. Jokela is professor and head, Department of Medicine, University of Illinois College of Medicine at Urbana-Champaign, Urbana, Illinois. L.M. Yarris is associate professor of emergency medicine and program director for emergency medicine, Oregon Health and Science University School of Medicine, Portland, Oregon
| | | | | | | | | | | |
Collapse
|
41
|
Lee V, Brain K, Martin J. Factors Influencing Mini-CEX Rater Judgments and Their Practical Implications: A Systematic Literature Review. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2017; 92:880-887. [PMID: 28030422 DOI: 10.1097/acm.0000000000001537] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
PURPOSE At present, little is known about how mini-clinical evaluation exercise (mini-CEX) raters translate their observations into judgments and ratings. The authors of this systematic literature review aim both to identify the factors influencing mini-CEX rater judgments in the medical education setting and to translate these findings into practical implications for clinician assessors. METHOD The authors searched for internal and external factors influencing mini-CEX rater judgments in the medical education setting from 1980 to 2015 using the Ovid MEDLINE, PsycINFO, ERIC, PubMed, and Scopus databases. They extracted the following information from each study: country of origin, educational level, study design and setting, type of observation, occurrence of rater training, provision of feedback to the trainee, research question, and identified factors influencing rater judgments. The authors also conducted a quality assessment for each study. RESULTS Seventeen articles met the inclusion criteria. The authors identified both internal and external factors that influence mini-CEX rater judgments. They subcategorized the internal factors into intrinsic rater factors, judgment-making factors (conceptualization, interpretation, attention, and impressions), and scoring factors (scoring integration and domain differentiation). CONCLUSIONS The current theories of rater-based judgment have not helped clinicians resolve the issues of rater idiosyncrasy, bias, gestalt, and conflicting contextual factors; therefore, the authors believe the most important solution is to increase the justification of rater judgments through the use of specific narrative and contextual comments, which are more informative for trainees. Finally, more real-world research is required to bridge the gap between the theory and practice of rater cognition.
Collapse
Affiliation(s)
- Victor Lee
- V. Lee is codirector of emergency medicine training, Department of Emergency Medicine, Austin Health, Heidelberg, Victoria, Australia.K. Brain is doctor, South West Healthcare, Warrnambool, Victoria, Australia.J. Martin is associate professor and director, Medical Student Programs, Monash University and Deakin University, Eastern Health Clinical School, Box Hill, Victoria, Australia
| | | | | |
Collapse
|
42
|
Sargeant J, Mann K, Manos S, Epstein I, Warren A, Shearer C, Boudreau M. R2C2 in Action: Testing an Evidence-Based Model to Facilitate Feedback and Coaching in Residency. J Grad Med Educ 2017; 9:165-170. [PMID: 28439347 PMCID: PMC5398131 DOI: 10.4300/jgme-d-16-00398.1] [Citation(s) in RCA: 55] [Impact Index Per Article: 7.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/05/2016] [Revised: 11/02/2016] [Accepted: 11/25/2016] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND Feedback is increasingly seen as a collaborative conversation between supervisors and learners, where learners are actively and reflectively engaged with feedback and use it to improve. Based on this, and through earlier research, we developed an evidence- and theory-informed, 4-phase model for facilitating feedback and practice improvement-the R2C2 model (relationship, reaction, content, coaching). OBJECTIVE Our goal was to explore the utility and acceptability of the R2C2 model in residency education, specifically for engaging residents in their feedback and in using it to improve, as well as the factors influencing its use. METHODS This qualitative study used the principles of design research. We recruited residents and their supervisors in 2 programs, internal medicine and pediatrics. We prepared supervisors to use the R2C2 model during their regular midrotation and/or end-of-rotation feedback sessions with participating residents to discuss their progress and assessment reports. We conducted debriefing interviews with supervisors and residents after each session. We analyzed transcripts as a team using template and content analysis. RESULTS Of 61 residents, 7 residents (11%) participated with their supervisors (n = 5). Schedules and sensitivity to feedback prevented broader enrollment. Supervisors found the structured R2C2 format useful. Residents and supervisors reported that the coaching phase was novel and helpful, and that the R2C2 model engaged both groups in collaborative, reflective, goal-oriented feedback discussions. CONCLUSIONS Participants found that using the R2C2 model enabled meaningful feedback conversations, identification of goals for improvement, and development of strategies to meet those goals.
Collapse
|
43
|
Apramian T, Cristancho S, Watling C, Ott M, Lingard L. "Staying in the Game": How Procedural Variation Shapes Competence Judgments in Surgical Education. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2016; 91:S37-S43. [PMID: 27779508 PMCID: PMC5578755 DOI: 10.1097/acm.0000000000001364] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/17/2023]
Abstract
PURPOSE Emerging research explores the educational implications of practice and procedural variation between faculty members. The potential effect of these variations on how surgeons make competence judgments about residents has not yet been thoroughly theorized. The authors explored how thresholds of principle and preference shaped surgeons' intraoperative judgments of resident competence. METHOD This grounded theory study included reanalysis of data on the educational role of procedural variations and additional sampling to attend to their impact on assessment. Reanalyzed data included 245 hours of observation across 101 surgical cases performed by 29 participants (17 surgeons, 12 residents), 39 semistructured interviews (33 with surgeons, 6 with residents), and 33 field interviews with residents. The new data collected to explore emerging findings related to assessment included two semistructured interviews and nine focused field interviews with residents. Data analysis used constant comparison to refine the framework and data collection process until theoretical saturation was reached. RESULTS The core category of the study, called staying in the game, describes how surgeons make moment-to-moment judgments to allow residents to retain their role as operators. Surgeons emphasized the role of principles in making these decisions, while residents suggested that working with surgeons' preferences also played an important role in such intraoperative assessment. CONCLUSIONS These findings suggest that surgeons' and residents' work with thresholds of principle and preference have significant implications for competence judgments. Making use of these judgments by turning to situated assessment may help account for the subjectivity in assessment fostered by faculty variations.
Collapse
Affiliation(s)
- Tavis Apramian
- T. Apramian is MD candidate and centre fellow, Centre for Education Research & Innovation, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada. S. Cristancho is assistant professor, Department of Surgery, and scientist, Centre for Education Research & Innovation, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada. C. Watling is associate dean, Postgraduate Medical Education, and scientist, Centre for Education Research & Innovation, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada. M. Ott is associate professor, Department of Surgery, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada. L. Lingard is professor, Department of Medicine, and director, Centre for Education Research & Innovation, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada
| | | | | | | | | |
Collapse
|
44
|
Warm EJ, Held JD, Hellmann M, Kelleher M, Kinnear B, Lee C, O'Toole JK, Mathis B, Mueller C, Sall D, Tolentino J, Schauer DP. Entrusting Observable Practice Activities and Milestones Over the 36 Months of an Internal Medicine Residency. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2016; 91:1398-1405. [PMID: 27355780 DOI: 10.1097/acm.0000000000001292] [Citation(s) in RCA: 65] [Impact Index Per Article: 8.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
PURPOSE Competency-based medical education and milestone reporting have led to increased interest in work-based assessments using entrustment over time as an assessment framework. Little is known about data collected from these assessments during residency. This study describes the results of entrustment of discrete work-based skills over 36 months in the University of Cincinnati internal medicine (IM) residency program. METHOD Attending physician and peer/allied health assessors provided entrustment ratings of resident performance on work-based observable practice activities (OPAs) mapped to Accreditation Council for Graduate Medicine Education/American Board of Internal Medicine reporting milestones for IM. These data were translated into milestones data and tracked longitudinally. The authors analyzed data from this new entrustment system's first 36 months (July 2012-June 2015). RESULTS During the 36-month period, assessors made 364,728 milestone assessments (mapped from OPAs) of 189 residents. Residents received an annualized average of 83 assessment encounters, producing means of 3,987 milestone assessments and 4,325 words of narrative assessment. Mean entrustment ratings (range 1-5) from all assessors for all milestones rose from 2.46 for first-month residents to 3.92 for 36th-month residents (r = 0.9252, P < .001). Attending physicians' entrustment ratings were lower than peer/allied health assessors' ratings. Medical knowledge and patient care milestones were rated lower than professionalism and interpersonal and communication skills milestones. CONCLUSIONS Entrustment of milestones appears to rise progressively over time, with differences by assessor type, competency, milestone, and resident. Further research is needed to elucidate the validity of these data in promotion, remediation, and reporting decisions.
Collapse
Affiliation(s)
- Eric J Warm
- E.J. Warm is Sue P. and Richard W. Vilter Professor of Medicine and categorical medicine program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.J.D. Held is assistant professor and associate program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.M. Hellmann is a fellow in pulmonary/critical care, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.M. Kelleher is assistant professor and fellow in medical education, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.B. Kinnear is assistant professor and assistant program director, Medicine-Pediatrics, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.C. Lee is assistant professor and associate program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.J.K. O'Toole is assistant professor and associate program director, Medicine-Pediatrics, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.B. Mathis is associate professor, associate chair for clinical affairs, and associate program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.C. Mueller is professor and program director, Medicine-Pediatrics, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.D. Sall is assistant professor and fellow in medical education, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.J. Tolentino is visiting associate professor and program director, Medicine-Pediatrics, Stony Brook University School of Medicine, Stony Brook, New York.D.P. Schauer is associate professor and associate program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio
| | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
45
|
Castanelli DJ, Jowsey T, Chen Y, Weller JM. Perceptions of purpose, value, and process of the mini-Clinical Evaluation Exercise in anesthesia training. Can J Anaesth 2016; 63:1345-1356. [DOI: 10.1007/s12630-016-0740-9] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2016] [Revised: 08/15/2016] [Accepted: 09/13/2016] [Indexed: 10/21/2022] Open
|
46
|
Fostering Dental Students’ Academic Achievements and Reflection Skills Through Clinical Peer Assessment and Feedback. J Dent Educ 2016. [DOI: 10.1002/j.0022-0337.2016.80.8.tb06171.x] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
47
|
Barrett A, Galvin R, Steinert Y, Scherpbier A, O'Shaughnessy A, Walsh G, Horgan M. Profiling postgraduate workplace-based assessment implementation in Ireland: a retrospective cohort study. SPRINGERPLUS 2016; 5:133. [PMID: 26933632 PMCID: PMC4761346 DOI: 10.1186/s40064-016-1748-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/06/2015] [Accepted: 02/09/2016] [Indexed: 11/16/2022]
Abstract
In 2010, workplace-based assessment (WBA) was formally integrated as a method of formative trainee assessment into 29 basic and higher specialist medical training (BST/HST) programmes in six postgraduate training bodies in Ireland. The aim of this study is to explore how WBA is being implemented and to examine if WBA is being used formatively as originally intended. A retrospective cohort study was conducted and approved by the institution’s Research Ethics Committee. A profile of WBA requirements was obtained from 29 training programme curricula. A data extraction tool was developed to extract anonymous data, including written feedback and timing of assessments, from Year 1 and 2 trainee ePortfolios in 2012–2013. Data were independently quality assessed and compared to the reference standard number of assessments mandated annually where relevant. All 29 training programmes mandated the inclusion of at least one case-based discussion (max = 5; range 1–5). All except two non-clinical programmes (93 %) required at least two mini-Clinical Evaluation Exercise assessments per year and Direct Observation of Procedural Skills assessments were mandated in 27 training programmes over the course of the programme. WBA data were extracted from 50 % of randomly selected BST ePortfolios in four programmes (n = 142) and 70 % of HST ePortfolios (n = 115) in 21 programmes registered for 2012–2013. Four programmes did not have an eligible trainee for that academic year. In total, 1142 WBAs were analysed. A total of 164 trainees (63.8 %) had completed at least one WBA. The average number of WBAs completed by HST trainees was 7.75 (SD 5.8; 95 % CI 6.5–8.9; range 1–34). BST trainees completed an average of 6.1 assessments (SD 9.3; 95 % CI 4.01–8.19; range 1–76). Feedback—of varied length and quality—was provided on 44.9 % of assessments. The majority of WBAs were completed in the second half of the year. There is significant heterogeneity with respect to the frequency and quality of feedback provided during WBAs. The completion of WBAs later in the year may limit available time for feedback, performance improvement and re-evaluation. This study sets the scene for further work to explore the value of formative assessment in postgraduate medical education.
Collapse
Affiliation(s)
- Aileen Barrett
- Education and Professional Development Unit, Royal College of Physicians of Ireland, Frederick House, 19 South Frederick St, Dublin 2, Ireland ; School of Medicine, College of Medicine and Health Sciences, Brookfield Health Sciences Complex, University College Cork, Cork, Ireland
| | - Rose Galvin
- Discipline of Physiotherapy, Department of Clinical Therapies, Faculty of Education and Health Sciences, University of Limerick, Limerick, Ireland
| | - Yvonne Steinert
- Centre for Medical Education, Faculty of Medicine, McGill University, Lady Meredith House, 1110 Pine Avenue West, Montreal, QC H3A 1A3 Canada
| | - Albert Scherpbier
- Faculty of Health, Medicine and Life Sciences, University of Maastricht, Universiteitssingel 60, 6229 ER Maastricht, The Netherlands
| | - Ann O'Shaughnessy
- Education and Professional Development Unit, Royal College of Physicians of Ireland, Frederick House, 19 South Frederick St, Dublin 2, Ireland
| | - Gillian Walsh
- Education and Professional Development Unit, Royal College of Physicians of Ireland, Frederick House, 19 South Frederick St, Dublin 2, Ireland
| | - Mary Horgan
- School of Medicine, College of Medicine and Health Sciences, Brookfield Health Sciences Complex, University College Cork, Cork, Ireland
| |
Collapse
|
48
|
Rekman J, Gofton W, Dudek N, Gofton T, Hamstra SJ. Entrustability Scales: Outlining Their Usefulness for Competency-Based Clinical Assessment. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2016; 91:186-90. [PMID: 26630609 DOI: 10.1097/acm.0000000000001045] [Citation(s) in RCA: 169] [Impact Index Per Article: 21.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2023]
Abstract
Meaningful residency education occurs at the bedside, along with opportunities for situated in-training assessment. A necessary component of workplace-based assessment (WBA) is the clinical supervisor, whose subjective judgments of residents' performance can yield rich and nuanced ratings but may also occasionally reflect bias. How to improve the validity of WBA instruments while simultaneously capturing meaningful subjective judgment is currently not clear. This Perspective outlines how "entrustability scales" may help bridge the gap between the assessment judgments of clinical supervisors and WBA instruments. Entrustment-based assessment evaluates trainees against what they will actually do when independent; thus, "entrustability scales"-defined as behaviorally anchored ordinal scales based on progression to competence-reflect a judgment that has clinical meaning for assessors. Rather than asking raters to assess trainees against abstract scales, entrustability scales provide raters with an assessment measure structured around the way evaluators already make day-to-day clinical entrustment decisions, which results in increased reliability. Entrustability scales help raters make assessments based on narrative descriptors that reflect real-world judgments, drawing attention to a trainee's readiness for independent practice rather than his/her deficiencies. These scales fit into milestone measurement both by allowing an individual resident to strive for independence in entrustable professional activities across the entire training period and by allowing residency directors to identify residents experiencing difficulty. Some WBA tools that have begun to use variations of entrustability scales show potential for allowing raters to produce valid judgments. This type of anchor scale should be brought into wider circulation.
Collapse
Affiliation(s)
- Janelle Rekman
- J. Rekman is a general surgery resident and master's in health professions education student, University of Ottawa, Ottawa, Ontario, Canada. W. Gofton is an orthopedic surgeon, University of Ottawa, Ottawa, Ontario, Canada. N. Dudek is associate professor, Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada. T. Gofton is Wissenschaftlicher Mitarbeiter, Department of Philosophy, Eberhard Karls Universität, Tübingen, Germany. S.J. Hamstra is vice president, Milestones Research and Evaluation, Accreditation Council for Graduate Medical Education, Chicago, Illinois
| | | | | | | | | |
Collapse
|
49
|
Toprak A, Luhanga U, Jones S, Winthrop A, McEwen L. Validation of a novel intraoperative assessment tool: The Surgical Procedure Feedback Rubric. Am J Surg 2016; 211:369-76. [DOI: 10.1016/j.amjsurg.2015.08.032] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2015] [Revised: 07/25/2015] [Accepted: 08/19/2015] [Indexed: 11/29/2022]
|
50
|
Daelmans HE, Mak-van der Vossen MC, Croiset G, Kusurkar RA. What difficulties do faculty members face when conducting workplace-based assessments in undergraduate clerkships? INTERNATIONAL JOURNAL OF MEDICAL EDUCATION 2016; 7:19-24. [PMID: 26803256 PMCID: PMC4724428 DOI: 10.5116/ijme.5689.3c7f] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/30/2015] [Accepted: 01/03/2016] [Indexed: 05/14/2023]
Abstract
OBJECTIVE Workplace-based assessments are based on the principle of providing feedback to medical students on clinical performance in authentic settings. In practice, however, the assessment often overshadows the feedback. The aim of this study was to determine what problems faculty perceived when performing workplace-based assessments and what solutions they suggested to overcome these difficulties. METHODS Discussion meetings were conducted with education coordinators and faculty (n=55) from 11 peripheral hospitals concerning the difficulties encountered when conducting workplace-based assessments. We analysed the reports from these discussion meetings using an integrated approach guided by our research questions to code the data. Two researchers analysed the data independently and resolved differences of opinion through consensus. RESULTS The problems perceived by faculty in workplace-based assessments (difficulties) and suggestions for improvement formed the overarching themes. Problems included the short duration of clerkships, students choosing the assessment moments, the use of grades for the mini-Clinical Evaluation Exercise, the difficulty in combining teacher and assessor roles and the difficulty in giving fail judgements. Suggestions for improvement included longer clerkship duration, faculty choosing the assessment moments, using a pass/fail system for the mini-Clinical Evaluation Exercise and forward feeding of performance from earlier clerkships following a fail judgement. CONCLUSIONS Our study indicates that faculty perceive difficulties when conducting workplace-based assessments. These assessments need periodical review to understand the difficulties faculty experience using them; they also require periodical feedback to ensure their proper and effective use.
Collapse
Affiliation(s)
- Hester E.M. Daelmans
- VUmc School of Medical Sciences, Institute of Education and Training, Amsterdam, the Netherlands
| | | | - Gerda Croiset
- VUmc School of Medical Sciences, Institute of Education and Training, Amsterdam, the Netherlands
| | - Rashmi A. Kusurkar
- VUmc School of Medical Sciences, Institute of Education and Training, Amsterdam, the Netherlands
| |
Collapse
|