1
|
Kitto S, Fantaye AW, Zevin B, Fowler A, Sachdeva AK, Raiche I. A Scoping Review of the Literature on Entrustable Professional Activities in Surgery Residency Programs. JOURNAL OF SURGICAL EDUCATION 2024; 81:823-840. [PMID: 38679495 DOI: 10.1016/j.jsurg.2024.02.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 02/20/2024] [Indexed: 05/01/2024]
Abstract
OBJECTIVE Entrustable professional activities (EPAs) are a crucial component of contemporary postgraduate medical education with many surgery residency programs having implemented EPAs as a competency assessment framework to assess and provide feedback on the performance of their residents. Despite broad implementation of EPAs, there is a paucity of evidence regarding the impact of EPAs on the learners and learning environments. A first step in improving understanding of the use and impact of EPAs is by mapping the rising number of EPA-related publications from the field of surgery. The primary objective of this scoping review is to examine the nature, extent, and range of articles on the development, implementation, and assessment of EPAs. The second objective is to identify the experiences and factors that influence EPA implementation and use in practice in surgical specialties. DESIGN Scoping review. Four electronic databases (Medline, Embase, Education Source, and ERIC) were searched on January 20, 2022, and then again on July 19, 2023. A quasi-statistical content analysis was employed to quantify and draw meaning from the information related to the development, implementation, assessment, validity, reliability, and experiences with EPAs in the workplace. PARTICIPANTS A total of 42 empirical and nonempirical articles were included. RESULTS Four thematic categories describe the topic areas in included articles related to: 1) the development and refinement of EPAs, including the multiple steps taken to develop and refine unique EPAs for surgery residency programs; 2) the methods for implementing EPAs; 3) outcomes of EPA use in practice; 4) barriers, facilitators, and areas for improvement for the implementation and use of EPAs in surgical education. CONCLUSIONS This scoping review highlights the key trends and gaps from the rapidly increasing number of publications on EPAs in surgery residency, from development to their use in the workplace. Existing EPA studies lack a theoretical and/or conceptual basis; future development and implementation studies should adopt implementation science frameworks to better structure and operationalize EPAs within surgery residency programs.
Collapse
Affiliation(s)
- Simon Kitto
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
| | - Arone W Fantaye
- Office of Continuing Professional Development, Faculty of Medicine, University of Ottawa, Ottawa, Canada
| | - Boris Zevin
- Department of Surgery, Queen's University, Kingston, Canada
| | - Amanda Fowler
- Department of Surgery, Memorial University of Newfoundland, St. John's, Newfoundland, Canada
| | - Ajit K Sachdeva
- Division of Education, American College of Surgeons, Chicago, Illinios
| | - Isabelle Raiche
- Department of Surgery, University of Ottawa, Ottawa, Canada.
| |
Collapse
|
2
|
Montgomery KB, Mellinger JD, Lindeman B. Entrustable Professional Activities in Surgery: A Review. JAMA Surg 2024; 159:571-577. [PMID: 38477902 DOI: 10.1001/jamasurg.2023.8107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/14/2024]
Abstract
Importance Entrustable professional activities (EPAs) compose a competency-based education (CBE) assessment framework that has been increasingly adopted across medical specialties as a workplace-based assessment tool. EPAs focus on directly observed behaviors to determine the level of entrustment a trainee has for a given activity of that specialty. In this narrative review, we highlight the rationale for EPAs in general surgery, describe current evidence supporting their use, and outline some of the practical considerations for EPAs among residency programs, faculty, and trainees. Observations An expanding evidence base for EPAs in general surgery has provided moderate validity evidence for their use as well as practical recommendations for implementation across residency programs. Challenges to EPA use include garnering buy-in from individual faculty and residents to complete EPA microassessments and engage in timely, specific feedback after a case or clinical encounter. When successfully integrated into a program's workflow, EPAs can provide a more accurate picture of residents' competence for a fundamental surgical task or activity compared with other assessment methods. Conclusions and Relevance EPAs represent the next significant shift in the evaluation of general surgery residents as part of the overarching progression toward CBE among all US residency programs. While pragmatic challenges to the implementation of EPAs remain, the best practices from EPA and other CBE assessment literature summarized in this review may assist individuals and programs in implementing EPAs. As EPAs become more widely used in general surgery resident training, further analysis of barriers and facilitators to successful and sustainable EPA implementation will be needed to continue to optimize and advance this new assessment framework.
Collapse
|
3
|
Chen XP, Harzman A, Go M, Arnold M, Ellison EC. Cumulative Sum Chart as Complement to Objective Assessment of Graduating Surgical Resident Competency: An Exploratory Study. J Am Coll Surg 2023; 237:894-901. [PMID: 37530413 DOI: 10.1097/xcs.0000000000000812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/03/2023]
Abstract
BACKGROUND Rater-based assessment and objective assessment play an important role in evaluating residents' clinical competencies. We hypothesize that a cumulative sum (CUSUM) chart of operative time is a complement to the assessment of chief general surgery residents' competencies with ACGME Milestones, aiding residency programs' determination of graduating residents' practice readiness. STUDY DESIGN We extracted ACGME Milestone evaluations of performance of operations and procedures (POP) and 3 objective metrics (operative time, case type, and case complexity) from 3 procedures (cholecystectomy, colectomy, and inguinal hernia) performed by 3 cohorts of residents (N = 15) during their PGY4-5. CUSUM charts were computed for each resident on each procedure type. A learning plateau was defined as at least 4 cases consistently locating around the centerline (target performance) at the end of a CUSUM chart with minimal deviations (range 0 to 1). RESULTS All residents reached the ACGME graduation targets for the overall POP by the end of chief year. A total of 2,446 cases were included (cholecystectomy N = 1234, colectomy N = 507, and inguinal hernia N = 705), and 3 CUSUM chart patterns emerged: skewed distribution, bimodal distribution, and peaks and valleys distribution. Analysis of CUSUM charts revealed surgery residents' development processes in the operating room towards a learning plateau vary, and only 46.7% residents reach a learning plateau in all 3 procedures upon graduation. CONCLUSIONS CUSUM charts of operative time complement the ACGME Milestones evaluations. The use of both may enable residency programs to holistically determine graduating residents' practice readiness and provide recommendations for their upcoming career/practice transition.
Collapse
|
4
|
Huynh A, Nguyen A, Beyer RS, Harris MH, Hatter MJ, Brown NJ, de Virgilio C, Nahmias J. Fixing a Broken Clerkship Assessment Process: Reflections on Objectivity and Equity Following the USMLE Step 1 Change to Pass/Fail. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:769-774. [PMID: 36780667 DOI: 10.1097/acm.0000000000005168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Clerkship grading is a core feature of evaluation for medical students' skills as physicians and is considered by most residency program directors to be an indicator of future performance and success. With the transition of the U.S. Medical Licensing Examination Step 1 score to pass/fail, there will likely be even greater reliance on clerkship grades, which raises several important issues that need to be urgently addressed. This article details the current landscape of clerkship grading and the systemic discrepancies in assessment and allocation of honors. The authors examine not only objectivity and fairness in clerkship grading but also the reliability of clerkship grading in predicting residency performance and the potential benefits and drawbacks to adoption of a pass/fail clinical clerkship grading system. In the promotion of a more fair and equitable residency selection process, there must be standardization of grading systems with consideration of explicit grading criteria, grading committees, and/or structured education of evaluators and assessors regarding implicit bias. In addition, greater adherence and enforcement of transparency in grade distributions in the Medical Student Performance Evaluation is needed. These changes have the potential to level the playing field, foster equitable comparisons, and ultimately add more fairness to the residency selection process.
Collapse
Affiliation(s)
- Ashley Huynh
- A. Huynh is a first-year medical student, University of California, Irvine, School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0002-4413-6829
| | - Andrew Nguyen
- A. Nguyen is a first-year medical student, University of Florida College of Medicine, Gainesville, Florida; ORCID: https://orcid.org/0000-0002-8131-150X
| | - Ryan S Beyer
- R.S. Beyer is a second-year medical student, University of California, Irvine, School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0002-0283-3749
| | - Mark H Harris
- M.H. Harris is a second-year medical student, University of California, Irvine, School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0002-1598-225X
| | - Matthew J Hatter
- M.J. Hatter is a second-year medical student, University of California, Irvine, School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0003-2922-6196
| | - Nolan J Brown
- N.J. Brown is a fourth-year medical student, University of California, Irvine, School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0002-6025-346X
| | - Christian de Virgilio
- C. de Virgilio is professor of surgery, Harbor-UCLA Medical Center, Torrance, California
| | - Jeffry Nahmias
- J. Nahmias is professor of trauma, burns, surgical critical care, and acute care surgery, University of California, Irvine, School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0003-0094-571X
| |
Collapse
|
5
|
Soelling S, Riley M, Smink DS. Does Past Performance Predict Future Success?-Why This Matters for Measuring Competency. JAMA Surg 2023; 158:521. [PMID: 36884242 DOI: 10.1001/jamasurg.2023.0020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/09/2023]
Affiliation(s)
| | - Max Riley
- Brigham and Women's Hospital, Boston, Massachusetts
| | | |
Collapse
|
6
|
Jensen RM, Kearse LE, Anand A, Dent DL, Korndorffer JR. The Program Director Perspective: Perceptions of PGY5 Residents' Operative Self-Efficacy and Entrustment. Am Surg 2023:31348231157862. [PMID: 36802912 DOI: 10.1177/00031348231157862] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2023]
Abstract
INTRODUCTION A 2020 survey of post-graduate year 5 (PGY5) general surgery residents linked to the American Board of Surgery In-Training Examination (ABSITE) revealed significant deficits in self-efficacy (SE), or personal judgment of one's ability to complete a task, for 10 commonly performed operations. Identifying whether this deficit is similarly perceived by program directors (PDs) has not been well established. We hypothesized that PDs would perceive higher levels of operative SE compared to PGY5s. METHODS A survey was distributed through the Association of Program Directors in Surgery listserv; PDs were queried about their PGY5 residents' ability to perform the same 10 operations independently and their accuracy of patient assessments and operative plans for components of several core entrustable professional activities (EPAs). Results of this survey were compared to PGY5 residents' perception of their SE and entrustment based on the 2020 post-ABSITE survey. Chi-squared tests were used for statistical analysis. RESULTS 108 responses were received, representing ∼32% (108/342) of general surgery programs. Perceptions from PDs of PGY5 residents' operative SE were highly concordant with resident perceptions; no significant differences were observed for 9 of 10 procedures. Both PGY5 residents and PDs perceived adequate levels of entrustment; no significant differences were observed for 6 of 8 EPA components. CONCLUSIONS These findings show concordance between PDs and PGY5 residents in their perceptions of operative SE and entrustment. Though both groups perceive adequate levels of entrustment, PDs corroborate the previously described operative SE deficit, illustrating the importance of improved preparation for independent practice.
Collapse
Affiliation(s)
| | - LaDonna E Kearse
- Department of Surgery, 8369Howard University, Washington, DC, USA
| | - Ananya Anand
- Department of Surgery, 6429Stanford University, CA, USA
| | - Daniel L Dent
- Department of Surgery, 14742University of Texas Health San Antonio, TX, USA
| | | |
Collapse
|
7
|
Ten Cate O, Schumacher DJ. Entrustable professional activities versus competencies and skills: Exploring why different concepts are often conflated. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2022; 27:491-499. [PMID: 35226240 PMCID: PMC9117349 DOI: 10.1007/s10459-022-10098-7] [Citation(s) in RCA: 33] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/14/2021] [Accepted: 01/23/2022] [Indexed: 05/25/2023]
Abstract
Despite explanations in the literature, a returning question in the use of entrustable professional activities (EPAs) is how to distinguish them from competencies and skills. In this article, we attempt to analyze the causes of the frequent confusion and conflation of EPAs with competencies and skills, and argue why the distinction is important for education, qualification and patient safety. 'Tracheotomy', 'lumbar puncture', 'interprofessional collaboration' for example are colloquially called 'skills', but its is a person's ability to perform these activities that is the actual skill; the EPA is simply the activity itself. We identify two possible causes for the confusion. One is a tendency to frame all educational objectives as EPAs. Many objectives of medical training can be conceptualized as EPAs, if 'the ability to do X' is the corresponding competency; but that does not work for all. We offer ways to deal with objectives of training that are not usefully conceptualized as EPAs. A more fundamental cause relates to entrustment decisions. The permission to contribute to health care reflects entrustment. Entrustment decisions are the links or pivots between a person's readiness for the task and the actual task execution. However, if entrustment decisions do not lead to increased autonomy in the practice of health care, but only serve to decide upon the advancement to a next stage of training, EPAs can become the tick boxes learners feel they need to collect to 'pass'. Gradually, then, EPAs can loose their original meaning of units of practice for which one becomes qualified.
Collapse
Affiliation(s)
- Olle Ten Cate
- Utrecht Center for Research and Development of Health Professions Education, University Medical Center Utrecht, P.O. Box # 85500, 3508 GA, Utrecht, The Netherlands.
| | - Daniel J Schumacher
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| |
Collapse
|