Creating a shared mental model for EPAs in surgery

A Scoping Review of the Literature on Entrustable Professional Activities in Surgery Residency Programs

OBJECTIVE

Entrustable professional activities (EPAs) are a crucial component of contemporary postgraduate medical education with many surgery residency programs having implemented EPAs as a competency assessment framework to assess and provide feedback on the performance of their residents. Despite broad implementation of EPAs, there is a paucity of evidence regarding the impact of EPAs on the learners and learning environments. A first step in improving understanding of the use and impact of EPAs is by mapping the rising number of EPA-related publications from the field of surgery. The primary objective of this scoping review is to examine the nature, extent, and range of articles on the development, implementation, and assessment of EPAs. The second objective is to identify the experiences and factors that influence EPA implementation and use in practice in surgical specialties.

DESIGN

Scoping review. Four electronic databases (Medline, Embase, Education Source, and ERIC) were searched on January 20, 2022, and then again on July 19, 2023. A quasi-statistical content analysis was employed to quantify and draw meaning from the information related to the development, implementation, assessment, validity, reliability, and experiences with EPAs in the workplace.

PARTICIPANTS

A total of 42 empirical and nonempirical articles were included.

RESULTS

Four thematic categories describe the topic areas in included articles related to: 1) the development and refinement of EPAs, including the multiple steps taken to develop and refine unique EPAs for surgery residency programs; 2) the methods for implementing EPAs; 3) outcomes of EPA use in practice; 4) barriers, facilitators, and areas for improvement for the implementation and use of EPAs in surgical education.

CONCLUSIONS

This scoping review highlights the key trends and gaps from the rapidly increasing number of publications on EPAs in surgery residency, from development to their use in the workplace. Existing EPA studies lack a theoretical and/or conceptual basis; future development and implementation studies should adopt implementation science frameworks to better structure and operationalize EPAs within surgery residency programs.

Entrustable Professional Activities in Surgery: A Review

Importance

Entrustable professional activities (EPAs) compose a competency-based education (CBE) assessment framework that has been increasingly adopted across medical specialties as a workplace-based assessment tool. EPAs focus on directly observed behaviors to determine the level of entrustment a trainee has for a given activity of that specialty. In this narrative review, we highlight the rationale for EPAs in general surgery, describe current evidence supporting their use, and outline some of the practical considerations for EPAs among residency programs, faculty, and trainees.

Observations

An expanding evidence base for EPAs in general surgery has provided moderate validity evidence for their use as well as practical recommendations for implementation across residency programs. Challenges to EPA use include garnering buy-in from individual faculty and residents to complete EPA microassessments and engage in timely, specific feedback after a case or clinical encounter. When successfully integrated into a program's workflow, EPAs can provide a more accurate picture of residents' competence for a fundamental surgical task or activity compared with other assessment methods.

Conclusions and Relevance

EPAs represent the next significant shift in the evaluation of general surgery residents as part of the overarching progression toward CBE among all US residency programs. While pragmatic challenges to the implementation of EPAs remain, the best practices from EPA and other CBE assessment literature summarized in this review may assist individuals and programs in implementing EPAs. As EPAs become more widely used in general surgery resident training, further analysis of barriers and facilitators to successful and sustainable EPA implementation will be needed to continue to optimize and advance this new assessment framework.

Surgical Education

This paper summarizes the proceedings of the joint European Surgical Association ESA/American Surgical Association symposium on Surgical Education that took place in Bordeaux, France, as part of the celebrations for 30 years of ESA scientific meetings. Three presentations on the use of quantitative metrics to understand technical decisions, coaching during training and beyond, and entrustable professional activities were presented by American Surgical Association members and discussed by ESA members in a symposium attended by members of both associations.

The entrustable professional activities of laparoscopic surgery: moving toward an integrated training model

BACKGROUND

Entrustable Professional Activities (EPAs) provide the opportunity to integrate multiple competencies into meaningful units that facilitate curriculum development and assessment design. As part of the process of reviewing and enhancing the Fundamentals of Laparoscopic of Surgery (FLS) program, we used the concept of EPAs to create a framework of reference that articulates a contemporary definition of Laparoscopic Surgery (LS).

METHODS

The framework of reference of LS was created with data gathered from a literature review and during series of educational retreats with subject matter experts (SMEs). Various activities were implemented during these retreats to develop the LS EPAs, their constitutive competencies, and related observable behaviors.

RESULTS

Ten EPAs and associated competency descriptors (articulated as observable behaviors) specific to LS were identified. In addition, knowledge areas were associated to each EPA.

DISCUSSION

A comprehensive list of EPAs for LS were identified. These EPAs will be used in the development and update of the FLS program. Further, they can be used to guide the development of curriculum, clinical teaching, and assessment in any surgical program with a laparoscopic training component. They are applicable to any level of training by defining the expected observable behaviors associated with a given level of expertise. These fundamental aspects of LS provide a common framework of reference across different surgical specialties.

Concurrent Validity Evidence for Entrustable Professional Activities in General Surgery Residents

BACKGROUND

A subset of Entrustable Professional Activities (EPAs) has been developed for general surgery. We aim to contribute validity evidence for EPAs as an assessment framework for general surgery residents, including concurrent validity compared to ACGME milestones, the current gold standard for evaluating competency.

STUDY DESIGN

This is a cross-sectional study in a general surgery training program within a tertiary academic medical center. EPA assessments were submitted using a mobile app and scored on a numerical scale, mirroring milestones. EPA score distribution was analyzed with respect to post-graduate year (PGY) level and phase of care. Proportional odds logistic regression identified significant predictors. Spearman rank and Wilcoxon rank tests were used for comparisons with milestone ratings.

RESULTS

From August 2018 to December 2019, 320 assessments were collected. EPA scores increased by PGY level. Operative phase EPA scores were significantly lower than nonoperative phase scores. PGY level, operative phase, and case difficulty significantly influenced entrustment scoring. EPA scores demonstrated strong correlation with nonoperative milestones patient care-1, medical knowledge-1, interpersonal and communication skills-2, interpersonal and communication skills-3, professionalism-1, professionalism-3, and practice-based learning and improvement-2 (ρ > 0.5, p < 0.05) and a weaker correlation with operative milestones patient care-3 and medical knowledge-2 (ρ < 0.5, p < 0.05).

CONCLUSIONS

The influence of PGY level and operative phase on entrustment scoring supports the validity of EPAs as a formative evaluation framework for general surgery resident performance. In addition, evident correlations between EPA scores and respective milestone ratings provide concurrent validity evidence.

Resident self-assessment of common endocrine procedures

BACKGROUND

General surgery residency graduates are expected to be proficient in straightforward endocrine operations. This study aimed to elucidate residents' self-assessment of their ability to perform common endocrine procedures.

METHODS

A fourteen-question survey was emailed to general surgery residents from seven U.S. residency programs regarding their self-assessed ability to perform each step of a straightforward thyroidectomy and parathyroidectomy. Demographics and perceived ability to perform the various procedures were collected.

RESULTS

A minority of respondents (17, 27.9%) agreed they could complete a straightforward thyroidectomy for benign disease, with only 11.7% (n = 7) agreeing they could complete a straightforward thyroidectomy for malignant disease. 26.2% (n = 16) of respondents agreed they could complete a straightforward parathyroidectomy. Completed number of cases was significantly associated with greater self-assessed ability to perform the endocrine operations (p = 0.02).

CONCLUSIONS

Most general surgery residents surveyed did not feel capable of performing common, straightforward endocrine procedures. Although confidence in operative ability increased with PGY-level and number of cases completed, the majority of PGY-5 residents still did not feel able to perform a thyroidectomy for malignant disease unassisted.