To What Degree Are the 13 Entrustable Professional Activities Already Incorporated Into Physicians' Performance Schemas for Medical Students?

Using entrustable professional activities to better prepare students for their postgraduate medical training: A medical student's perspective

THE PROBLEM

Medical students graduate underprepared for postgraduate medical training despite years of classroom and clinical training. In this article, a medical student shares her personal perspectives on three factors contributing to this problem in undergraduate medical education: students' peripheral roles in the clinical environment impede learning, students receive inadequate feedback, and assessments do not measure desired learning outcomes.

A SOLUTION

The authors describe how using entrustable professional activities (EPAs) could address these issues and promote students' clinical engagement by clarifying their roles, providing them with frequent and actionable feedback, and aligning their assessments with authentic work. These factors combined with grading schemes rewarding improvement could contribute to a growth mindset that reprioritizes clinical skill acquisition. The authors explore how medical schools have begun implementing the EPA framework, highlight insights from these efforts, and describe barriers that must be addressed.

THE FUTURE

Incorporating EPAs into medical school curricula could better prepare students for postgraduate training while also alleviating issues that contribute to student burnout by defining students' roles, improving feedback, and aligning assessments with desired learning outcomes.

The Role of Entrustable Professional Activities in Competency-based Medical Education for Anesthesiology Residents: A Pilot Phase

Background

After graduation, physicians should be able to provide professional and safe services without the need for supervision by their clinical professors, mandating a competency-based medical education (CBME) approach.

Objectives

This study aimed to develop a national model of entrustable professional activities (EPAs) based on our experiences in the Department of Anesthesiology and Critical Care (DACC), Shahid Beheshti University of Medical Sciences (SBMU).

Methods

The primary EPA design plan was designed in a 10-step model as a career roadmap for the project. The texts were prepared according to a consensus-based approach. On the other hand, the texts were reviewed and revised by a broad team of faculty in a daily workshop.

Results

The final product included 14 topics for EPA as the first round of targeting topics for anesthesiology residents. The texts were developed using previous studies and were standardized considering national standards.

Conclusions

We described a clear path toward designing and implementing EPAs in anesthesiology residency programs to improve the quality of the graduated residents. Though the basic theory is the same, each country needs its formula for implementing the process.

Comparing Entrustment Decision-Making Outcomes of the Core Entrustable Professional Activities Pilot, 2019-2020

IMPORTANCE

Gaps in readiness for indirect supervision have been identified for essential responsibilities encountered early in residency, presenting risks to patient safety. Core Entrustable Professional Activities (EPAs) for entering residency have been proposed as a framework to address these gaps and strengthen the transition from medical school to residency.

OBJECTIVE

To assess progress in developing an entrustment process in the Core EPAs framework.

DESIGN, SETTING, AND PARTICIPANTS

In this quality improvement study in the Core EPAs for Entering Residency Pilot, trained faculty made theoretical entrustment determinations and recorded the number of workplace-based assessments (WBAs) available for each determination in 2019 and 2020. Four participating schools attempted entrustment decision-making for all graduating students or a randomly selected subset of students. Deidentified, individual-level data were merged into a multischool database.

INTERVENTIONS

Schools implemented EPA-related curriculum, WBAs, and faculty development; developed systems to compile and display data; and convened groups to make theoretical summative entrustment determinations.

MAIN OUTCOMES AND MEASURES

On an EPA-specific basis, the percentage of students for whom an entrustment determination could be made, the percentage of students ready for indirect supervision, and the volume of WBAs available were recorded.

RESULTS

Four participating schools made 4525 EPA-specific readiness determinations (2296 determinations in 2019 and 2229 determinations in 2020) for 732 graduating students (349 students in 2019 and 383 students in 2020). Across all EPAs, the proportion of determinations of "ready for indirect supervision" increased from 2019 to 2020 (997 determinations [43.4%] vs 1340 determinations [60.1%]; 16.7 percentage point increase; 95% CI, 13.8-19.6 percentage points; P < .001), as did the proportion of determinations for which there were 4 or more WBAs (456 of 2295 determinations with WBA data [19.9%] vs 938 [42.1%]; 22.2 percentage point increase; 95% CI, 19.6-24.8 percentage points; P < .001). The proportion of EPA-specific data sets considered for which an entrustment determination could be made increased from 1731 determinations (75.4%) in 2019 to 2010 determinations (90.2%) in 2020 (14.8 percentage point increase; 95% CI, 12.6-16.9 percentage points; P < .001). On an EPA-specific basis, there were 5 EPAs (EPA 4 [orders], EPA 8 [handovers], EPA 10 [urgent care], EPA 11 [informed consent], and EPA 13 [patient safety]) for which few students were deemed ready for indirect supervision and for which there were few WBAs available per student in either year. For example, for EPA 13, 0 of 125 students were deemed ready in 2019 and 0 of 127 students were deemed ready in 2020, while 0 determinations in either year included 4 or more WBAs.

CONCLUSIONS AND RELEVANCE

These findings suggest that there was progress in WBA data collected, the extent to which entrustment determinations could be made, and proportions of entrustment determinations reported as ready for indirect supervision. However, important gaps remained, particularly for a subset of Core EPAs.

Entrustment Decision Making in the Core Entrustable Professional Activities: Results of a Multi-Institutional Study

PURPOSE

In 2014, the Association of American Medical Colleges defined 13 Core Entrustable Professional Activities (EPAs) that all graduating students should be ready to do with indirect supervision upon entering residency and commissioned a 10-school, 5-year pilot to test implementing the Core EPAs framework. In 2019, pilot schools convened trained entrustment groups (TEGs) to review assessment data and render theoretical summative entrustment decisions for class of 2019 graduates. Results were examined to determine the extent to which entrustment decisions could be made and the nature of these decisions.

METHOD

For each EPA considered (4-13 per student), TEGs recorded an entrustment determination (ready, progressing but not yet ready, evidence against student progressing, could not make a decision); confidence in that determination (none, low, moderate, high); and the number of workplace-based assessments (WBAs) considered (0->15) per determination. These individual student-level data were de-identified and merged into a multischool database; chi-square analysis tested the significance of associations between variables.

RESULTS

The 2,415 EPA-specific determinations (for 349 students by 4 participating schools) resulted in a decision of ready (n = 997/2,415; 41.3%), progressing but not yet ready (n = 558/2,415; 23.1%), or evidence against student progression (n = 175/2,415; 7.2%). No decision could be made for the remaining 28.4% (685/2,415), generally for lack of data. Entrustment determinations' distribution varied across EPAs (chi-square P < .001) and, for 10/13 EPAs, WBA availability was associated with making (vs not making) entrustment decisions (each chi-square P < .05).

CONCLUSIONS

TEGs were able to make many decisions about readiness for indirect supervision; yet less than half of determinations resulted in a decision of readiness to perform this EPA with indirect supervision. More work is needed at the 10 schools to enable authentic summative entrustment in the Core EPAs framework.

Core Entrustable Professional Activities (EPAs) and the Transition from Medical School to Residency: the Postgraduate Year One Resident Perspective

INTRODUCTION

The Association of American Medical Colleges (AAMC) proposed thirteen core Entrustable Professional Activities (EPAs) that all graduates should be able to perform under indirect supervision upon entering residency. As an underlying premise is that graduates ready to do so will be better prepared to transition to the responsibilities of residency, we explored the relationship between postgraduate year (PGY)-1 residents' self-assessed preparedness to perform core EPAs under indirect supervision at the start of residency with their ease of transition to residency.

METHODS

Using response data to a questionnaire administered in September 2019 to PGY-1 residents who graduated from AAMC core EPA pilot schools, we examined between-group differences and independent associations for each of PGY-1 position type, specialty, and "EPA-preparedness" score (proportion of EPAs the resident reported as prepared to perform under indirect supervision at the start of residency) and ease of transition to residency (from 1 = much harder to 5 = much easier than expected).

RESULTS

Of 274 questionnaire respondents (19% of 1438 graduates), 241 (88% of 274) had entered PGY-1 training and completed all questionnaire items of interest. EPA-preparedness score (mean 0.71 [standard deviation 0.26]) correlated with ease of transition (3.1 [0.9]; correlation = .291, p < .001). In linear regression controlling for specialty (among other variables), EPA-preparedness score (β-coefficient 1.08; 95% confidence interval .64-1.52; p < .001) predicted ease of transition to residency.

CONCLUSION

Graduates who felt prepared to perform many of the core EPAs under indirect supervision at the start of PGY-1 training reported an easier-than-expected transition to residency.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-021-01370-3.

Assessment of Entrustable Professional Activities Using a Web-Based Simulation Platform During Transition to Emergency Medicine Residency: Mixed Methods Pilot Study

BACKGROUND

The 13 core entrustable professional activities (EPAs) are key competency-based learning outcomes in the transition from undergraduate to graduate medical education in the United States. Five of these EPAs (EPA2: prioritizing differentials, EPA3: recommending and interpreting tests, EPA4: entering orders and prescriptions, EPA5: documenting clinical encounters, and EPA10: recognizing urgent and emergent conditions) are uniquely suited for web-based assessment.

OBJECTIVE

In this pilot study, we created cases on a web-based simulation platform for the diagnostic assessment of these EPAs and examined the feasibility and acceptability of the platform.

METHODS

Four simulation cases underwent 3 rounds of consensus panels and pilot testing. Incoming emergency medicine interns (N=15) completed all cases. A maximum of 4 "look for" statements, which encompassed specific EPAs, were generated for each participant: (1) performing harmful or missing actions, (2) narrowing differential or wrong final diagnosis, (3) errors in documentation, and (4) lack of recognition and stabilization of urgent diagnoses. Finally, we interviewed a sample of interns (n=5) and residency leadership (n=5) and analyzed the responses using thematic analysis.

RESULTS

All participants had at least one missing critical action, and 40% (6/15) of the participants performed at least one harmful action across all 4 cases. The final diagnosis was not included in the differential diagnosis in more than half of the assessments (8/15, 54%). Other errors included selecting incorrect documentation passages (6/15, 40%) and indiscriminately applying oxygen (9/15, 60%). The interview themes included psychological safety of the interface, ability to assess learning, and fidelity of cases. The most valuable feature cited was the ability to place orders in a realistic electronic medical record interface.

CONCLUSIONS

This study demonstrates the feasibility and acceptability of a web-based platform for diagnostic assessment of specific EPAs. The approach rapidly identifies potential areas of concern for incoming interns using an asynchronous format, provides feedback in a manner appreciated by residency leadership, and informs individualized learning plans.

Entrustable professional activities in entry-level health professional education: A scoping review

Entrustable professional activities (EPAs) are a recent enhancement to competency-based health professional education that describe the observable work done by a competent health professional. Through defining education outcomes in a work-based context, EPAs offer potential to identify skill gaps in individual or student cohorts and focus improvements. Entrustable professional activities have been pioneered and gained rapid acceptance in postgraduate medical education; however, less is known about their application and use in undergraduate or entry-level health professional education. The Joanna Briggs Institute scoping review methodology was used to explore how and in what context EPAs are being used in entry-level health professional education. Databases searched include CINAHL, EMBASE, MEDLINE, Web of Science and PsycINFO. A total of 748 abstracts were returned after duplicates removed, and 127 full-text articles were screened with 30 included for data extraction. Publications in this area have recently accelerated with disciplines of professions of medicine, pharmacy, dietetics and physician assistants reporting on EPA development, implementation and evaluation. EPA use has been reported in the United States, Canada, Europe Australia and Central America. Major motivation reported for EPA use is to improve patient safety by aligning performance and expectations and to improve student assessment. Several studies report on the use of EPAs to evaluate different curriculum models or identify curriculum gaps representing potential application in education research.

Factors influencing the rating of sonographer students' clinical performance

INTRODUCTION

Little is known about the factors influencing clinical supervisor-assessors' ratings of sonographer students' performance. This study identifies these influential factors and relates them to professional competency standards, with the aim of raising awareness and improving assessment practice.

METHODS

This study used archived written comments from 94 clinical assessors describing 174 sonographer students' performance one month into their initial clinical practice (2015-6). Qualitative mixed method analysis revealed factors influencing assessor ratings of student performance and provided an estimate of the valency, association, and frequency of these factors.

RESULTS

Assessors provided written comments for 93 % (n = 162/174) of students. Comments totaled 7190 words (mean of 44 words/student). One-third of comment paragraphs were wholly positive, two-thirds were equivocal. None were wholly negative. Thematic analysis revealed eleven factors, and eight sub-factors, influencing assessor impressions of five dimensions of performance. Of the factors mentioned, 84.6 % (n = 853/1008) related to professional competencies. While 15.4 % (n = 155/1008) were unrelated to competencies, instead reflecting humanistic factors such as student motivation, disposition, approach to learning, prospects and impact on supervisor and staff. Factors were prioritised and combined independently, although some associated.

CONCLUSION

Clinical assessors formed impressions based on student performance, humanistic behaviours and personal qualities not necessarily outlined in educational outcomes or professional competency standards. Their presence, and interrelations, impact success in clinical practice, through their contribution to, and indication of, competence.

IMPLICATIONS FOR PRACTICE

Sonographer student curricula and assessor training should raise awareness of the factors influencing performance ratings and judgement of clinical competence, particularly the importance of humanistic factors. Inclusion of narrative comments, multiple assessors, and broad performance dimensions would enhance clinical assessment of sonographer student performance.

Numbers Encapsulate, Words Elaborate: Toward the Best Use of Comments for Assessment and Feedback on Entrustment Ratings

The adoption of entrustment ratings in medical education is based on a seemingly simple premise: to align workplace-based supervision with resident assessment. Yet it has been difficult to operationalize this concept. Entrustment rating forms combine numeric scales with comments and are embedded in a programmatic assessment framework, which encourages the collection of a large quantity of data. The implicit assumption that more is better has led to an untamable volume of data that competency committees must grapple with. In this article, the authors explore the roles of numbers and words on entrustment rating forms, focusing on the intended and optimal use(s) of each, with a focus on the words. They also unpack the problematic issue of dual-purposing words for both assessment and feedback. Words have enormous potential to elaborate, to contextualize, and to instruct; to realize this potential, educators must be crystal clear about their use. The authors set forth a number of possible ways to reconcile these tensions by more explicitly aligning words to purpose. For example, educators could focus written comments solely on assessment; create assessment encounters distinct from feedback encounters; or use different words collected from the same encounter to serve distinct feedback and assessment purposes. Finally, the authors address the tyranny of documentation created by programmatic assessment and urge caution in yielding to the temptation to reduce words to numbers to make them manageable. Instead, they encourage educators to preserve some educational encounters purely for feedback, and to consider that not all words need to become data.

Revisiting the Core Entrustable Professional Activities for Entering Residency

The Core EPAs for Entering Residency Pilot project aimed to test the feasibility of implementing 13 entrustable professional activities (EPAs) at 10 U.S. medical schools and to gauge whether the use of the Core EPAs could improve graduates' performance early in residency. In this manuscript, the authors (members of the pilot institutions and Association of American Medical Colleges staff supporting the project evaluation) describe the schools' capacity to collect multimodal evidence about their students' performance in each of the Core EPAs and the ability of faculty committees to use those data to make decisions regarding learners' readiness for entrustment. In reviewing data for each of the Core EPAs, the authors reflected on how each activity performed as an EPA informed by how well it could be assessed and entrusted. For EPAs that did not perform well, the authors examined whether there are underlying practical and/or theoretical issues limiting its utility as a measure of student performance in medical school.

Workplace-based assessments of entrustable professional activities in a psychiatry core clerkship: an observational study

BACKGROUND

Entrustable professional activities (EPAs) in competency-based, undergraduate medical education (UME) have led to new formative workplace-based assessments (WBA) using entrustment-supervision scales in clerkships. We conducted an observational, prospective cohort study to explore the usefulness of a WBA designed to assess core EPAs in a psychiatry clerkship.

METHODS

We analyzed changes in self-entrustment ratings of students and the supervisors' ratings per EPA. Timing and frequencies of learner-initiated WBAs based on a prospective entrustment-supervision scale and resultant narrative feedback were analyzed quantitatively and qualitatively. Predictors for indirect supervision levels were explored via regression analysis, and narrative feedback was coded using thematic content analysis. Students evaluated the WBA after each clerkship rotation.

RESULTS

EPA 1 ("Take a patient's history"), EPA 2 ("Assess physical & mental status") and EPA 8 ("Document & present a clinical encounter") were most frequently used for learner-initiated WBAs throughout the clerkship rotations in a sample of 83 students. Clinical residents signed off on the majority of the WBAs (71%). EPAs 1, 2, and 8 showed the largest increases in self-entrustment and received most of the indirect supervision level ratings. We found a moderate, positive correlation between self-entrusted supervision levels at the end of the clerkship and the number of documented entrustment-supervision ratings per EPA (p < 0.0001). The number of entrustment ratings explained 6.5% of the variance in the supervisors' ratings for EPA 1. Narrative feedback was documented for 79% (n = 214) of the WBAs. Most narratives addressed the Medical Expert role (77%, n = 208) and used reinforcement (59%, n = 161) as a feedback strategy. Students perceived the feedback as beneficial.

CONCLUSIONS

Using formative WBAs with an entrustment-supervision scale and prompts for written feedback facilitated targeted, high-quality feedback and effectively supported students' development toward self-entrusted, indirect supervision levels.

Working with entrustable professional activities in clinical education in undergraduate medical education: a scoping review

BACKGROUND

Entrustable professional activities (EPAs) are increasingly used in undergraduate medical education (UME). We conducted a scoping review to summarize the evidence for the use of EPAs in clinical rotations in UME.

METHODS

We searched multiple databases for scoping reviews based on the PRISMA guidelines for articles reporting qualitative and quantitative research, as well as conceptual and curriculum development reports, on EPAs in UME clinical rotations.

RESULTS

We identified 3309 records by searching through multiple databases. After the removal of duplicates, 1858 reports were screened. A total of 36 articles were used for data extraction. Of these, 47% reported on EPA and EPA-based curriculum development for clerkships, 50% reported on implementation strategies, and 53% reported on assessment methods and tools used in clerkships. Validity frameworks for developing EPAs in the context of clerkships were inconsistent. Several specialties reported feasible implementation strategies for EPA-based clerkship curricula, however, these required additional faculty time and resources. Limited exposure to clinical activities was identified as a barrier to relevant learning experiences. Educators used nationally defined, or specialty-specific EPAs, and a range of entrustability and supervision scales. We found only one study that used an empirical research approach for EPA assessment. One article reported on the earlier advancement of trainees from UME to graduate medical education based on summative entrustment decisions.

CONCLUSIONS

There is emerging evidence concerning how EPAs can be effectively introduced to clinical training in UME. Specialty-specific, nested EPAs with context-adapted, entrustment-supervision scales might be helpful in better leveraging their formative assessment potential.

Relationship Between "Simulated Patient Scenarios and Role-Playing" Method and OSCE Performance in Senior Anesthesiology Residents: A Correlation Assessment Study

BACKGROUND

"Simulated-patient scenarios and role-playing" and OSCE are among the many non-traditional education methods with variable results in different clinical settings.

OBJECTIVES

This cross-sectional study was performed to assess the correlation between the results of these two methods in senior anesthesiology residents, with a special focus on four of the six ACGME core competencies.

METHODS

During two years, senior anesthesiology residents were subject to "simulated patient scenario and role-playing" sessions. Two faculty members took the role of the patient and one of the relatives. An objective checklist with 15 items was prepared to be rated by other department faculty members. Meanwhile, an ordered pattern of OSCE was prepared to cover four core competencies that were more related to this academic process (from a total of six core competencies). The mean and standard deviation of the score of each of the 15 items in the checklist were calculated. The correlation between cumulative checklist scoring results and OSCE exam results was assessed. A P value of less than 0.05 was considered significant.

RESULTS

A total of 40 senior anesthesiology residents, with 344 assessments by faculty members in 40 sessions, were enrolled in the study. The questionnaire's Cronbach's alpha reliability was 0.74. No statistically significant disparity was detected between the results of the two assessment methods, while the results of the two assessments had a significant correlation (two-tailed correlation coefficient = 0.886; P value < 0.001).

CONCLUSIONS

There was an objective relationship between the results of "simulated patient scenario and role-playing" strategies and the results of OSCE exams using an observer-based rating method. Thus, they could be used as surrogates in the assessment of core clinical competencies of senior anesthesiology residents.