Toward Defining the Foundation of the MD Degree: Core Entrustable Professional Activities for Entering Residency

The Transformative Care Continuum: implementing an accelerated pathway that addresses the new roles of the family medicine physician

BACKGROUND

The Transformative Care Continuum (TCC) emerged in 2018 at Ohio University's Heritage College of Osteopathic Medicine, combining a three-year medical education track with a three-year family medicine residency. TCC aligns evolving family physician roles through the Kern model, AMA's Master Adaptive Learner model, Health Systems Science Training, and Kirkpatrick's evaluation model.

METHODS

The TCC curriculum emphasizes intensive coaching, clinical encounter video evaluation, reflection, and case-log review. It fosters longitudinal clinical integration, community engagement, and a dynamic learning atmosphere. Students receive rigorous patient-centered communication training and engage in residency-based quality improvement projects, targeting care gap closure and community health in an accelerated 3-year program.

OUTCOMES

Assessment of TCC graduates demonstrates advanced team communication, leadership, and project management skills, with entrustable professional activities (EPA) scores meeting or surpassing those of traditional program graduates. Projects led by students have yielded notable clinical enhancements, national recognition, and significant philanthropic funding for non-medical determinants of health. Finally, there is an overall increase in scholarly activity and leadership roles within the residency programs that have engaged these students.

DISCUSSION

Lessons reveal intrinsic challenges and heightened academic demands for students and residency programs. Additional educational support for students may be necessary, though costly. Limitations in residency slots and faculty availability as student educators potentially hinder scalability. Ongoing faculty training, cultural support, and early integration of digital systems for curriculum management and evaluation are vital for success. Obtaining patient satisfaction, health outcomes, and program measures remains challenging due to privacy concerns and approval processes between institutions.

CONCLUSION

Programs like TCC effectively prepare students for family physician leadership and change management roles through tailored learning, longitudinal experiences, health systems training, and addressing critiques of traditional medical education. Continuous feedback and robust communication strategies are essential for program improvement, fostering well-prepared family physicians committed to health system enhancement.

Introduction to Entrustable Professional Activities for Pediatric Surgery

BACKGROUND

Entrustable Professional Activities (EPAs) have emerged as a valuable tool in medical education, enabling the assessment of trainee competence in a real-world context. Despite its growing popularity in other medical specialties, the use of EPAs in pediatric surgery is still relatively new.

METHODS

This article provides an overview of the development and application of EPAs in pediatric surgery. The key concepts and principles of EPAs, including the historical development and current implementation in other medical specialties, will also be reviewed. Definition of pediatric surgery EPAs, advantages, and challenges of implementing EPAs in pediatric surgery, and the evidence supporting the use of EPAs in pediatric surgical training will also be examined.

RESULTS

The use of EPAs in pediatric surgery has shown promise in providing a comprehensive and standardized approach to assessing trainee competence. By focusing on the essential tasks and responsibilities of a pediatric surgeon, EPAs can provide a framework for measuring trainee progress and ensuring that they possess the necessary skills to perform their duties independently. Challenges in implementing EPAs in pediatric surgery include the need for consensus on the definition and scope of EPAs, as well as the establishment of appropriate assessment and evaluation tools.

CONCLUSION

The use of EPAs in pediatric surgery education and training has the potential to enhance the training and ultimately the quality of care delivered to pediatric surgical patients. The development of appropriate assessment and evaluation tools and frameworks will be crucial in implementing EPAs effectively in this field.

TYPE OF STUDY

Review.

LEVEL OF EVIDENCE: 4

For which entrustable professional activities must medical students be prepared if unsupervised patient care without further training is an expectation? An international Global South study

INTRODUCTION

Training medical students requires objectives that are often translated into frameworks of competencies. Since the introduction of entrustable professional activities (EPAs), these tasks or 'units of professional practice', originally conceived for postgraduate training, define what residents must be prepared to do unsupervised. Nowadays, EPAs are also applied to undergraduate training, with the prospect of entering residency. However, in many countries, especially in the Global South, a substantial number of medical graduates will be working in healthcare with little or no supervision and no further training soon or ever. EPAs for these conditions cannot be copied from undergraduate medical education EPA frameworks in Global North countries.

METHODS

We conducted a generative investigation to identify and elaborate EPAs for Global South countries who must train students for unsupervised general practice. We included 39 medical educators from 13 Global South countries and 17 specialties in either one of two online focus group sessions using a nominal group technique (NGT) or as Delphi panel member. Results from the two NGT sessions were merged and fed into the two-round Delphi investigation.

RESULTS

A framework of 11 EPAs resulted, each with an elaborate description (specification, setting and limitations).

CONCLUSION

This framework of undergraduate medical education Global South EPAs differs in its nature and specifications from existing Global North EPAs. The authors do not pretend universality for all Global South countries with graduates who face expectations of unsupervised practice but present it to support countries that consider introducing the model of entrustable professional activities.

How to develop entrustable professional activities in a training dermatology program; practical tips for dermatology educators

Entrustable professional activities (EPAs) are units of professional practice that could be observed, assessed, monitored, documented, and entrusted. EPAs when entrusted, outline the physician's qualifications, and shape the scope of practice. This insight highlights the importance of development of EPAs in all medical specialties including dermatology to ensure the best standards for patient's care. Development of EPAs-based training program is considered a challenge for clinical educators. In this paper, we describe practical tips and reflections on our experience in developing EPAs in dermatology doctoral training program that could be a guide for dermatology educators to implement EPAs in dermatology training programs.

How Internal Medicine Clerkship Directors Are Using Entrustable Professional Activities: A National Survey Study

BACKGROUND

In the present assessment environment in undergraduate medical education at U.S. medical schools, the prevalence and implementation of Entrustable Professional Activities (EPAs) in internal medicine (IM) clerkships are not well understood.

OBJECTIVE

To describe the prevalence and approach to EPA use in U.S. IM clerkships.

DESIGN

Cross-sectional, nationally representative survey of core IM clerkship directors.

PARTICIPANTS

One-hundred forty IM clerkship directors at Liaison Committee on Medical Education-accredited U.S./U.S. territory-based allopathic medical schools with membership in the Clerkship Directors in Internal Medicine (CDIM) as of December 2022.

MAIN MEASURES

Use of EPAs in IM core clerkships, including use for grading, types of EPAs, use of supportive measures for assessment, and current validity frameworks.

KEY RESULTS

The survey response was 80% (112/140); two additional respondents completed the section on EPA use (n = 114). Approximately half of respondents (47%) reported their IM clerkship used EPAs. Among schools accredited after 1977, a higher percentage was associated with having incorporated EPAs (p = 0.03). The Association of American Medical Colleges Core EPAs for Entering Residency (CEPAER) was the most common framework used by Clerkship Directors (CDs) for developing EPAs (55%). Most CDs (56%) used EPAs for both formative and summative assessments, and approximately half of CDs (48%) used EPAs for a portion of the final grade determination. CDs who used EPAs were no more likely to report efforts to ensure the validity of assessment, the use of faculty development, or that written assessments were a valid measure of students' performance compared to those who did not use EPAs.

CONCLUSIONS

Although EPAs have experienced substantial uptake in the IM clerkship and contribute to formative and summative assessment of learners, their use does not appear to be associated with enhanced efforts to obtain validity information.

Next steps for workplace-based assessments of entrustable professional activities

BACKGROUND

Florida International University Herbert Wertheim College of Medicine (FIU-HWCOM) participated in the AAMC Core Entrustable Professional Activities (EPA) implementation pilot. Entrustment decision processes based on data from workplace-based assessments (WBAs) were piloted. Outcomes illustrated challenges including variability across EPAs with regards to learner level alignment and feasibility of data collection in the form of WBAs. In addition, students reported discomfort requesting WBA completion by preceptors and dissatisfaction with associated feedback.

APPROACH

To guide future directions, we conducted a survey of third-year students to better understand their experience with and perceptions of WBAs used to evaluate EPAs at FIU-HWCOM.

EVALUATION

Survey response was 96% (n = 107/112). Most (84%) reported that WBAs were not valuable to their development and that preceptors often did not complete WBAs in a timely fashion. Many (47%) reported not receiving verbal feedback. Most students (78%) used language in written responses demonstrating confusion between the EPAs and the WBAs used to assess them.

IMPLICATIONS

The use of WBAs to assess EPAs did not have its intended impact at FIU-HWCOM. For future classes, WBA forms will consist of paper cards with questions directly assessing performance of skills aligned with EPAs 1, 5 and 6 only. To continue to promote feedback, students will be required to collect WBAs on all clerkships, but the number of required WBAs will be less than prior and no entrustment decisions will be made.

Implementation and faculty perception of outpatient medical student workplace-based assessments

BACKGROUND

There is growing interest in use of entrustable professional activity (EPA)-grounded workplace-based assessments (WBAs) to assess medical students through direct observation in the clinical setting. However, there has been very little reflection on how these tools are received by the faculty using them to deliver feedback. Faculty acceptance of WBAs is fundamentally important to sustained utilisation in the clinical setting, and understanding faculty perceptions of the WBA as an adjunct for giving targeted feedback is necessary to guide future faculty development in this area.

APPROACH

Use of a formative EPA-grounded WBA was implemented in the ambulatory setting during the paediatrics clerkship following performance-driven training and frame-of-reference training with faculty. Surveys and semi-structured interviews with faculty members explored how faculty perceived the tool and its impact on feedback delivery.

EVALUATION

Faculty reported providing more specific, task-oriented feedback following implementation of the WBA, as well as greater timeliness of feedback and greater satisfaction with opportunities to provide feedback, although these later two findings did not reach significance. Themes from the interviews reflected the benefits of WBAs, persistent barriers to the provision of feedback and suggestions for improvement of the WBA.

IMPLICATIONS

EPA-grounded WBAs are feasible to implement in the outpatient primary care setting and improve feedback delivery around core EPAs. The WBAs positively impacted the way faculty conceptualise feedback and provide learners with more actionable, behaviour-based feedback. Findings will inform modifications to the WBA and future faculty development and training to allow for sustainable WBA utilisation in the core clerkship.

Internal Medicine Acting Internship Trends in Rotation Structure and Student Responsibilities: Results from a 2023 National Survey

BACKGROUND

The acting internship (AI) in internal medicine plays a key role in the transition from medical school to residency. While there have been recent changes in medical education including a pass/fail USMLE Step 1 and increasing use of competency-based assessment, there has not been a large survey of the state of the AI in many years.

OBJECTIVE

To assess the current landscape of the internal medicine AI and identify areas in need of standardization.

DESIGN

This was a voluntary online survey of medical schools in the United States (U.S.).

PARTICIPANTS

Course directors of the AI rotation at U.S. medical schools.

MAIN MEASURES

Number of AI rotations required for graduation, length of AI rotation, types of services allowed for AI, clinical responsibilities of students, curricular components.

KEY RESULTS

Response rate was 50.7% (71/140 LCME accredited schools). All responding institutions require at least one AI for graduation, with nearly all schools integrating students into resident teaching teams, and almost half also allowing AI students to work on hospitalist services. Students carry 3-4 patients per day on average with a maximum of 5-6 in most institutions. Students are responsible for most aspects of patient care including notes, orders, interprofessional communication, and transitions of care. Night call or night float responsibilities are infrequently required. The structured curriculum published by AAIM is used by only 41% of schools.

CONCLUSIONS

The internal medicine AI continues to be a staple in the medical school experience, but there is variation in the structure, curriculum, and expectations on the rotation. Opportunities exist to improve standardization of the AI experience and expectations to better prepare medical students for the transition from medical school to residency.

How well do workplace-based assessments support summative entrustment decisions? A multi-institutional generalisability study

BACKGROUND

Assessment of the Core Entrustable Professional Activities for Entering Residency requires direct observation through workplace-based assessments (WBAs). Single-institution studies have demonstrated mixed findings regarding the reliability of WBAs developed to measure student progression towards entrustment. Factors such as faculty development, rater engagement and scale selection have been suggested to improve reliability. The purpose of this investigation was to conduct a multi-institutional generalisability study to determine the influence of specific factors on reliability of WBAs.

METHODS

The authors analysed WBA data obtained for clerkship-level students across seven institutions from 2018 to 2020. Institutions implemented a variety of strategies including selection of designated assessors, altered scales and different EPAs. Data were aggregated by these factors. Generalisability theory was then used to examine the internal structure validity evidence of the data. An unbalanced cross-classified random-effects model was used to decompose variance components. A phi coefficient of >0.7 was used as threshold for acceptable reliability.

RESULTS

Data from 53 565 WBAs were analysed, and a total of 77 generalisability studies were performed. Most data came from EPAs 1 (n = 17 118, 32%) 2 (n = 10 237, 19.1%), and 6 (n = 6000, 18.5%). Low variance attributed to the learner (<10%) was found for most (59/77, 76%) analyses, resulting in a relatively large number of observations required for reasonable reliability (range = 3 to >560, median = 60). Factors such as DA, scale or EPA were not consistently associated with improved reliability.

CONCLUSION

The results from this study describe relatively low reliability in the WBAs obtained across seven sites. Generalisability for these instruments may be less dependent on factors such as faculty development, rater engagement or scale selection. When used for formative feedback, data from these instruments may be useful. However, such instruments do not consistently provide reasonable reliability to justify their use in high-stakes summative entrustment decisions.

A realist synthesis of prospective entrustment decision making by entrustment or clinical competency committees

INTRODUCTION

The real-world mechanisms underlying prospective entrustment decision making (PEDM) by entrustment or clinical competency committees (E/CCCs) are poorly understood. To advance understanding in this area, the authors conducted a realist synthesis of the published literature to address the following research question: In E/CCC efforts to make defensible prospective entrustment decisions (PEDs), what works, for whom, under what circumstances and why?

METHODS

Realist work seeks to understand the contexts (C), mechanisms (M) and outcomes (O) that explain how and why things work (or do not). In the authors' study, contexts included individual E/CCC members, E/CCC structures and processes, and training programmes. The outcome (i.e. desired outcome) was a PED. Mechanisms were a substantial focus of the analysis and informed the core findings. To define a final corpus of 52 included papers, the authors searched four databases, screened all results from those searches and performed a full-text review of a subset of screened papers. Data extraction focused on developing context-mechanism-outcome configurations from the papers, which were used to create a theory for how PEDM leads to PEDs.

RESULTS

PEDM is often driven by default (non-deliberate) decision making rather than a deliberate process of deciding whether a trainee should be entrusted or not. When defaulting, some E/CCCs find red flags that sometimes lead to being more deliberate with decision making. E/CCCs that seek to be deliberate describe PEDM that can be effortful (when data are insufficient or incongruent) or effortless (when data are robust and tell a congruent story about a trainee). Both information about trainee trustworthiness and the sufficiency of data about trainee performance influence PEDM. Several moderators influence what is considered to be sufficient data, how trustworthiness data are viewed and how PEDM is carried out. These include perceived consequences and associated risks, E/CCC member trust propensity, E/CCC member personal knowledge of and experience with trainees and E/CCC structures and processes.

DISCUSSION

PEDM is rarely deliberate but should be. Data about trainee trustworthiness are foundational to making PEDs. Bias, equity and fairness are nearly absent from the papers in this synthesis, and future efforts must seek to advance understanding and practice regarding the roles of bias, equity and fairness in PEDM.

Integration of Disability Awareness Improves Medical Students' Attitudes Toward People with Disabilities

Persons with disabilities (PWD) often require special accommodations and more comprehensive healthcare yet frequently have encounters with physicians who are unprepared to provide tailored and complete care. A multipronged disability awareness curriculum for second-year medical students was implemented, including content on disability etiquette, patient-centered and interprofessional learning sessions for individuals with physical disabilities and intellectual and developmental disabilities, and a debriefing session with physiatrists. The objective of this study was to utilize a mixed methods approach to evaluate the disability awareness curriculum in undergraduate medical education (UME). Assessment was conducted using course evaluations, pre- and post-surveys including the Attitudes and Perspectives Towards Persons with Disabilities (APPD) scale and Multidimensional Attitudes Scale Toward Persons with Disabilities (MAS), and student focus groups. The mean scores from both the APPD (2.11 ± 0.43 pre-score vs. 1.7 ± 0.39 post-score) and MAS (2.45 ± 0.43 pre-score vs. 2.25 ± 0.55 post-score) indicate the curriculum improved medical students' attitudes toward PWD (p < 0.05), with lower numbers representing more favorable attitudes. After completing the curriculum, medical students' attitudes were comparable to those of doctor of physical therapy (DPT) students. Qualitative analysis from focus groups highlighted four major themes: education, comfort level, impact on future practice, and disability differences. This curriculum has potential as a valuable framework for delivering effective disability education to medical students to prepare future physicians to serve PWD and their unique needs. It meets core competencies, provides an opportunity to learn in interprofessional environments, and integrates PWD into the educational process.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-024-02004-0.

Faculty and Resident Perspectives on the Implementation of Entrustable Professional Activities in General Surgery Residency

OBJECTIVE

This study aimed to identify what best practices facilitate implementation of Entrustable Professional Activities (EPAs) into surgical training programs.

DESIGN

This is a mixed methods study utilizing both survey data as well as semi-structured interviews of faculty and residents involved in the American Board of Surgery (ABS) EPA pilot study.

SETTING

From 2018 to 2020, the ABS conducted a pilot that introduced five EPAs across 28 general surgery training programs.

PARTICIPANTS

All faculty members and residents at the 28 pilot programs were invited to participate in the study.

RESULTS

About 117 faculty members and 79 residents responded to the survey. The majority of faculty (81%) and residents (66%) felt that EPAs were useful and were a valuable addition to training. While neither group felt that EPAs were overly time consuming to complete, residents did report difficulty incorporating them into their daily workflow (44%). Semi-structured interviews found that programs that focused on faculty and resident -development and utilized frequent reminders about the importance and necessity of EPAs tended to perform better.

CONCLUSIONS

EPA implementation is feasible in general surgery training programs but requires significant effort and engagement from all levels of program personnel. As EPAs are implemented by the ABS nationally a focus on resident and faculty development will be critical to success.

Lessons From Organic Chemistry: The Case for Considering Both High Standards and Equity in Assessment

ABSTRACT

In this commentary, the authors explore the tension of balancing high performance standards in medical education with the acceptability of those standards to stakeholders (e.g., learners and patients). The authors then offer a lens through which this tension might be considered and ways forward that focus on both patient outcomes and learner needs.In examining this phenomenon, the authors argue that high performance standards are often necessary. Societal accountability is key to medical education, with the public demanding that training programs prepare physicians to provide high-quality care. Medical schools and residency programs, therefore, require rigorous standards to ensure graduates are ready to care for patients. At the same time, learners' experience is important to consider. Making sure that performance standards are acceptable to stakeholders supports the validity of assessment decisions.Equity should also be central to program evaluation and validity arguments when considering performance standards. Currently, learners across the continuum are variably prepared for the next phase in training and often face inequities in resource availability to meet high passing standards, which may lead to learner attrition. Many students who face these inequities come from underrepresented or disadvantaged backgrounds and are essential to ensuring a diverse medical workforce to meet the needs of patients and society. When these students struggle, it contributes to the leaky pipeline of more socioeconomically and racially diverse applicants.The authors posit that 4 key factors can balance the tension between high performance standards and stakeholder acceptability: standards that are acceptable and defensible, progression that is time variable, requisite support structures that are uniquely tailored for each learner, and assessment systems that are equitably designed.

"Doing well": Intraoperative entrustable professional activity assessments provided limited technical feedback

Background

Entrustable Professional Activities (EPAs) allow for the assessment of specific, observable, essential tasks in medical education. Since being developed in non-surgical fields, EPA assessments have been implemented in surgery to explore intraoperative entrustment. However, assessment burden is a significant problem for faculty, and it is unknown whether EPA assessments enable formative technical feedback. EPAs' formative utility could inform how surgical programs facilitate technical feedback for trainees. We aimed to assess the extent to which narrative comments provided through the Fellowship Council (FC) EPA assessments contained technical feedback.

Methods

The FC previously collected EPA assessments for subspecialty surgical fellows from September 2020 to October 2022. Two raters reviewed assessments' narrative comments for inclusion of each skill area that makes up part of the Objective Structured Assessment of Technical Skills (OSATS). A third rater reconciled discrepant ratings.

Results

During the study period, there were 3302 completed EPA assessments, including 1191 fellow self-assessments, 1124 faculty assessments, and 987 assessments without an identified assessor role. We found that assessments' narrative comments related to a median of two of the seven OSATS areas (IQR:1-2). There were no comments relevant to any of the seven OSATS areas in 16.0 % of all assessments.

Conclusions

In this review of narrative comments for EPA assessments from the FC, we found that limited technical feedback of the kind included in the OSATS was provided in many assessments. These results suggest benefit to adjusting the EPA form, enhancing faculty development, or continuing additional types of targeted technical assessment intraoperatively.

Key message

This analysis of narrative comments from fellowship EPA assessments showed that many assessments included limited technical feedback. To allow for continued technical feedback for fellows, these results highlight the need for further refinements of the EPA assessment form, additional faculty development, or ongoing use of other types of technical assessment.

Twelve tips for creating a longitudinal quality improvement and safety education for early health professions students

Despite the numerous calls for integrating quality improvement and patient safety (QIPS) curricula into health professions education, there are limited examples of effective implementation for early learners. Typically, pre-clinical QIPS experiences involve lectures or lessons that are disconnected from the practice of medicine. Consequently, students often prioritize other content they consider more important. As a result, they may enter clinical settings without essential QIPS skills and struggle to incorporate these concepts into their early professional identity formation. In this paper, we present twelve tips aimed at assisting educators in developing QIPS education early in the curricula of health professions students. These tips address various key issues, including aligning incentives, providing longitudinal experiences, incorporating real-world care outcomes, optimizing learning environments, communicating successes, and continually enhancing education and care delivery processes.

Moving Beyond the Dichotomous Assessment of Professionalism in the Internal Medicine Clerkship: Results of a National Survey of Clerkship Directors

PURPOSE

This study examines how internal medicine clerkship faculty and leadership conceptualize professionalism and professional behaviors and attitudes, identifies whether and how faculty use metrics to assess professionalism and factor it into clerkship grades, and describes barriers that prevent faculty from feeling prepared to support the development of professional behaviors in students.

METHOD

The Clerkship Directors in Internal Medicine opened a call for thematic survey section proposals to its physician-faculty members, blind-reviewed all submissions, and selected 4 based on internal medicine clinical clerkship training experience relevance. The survey launched on October 5 and closed on December 7, 2021. Data were analyzed using descriptive statistics.

RESULTS

Of 137 core clerkship directors (CDs) at Liaison Committee on Medical Education-accredited medical schools, 103 (75.2%) responded to the survey. Of 102 respondents (1 nonrespondent), 84 (82.4%) identified professional behavior lapses in involvement and 60 (58.8%) identified introspection lapses. Of 103 respondents, 97 (94.2%) reported that their clerkships ask clinical faculty and residents to formally evaluate professionalism, and 64 (62.1%) reported that they factor professionalism assessments into final clerkship grades. CDs reported multiple barriers to addressing professionalism directly with students, including logistical barriers, professionalism assessment subjectivity concerns, and the possible adverse effect of an unprofessional label for students.

CONCLUSIONS

Professionalism assessment and remediation in medical education currently center on a deficit model that seeks to identify and remediate professionalism lapses, rather than a developmental model that seeks to nurture growth. This dichotomous characterization of behaviors as professional or unprofessional limits assessment and can adversely affect the learning environment. The authors propose a shift to a developmental model that considers professionalism as a continuous process parallel to the acquisition of clinical skills and medical knowledge.

Exploration of validity evidence for core residency entrustable professional activities in Chinese pediatric residency

Introduction

This study seeks to explore validity and reliability evidence for core residency entrustable professional activities (CR-EPAs) that were developed by Peking University First Hospital (PKUFH) in 2020.

Methods

A prospective cohort study was conducted in PKUFH. Trainers (raters) assessed pediatric residents on CR-EPAs over 1 academic year, bi-annually. Critical components within a validity evidence framework were examined: response process (rater perceptions), the internal structure (reliability and contributions of different variance sources), and consequences (potential use of a cutoff score).

Results

In total, 37 residents were enrolled, and 111 and 99 trainers' ratings were collected in Fall 2020 and Spring 2021, respectively. For rater perceptions, all the raters considered CR-EPAs highly operational and convenient. In all ratings, individual EPAs correlate with total EPA moderately, with Spearman correlation coefficients spanning from 0.805 to 0.919. EPA 2 (select and interpret the auxiliary examinations), EPA 5 (prepare and complete medical documents), EPA 6 (provide an oral presentation of a case or a clinical encounter), and EPA 7 (identify and manage the general clinical conditions) were EPAs correlated with other EPAs significantly. The results of the generalizability theory indicated that the variability due to residents is the highest (nearly 78.5%), leading to a large size of the reliability estimates. The matching results indicate that the lowest error locates at 5.933.

Conclusion

The rating showed good validity and reliability. The ratings were reliable based on G-theory. CR-EPAs have a magnificent internal structure and have promising consequences. Our results indicate that CR-EPAs are a robust assessment tool in workplace-based training in a carefully designed setting.

Applying self-determination theory to stem medical schools' clinical teacher sustainability crisis

THE PROBLEM

Medical schools require highly skilled and committed clinical faculty to teach, assess, supervise and mentor students' clinical care. Medical education is facing a crisis in recruiting and sustaining these clinical teachers. Faced with multiple demands and responsibilities in fast-paced clinical environments, teachers may not have the time, resources or stamina to sustain these critical roles. Medical school leaders must commit to and provide structures and processes to attract, sustain and retain clinical teachers.

CONCEPTUAL FRAMEWORK

The authors use the lens of self-determination theory to frame approaches to support teacher sustainability. Self-determination theory describes sources of human motivation. The theory and its evidence base characterise three human psychological needs: autonomy, competence, and relatedness. This theory can bridge individual psychological and institutional leadership perspectives to help medical school leaders anticipate and respond to their clinical teachers' needs. The authors propose three practical steps: practices to advance employee-centredness, processes to align individual and institutional values, and restructuring education to support clinical teachers' needs alongside student and patient needs. The authors describe limitations to this relational approach that focuses on leadership actions and consider individual agency as another key factor for sustainability.

DISCUSSION

Medical school leaders can develop and apply theory-driven approaches to advance sustainability. Sustainability now and in the future requires careful attention to the needs of clinical teachers and to their relationships with and within medical schools.

Entrustable professional activities as a training and assessment framework in undergraduate medical education: A case study of a multi-institutional pilot

In 2014, the Association of American Medical Colleges (AAMC) published 13 Core Entrustable Professional Activities (EPAs) that graduating students should be able to perform with indirect supervision when entering residency. A ten-school multi-year pilot was commissioned to test feasibility of implementing training and assessment of the AAMC's 13 Core EPAs. In 2020-21, a case study was employed to describe pilot schools' implementation experiences. Teams from nine of ten schools were interviewed to identify means and contexts of implementing EPAs and lessons learned. Audiotapes were transcribed then coded by investigators using conventional content analysis and a constant comparative method. Coded passages were organized in a database and analyzed for themes. Consensus among school teams regarding facilitators of EPA implementation included team commitment to piloting EPAs; agreement that: proximal EPA adoption with curriculum reform facilitates EPA implementation; EPAs 'naturally fit' in clerkships and provided opportunity for schools to reflect on and adjust curricula and assessments; and inter-school collaboration bolstered individual school progress. Schools did not make high-stakes decisions about student progress (e.g., promotion, graduation), yet EPA assessment results complemented other forms of assessment in providing students with robust formative feedback about their progress. Teams had varied perceptions of school capability to implement an EPA framework, influenced by various levels of dean involvement, willingness, and capability of schools to invest in data systems and provide other resources, strategic deployment of EPAs and assessments, and faculty buy-in. These factors affected varied pace of implementation. Teams agreed on the worthiness of piloting the Core EPAs, but substantial work is still needed to fully employ an EPA framework at the scale of entire classes of students with enough assessments per EPA and with required data validity/reliability. Recommendations stemming from findings may help inform further implementation efforts across other schools adopting or considering an EPA framework.

Implementation of entrustable professional activities in multiple surgical residencies: A quality improvement approach

BACKGROUND

The COVID-19 pandemic decreased the operative case volume for surgical residents. Our institution implemented Entrustable Professional Activities (EPAs) in all core surgical training programs to document the competency of graduating residents. Continuation of this project aimed to improve implementation.

METHODS

This project occurred at a large academic center with eight surgical specialties during the 2020-21 (Year 1) and 2021-22 (Year 2) academic years. Each specialty chose five EPAs, and residents were asked to obtain three micro-assessments per EPA. After the initial pilot year, program directors were surveyed regarding perceptions of EPA utility and barriers to implementation.

RESULTS

Seventy senior residents completed 732/906 (80.8%) micro-assessments. Of these, 99.6% were deemed practice ready. Total micro-assessment completion rates in four specialties, four specific EPAs (including one EPA identified "at risk" due to the COVID-19 pandemic), and overall were significantly higher in Year 2 than Year 1 (p ​< ​0.05) CONCLUSIONS: Implementing EPAs in all core surgical specialties at an institution is achievable, though expectedly initially imperfect. An ongoing quality collaborative initiative focused on barriers to implementation can improve completion rates.

Competency-Based Medical Education in a Norm-Referenced World: A Root Cause Analysis of Challenges to the Competency-Based Paradigm in Medical School

Competency-based medical education (CBME) requires a criterion-referenced approach to assessment. However, despite best efforts to advance CBME, there remains an implicit, and at times, explicit, demand for norm-referencing, particularly at the junction of undergraduate medical education (UME) and graduate medical education (GME). In this manuscript, the authors perform a root cause analysis to determine the underlying reasons for continued norm-referencing in the context of the movement toward CBME. The root cause analysis consisted of 2 processes: (1) identification of potential causes and effects organized into a fishbone diagram and (2) identification of the 5 whys. The fishbone diagram identified 2 primary drivers: the false notion that measures such as grades are truly objective and the importance of different incentives for different key constituents. From these drivers, the importance of norm-referencing for residency selection was identified as a critical component. Exploration of the 5 whys further detailed the reasons for continuation of norm-referenced grading to facilitate selection, including the need for efficient screening in residency selection, dependence upon rank-order lists, perception that there is a best outcome to the match, lack of trust between residency programs and medical schools, and inadequate resources to support progression of trainees. Based on these findings, the authors argue that the implied purpose of assessment in UME is primarily stratification for residency selection. Because stratification requires comparison, a norm-referenced approach is needed. To advance CBME, the authors recommend reconsideration of the approach to assessment in UME to maintain the purpose of selection while also advancing the purpose of rendering a competency decision. Changing the approach will require a collaboration between national organizations, accrediting bodies, GME programs, UME programs, students, and patients/societies. Details are provided regarding the specific approaches required of each key constituent group.

Listen Before You Auscultate: An Active-Learning Approach to Bedside Cardiac Assessment

Introduction

Bedside cardiac assessment (BCA) is deficient across a spectrum of noncardiology trainees. Learners not taught BCA well may become instructors who do not teach well, creating a self-perpetuating problem. To improve BCA teaching and learning, we developed a high-quality, patient-centered curriculum for medicine clerkship students that could be flexibly implemented and accessible to other health professions learners.

Methods

With a constructivist perspective, we aligned learning goals, activities, and assessments. The curriculum used a "listen before you auscultate" framework, capturing patient history as context for a six-step, systematic approach. In the flipped classroom, short videos and practice questions preceded two 1-hour class activities that integrated diagnostic reasoning, pathophysiology, physical diagnosis, and reflection. Activities included case discussions, jugular venous pressure evaluation, heart sound competitions, and simulated conversations with patients. Two hundred sixty-eight students at four US and international medical schools participated. We incorporated feedback, performed thematic analysis, and assessed learners' confidence and knowledge.

Results

Low posttest data capture limited quantitative results. Students reported increased confidence in BCA ability. Knowledge increased in both BCA and control groups. Thematic analysis suggested instructional design strategies were effective and peer encounters, skills practice, and encounters with educators were meaningful.

Discussion

The curriculum supported active learning of day-to-day clinical competencies and promoted professional identity formation alongside BCA ability. Feedback and increased confidence on the late-clerkship posttest suggested durable learning. We recommend approaches to confirm this and other elements of knowledge, skill acquisition, or behaviors and are surveying impacts on professional identity formation-related constructs.

Understanding medical student paths to communication skills expertise using latent profile analysis

PURPOSE

To describe patterns of clinical communication skills that inform curriculum enhancement and guide coaching of medical students.

MATERIALS AND METHODS

Performance data from 1182 consenting third year medical students in 9 cohorts (2011-2019), on a 17-item Clinical Communication Skills Assessment Tool (CCSAT) completed by trained Standardized Patients as part of an eight case high stakes Comprehensive Clinical Skills Exam (CCSE) were analyzed using latent profile analysis (LPA). Assessment domains included: information gathering (6 items), relationship development (5 items), patient education (3 items), and organization/time management (3 items). LPA clustered learners with similar strength/weakness into profiles based on item response patterns across cases. One-way analysis of variance (ANOVA) assessed for significant differences by profile for CCSAT items.

RESULTS

Student performance clustered into six profiles in three groups, high performing (HP1 and HP2-Low Patient Education, 15.7%), average performing (AP1 and AP2-Interrupters, 40.9%), and lower performing profiles (LP1-Non-interrupters and LP2, 43.4%) with adequate model fit estimations and similar distribution in each cohort. We identified 3 CCSAT items that discriminated among learner's skill profiles.

CONCLUSION

Clinical communication skill performance profiles provide nuanced, benchmarked guidance for curriculum improvement and tailoring of communication skills coaching.

Sexual and reproductive health teaching in undergraduate medical education: A narrative review

OBJECTIVE

Contraception use, undesired pregnancy, and abortion care are common medical experiences that most physicians will encounter for their patients. Future physicians should therefore have some formal education on these topics. In this narrative review, we focused on how medical education approaches these sometimes polarizing yet fundamental topics.

METHODS

We assessed the published literature on sexual and reproductive health education in undergraduate medical education from 2000 to 2021, screening 868 articles and including 52 articles. We included articles that discussed contraception, emergency contraception, pregnancy options counseling, abortion, and ethics related to sexual and reproductive health.

RESULTS

Included studies came from 14 countries and described both preclinical and clinical education. Studies assessed medical student knowledge, the effectiveness of educational interventions and medical school faculty perspectives on sexual and reproductive health curricula. Medical educators have employed a variety of approaches to teach sexual and reproductive health including simulation, objective structured clinical examinations, team-based learning, narrative medicine, online modules, and flipped classrooms.

CONCLUSION

Students generally received sexual and reproductive health education favorably, demonstrating increased knowledge and comfort with these topics after an education session. Studies also identified curricular gaps and deficiencies in student knowledge, which may indicate a need for improved and consistent medical school education on contraception and abortion.

Would I Trust or Will I Trust? The Gap between Entrustment Determinations and Entrustment Decisions for Trainees in Pharmacy and Other Health Professions

Entrustable Professional Activities (EPAs) and entrustment decision making are rapidly becoming mainstream in competency-based education in the health professions. EPAs are the units of professional practice to entrust graduates with once they have developed the required competencies. They were conceived to enable a gradual increase in professional autonomy during training, by allowing trainees to practice activities in which they have demonstrated they have mastered well, with decreasing supervision. However, practicing health care unsupervised generally requires licensure. The question for pharmacy education, as well as for undergraduate medical education, is can students be given any autonomy in practice, even when they have fully mastered an EPA yet remain unlicensed? While entrustment decisions for licensed practitioners have autonomy consequences, some educators in undergraduate programs speak of 'entrustment determinations', to avoid decisions about students that affect patient care, in other words saying, we would trust you, rather than we will trust you. However, graduating learners without the experience of responsibility and reasonable autonomy creates a gap with full practice responsibilities, which may jeopardize patient safety after training. What can programs do to retain the power of using EPAs while at the same time guarding patient safety?

Validity and Educational Impact of Clinical Entrustable Professional Activities Linked to the Nutrition Care Process for Work-Based Assessment of Entry-Level Dietetics Students: Evaluation of a 3-Year Implementation Study in Australia

BACKGROUND

Entrustable professional activities (EPAs) describe key workplace activities and are increasingly being used for student work-based assessment. An EPA-based work-based assessment tool offers potential to increase understanding of dietetics student skill development and opportunity for standardised work-based student assessment.

OBJECTIVE

To determine construct validity and educational impact of an EPA work-based assessment tool for dietetics placement students in clinical settings.

DESIGN

Using a time series design, supervisor and student self-assessment data collected from an EPA-based assessment tool from three cohorts of dietetics students and supervisors from 2017-2019 was analyzed.

PARTICIPANTS AND SETTING

Dietetics students (n = 145) from an accredited dietetic training program in Australia and affiliated metropolitan and rural hospitals.

MAIN OUTCOME MEASURES

Construct validity was determined through analysis of supervisor evaluation of student performance against EPAs over time. Educational impact was determined through comparing student performance across EPAs to predicted milestones to identify areas of least entrustment.

STATISTICAL ANALYSES PERFORMED

Student performance over time and differences between student self-assessment and supervisor assessment for each EPA were investigated using Wilcoxon signed-rank tests and linear mixed-model analysis. Descriptive statistics summarised student performance against each EPA.

RESULTS

Performance significantly increased over time in 35 out of 37 EPAs. Significant differences between supervisor and student self-assessment were evident in 9 out of 37 EPAs. Dietetics student performance varied across EPAs with 88.2% of students achieving entrustment for nutrition management EPAs compared with 100% for professionalism EPAs.

CONCLUSIONS

The tool's construct validity was established. EPAs identified areas of Nutrition Care Process skills development requiring additional support. Data collected by the tool can inform teaching.

Master clinical teachers and personalised learning

Clinical history taking and physical examination are two of the most important competencies of physicians. In addition to informing diagnoses, these activities build rapport and establish relationships between caregivers and patients. Despite this, emphasis on the assessment of bedside clinical skills is declining. To prepare our students for clinical work, we began a clinical competency, personalised teaching programme in which students perform a history and physical examination in front of a master clinical teacher (MCT) approximately every 2 weeks throughout their core clerkship year. The MCT works with the student in a clinical encounter, providing personalised bedside instruction on all features of being a clinician including bedside manner, history-taking skills, physical examination skills, and clinical reasoning. The MCT then provides an assessment of student's competency development and gives feedback to the student about what they do well and where they have opportunities for growth. Assessment data are collected and tracked longitudinally across the clerkship phase to ensure that each student is progressing developmentally. With over 6000 observations of student performance, we are able to discern competency development and growth over time. We can identify if a student is not improving as expected during their clerkship phase and intervene by providing extra practice and training. This core clerkship teaching programme has been well received by both students and instructors and has led us to pilot this approach during the post-clerkship phase of our medical training.

Exploring training needs of newly graduated medical doctors to inform the undergraduate simulation-based curriculum: a national Delphi consensus study

PURPOSE

Mastering technical procedures is a key component in succeeding as a newly graduated medical doctor and is of critical importance to ensure patient safety. The efficacy of simulation-based education has been demonstrated but medical schools have different requirements for undergraduate curricula. We aimed to identify and prioritize the technical procedures needed by newly graduated medical doctors.

METHODS

We conducted a national needs assessment survey using the Delphi technique to gather consensus from key opinion leaders in the field. In the first round, a brainstorm was conducted to identify all potential technical procedures. In the second round, respondents rated the need for simulation-based training of each procedure using the Copenhagen Academy for Medical Education and Simulation Needs Assessment Formula (CAMES-NAF). The third round was a final elimination and prioritization of the procedures.

RESULTS

In total, 107 experts from 21 specialties answered the first round: 123 unique technical procedures were suggested. Response rates were 58% and 64% in the second and the third round, respectively. In the third round, 104 procedures were eliminated based on the consensus criterion, and the remaining 19 procedures were included and prioritized. The top five procedures were: (i) insert peripheral intravenous catheter, (ii) put on personal protection equipment, (iii) perform basic airway maneuvers, (iv) perform basic life support, and (v) perform radial artery puncture.

CONCLUSION

Based on the Delphi process a final list of 19 technical procedures reached expert consensus to be included in the undergraduate curriculum for simulation-based education.

Competency-Based Workforce Development and Education in Global Oncology

The healthcare workforce plays a pivotal role in cancer care delivery, leadership, policy, education, and research in complex cancer systems. To ensure quality and relevance, health professionals must have the necessary competencies to deliver patient-centered and efficient care, coupled with the ability to work in teams and manage health resources wisely. This paper aims to review the concept of competency-based medical education (CBME) in the context of oncology to provide insights and guidance for those interested in adopting or adapting competency-based education in training programs. The results of a scoping review of CBME in oncology are presented here to describe the current status of CBME in oncology. The literature describing the implementation and evaluation of CBME in oncology training programs for medical professionals internationally is summarized and key themes identified to provide practical guidance for educators. Further, the paper identifies critical competencies for oncology education and training globally and presents recommendations and opportunities for collaboration in competency-based education and training in oncology. The authors argue for increased global collaboration and networking in the realm of CBME to facilitate the establishment of a competent global cancer care workforce.

Advancements in Surgical Performance Evaluation and Feedback

PURPOSE OF REVIEW

How today's urology trainees acquire surgical skills has changed dramatically due to multiple forces placing strain on the graduate medical education mission. The development of workplace-based assessments that deliver feedback while capturing performance data has led to a paradigm shift toward individualized learning.

RECENT FINDINGS

Delivering feedback that drives surgical skill development requires the educator to provide a meaningful assessment of the learner after an operative experience. Workplace-based assessment involves direct observation of routine clinical practice and has become a central component of competency-based medical education. Urology has the chance to fully embrace competency-based medical education, employing robust feedback mechanisms and workplace-based assessments. We must first define what it means to be a proficient urologist and design an assessment system that captures this collective sentiment. This can only be done through effective engagement and collaboration with stakeholders across our specialty.

Using entrustable professional activities to better prepare students for their postgraduate medical training: A medical student's perspective

THE PROBLEM

Medical students graduate underprepared for postgraduate medical training despite years of classroom and clinical training. In this article, a medical student shares her personal perspectives on three factors contributing to this problem in undergraduate medical education: students' peripheral roles in the clinical environment impede learning, students receive inadequate feedback, and assessments do not measure desired learning outcomes.

A SOLUTION

The authors describe how using entrustable professional activities (EPAs) could address these issues and promote students' clinical engagement by clarifying their roles, providing them with frequent and actionable feedback, and aligning their assessments with authentic work. These factors combined with grading schemes rewarding improvement could contribute to a growth mindset that reprioritizes clinical skill acquisition. The authors explore how medical schools have begun implementing the EPA framework, highlight insights from these efforts, and describe barriers that must be addressed.

THE FUTURE

Incorporating EPAs into medical school curricula could better prepare students for postgraduate training while also alleviating issues that contribute to student burnout by defining students' roles, improving feedback, and aligning assessments with desired learning outcomes.

Core Entrustable Professional Activities for Entering Residency: A National Survey of Graduating Medical Students' Self-Assessed Skills by Specialty

BACKGROUND

The Association of American Medical Colleges described 13 Core Entrustable Professional Activities (EPAs) that graduating students should be prepared to perform under indirect supervision on day one of residency. Surgery program directors recently recommended entrustability in these Core EPAs for incoming surgery interns. We sought to determine if graduating students intending to enter surgery agreed they had the skills to perform these Core EPAs.

STUDY DESIGN

Using de-identified, individual-level data collected from and about 2019 Association of American Medical Colleges Graduation Questionnaire respondents, latent profile analysis was used to group respondents based on their self-assessed Core EPAs skills' response patterns. Associations between intended specialty, among other variables, and latent profile analysis group were assessed using independent sample t -tests and chi-square tests and multivariable logistic regression methods.

RESULTS

Among 12,308 Graduation Questionnaire respondents, latent profile analysis identified 2 respondent groups: 7,863 (63.9%) in a high skill acquisition agreement (SAA) group and 4,445 (36.1%) in a moderate SAA group. Specialty was associated with SAA group membership (p < 0.001), with general surgery, orthopaedic surgery, and emergency medicine respondents (among others) overrepresented in the high SAA group. In the multivariable logistic regression models, each of anesthesiology, ophthalmology, pediatrics, psychiatry, and radiology (vs general surgery) specialty intention was associated with a lower odds of high SAA group membership.

CONCLUSION

Graduating students' self-assessed Core EPAs skills were higher for those intending general surgery than for those intending some other specialties. Our findings can inform collaborative efforts to ensure graduates' acquisition of the skills expected of them at the start of residency.

Competency-Based Frameworks in Medical School Education Programs: A Thematic Analysis of the Academic Medicine Snapshots, 2020

PURPOSE

Educational program objectives (EPOs) provide the foundation for a medical school's curriculum. In recent years, the Liaison Committee on Medical Education (LCME) endorsed an outcomes-based approach to objectives, to embrace the movement toward competency-based medical education (CBME). The purpose of this study was to explore the CBME frameworks used by medical schools in formulating their EPOs. A secondary aim was to determine factors related to the selection of specific frameworks.

METHOD

The authors performed a quantitative content analysis of entries to the 2020 Academic Medicine Snapshot. Publicly available data gathered included demographic features of each program (e.g., year founded, accreditation status, affiliation, etc.), participation in national medical education consortia, and presence of specific CBME frameworks identified in EPOs. Descriptive statistics were used to examine trends in frameworks used by medical schools. Bivariate comparisons between factors and frameworks were conducted using chi-square tests. Logistic regression was used to examine factors predicting use of more recently developed CBME frameworks.

RESULTS

A total of 135 institutions submitted Snapshots (RR = 88%). All institutions endorsed 1 or more CBME frameworks, with 37% endorsing 2 and 20% endorsing 3 or more. The most common was the Accreditation Council for Graduate Medical Education core competencies (63%). In addition to published frameworks, 36% of institutions developed their own competencies. Schools with pending LCME visits were 2.61 times more likely to use a more recently developed curricular framework, P = .022.

CONCLUSIONS

Medical schools in the United States have embraced the CBME movement through incorporation of competency-based frameworks in their EPOs. While it is encouraging that CBME frameworks have been integrated in medical school EPOs, the variability and use of multiple frameworks identifies the pressing need for a unified CBME framework in undergraduate medical education.

Developing and Mapping Entrustable Professional Activities with Saudi Meds Competency Framework: A Consensus Study

PURPOSE

This study aimed at developing a national consensus on entrustable professional activities (EPAs) for Saudi undergraduate medical education and mapping them with the "Saudi Meds" competency framework.

METHODS

A three phased approach was used. Phase 1 consisted of identifying and developing EPAs; Phase 2 consisted of building a national consensus on developed EPAs (validation process); and Phase 3 consisted of mapping the validated EPAs with the Saudi Meds competency framework. Nominal group and modified Delphi techniques were used to develop consensus on EPAs. Classical test theory-based item analysis was conducted to establish validity and reliability of finalized EPAs.

RESULTS

Fifteen expert medical educationists and 109 academic leaders from 23 medical schools participated in the validation process. The study achieved a consensus on 10 core EPAs with an overall reliability (Cronbach's Alpha) of 0.814. The item-total correlation ranged from 0.341 to 0.642.

CONCLUSION

This study results in a national consensus on generic, comprehensive and region-specific EPAs that have been mapped with Saudi Meds competency framework. Our study is the first step in the direction of facilitating EPA-based curricular reforms in Saudi medical schools.

Entrustable professional activities for Junior Brazilian Medical Students in community medicine

BACKGROUND

Entrustable professional activities (EPAs) have been defined to promote the workplace participation of undergraduate medical students, generally in the context of high-income countries with a focus on the secondary and tertiary health care sectors. These EPAs have limited applicability to training and health care contexts in low- to middle-income countries that have a focus on primary health care, for instance, the context of community medicine. The purpose of this article is to report the process and results of defining EPAs for undergraduate medical training in a community health care setting.

METHODS

A modified Delphi study was performed to develop EPAs for the training of medical students in community medicine during their first and second years of education at the Marília Medical School (FAMEMA), Brazil. The supervision level was operationalized in terms of a student's ability to perform the EPA autonomously in an effective and safe manner with supervision readily available on request. Panellists (9 physicians and 6 nurses) rated the completeness of the proposed list of EPAs and EPA categories on four-point Likert scales. The threshold for consensus among panellists was a mean content validity index of at least 80%.

RESULTS

Consensus was reached after two Delphi rounds, resulting in 11 EPAs for undergraduate medical education and training in community medicine. These EPAs were organized into three overarching EPA domains: integrality of care for individual health needs in all phases of the life cycle (5 EPAs), integrality of care for family health needs (3 EPAs), and integrality of care for community health needs (3 EPAs). For each EPA, descriptions of the following categories were created: title; specifications and limitations; conditions and implications of the entrustment decision; knowledge, skills, and attitudes; links to competencies; and assessment sources.

CONCLUSION

The resulting 11 EPAs for training medical students in community medicine expand the application of the EPA framework to both early undergraduate medical education and the context of primary health care. This report can support and guide other medical schools in their attempts to train students in primary health care contexts and to incorporate EPAs into their curricula.

[Contribution and limits of "OSCE", "long-case" and "global end-of-placement marking" as end-of-rotation assessment methods. Experience from two internal medicine wards]

INTRODUCTION

During placements, there is an opportunity to learn clinical skills and to assess their application. However, it represents two different goals. The validity of an end-of-placement assessment is questionable, as the medical competency is contextual. We decided to evaluate the contribution and limits of different assessment modalities as an end-of-placement assessment.

MATERIAL AND METHODS

Internal medicine clerks were assessed using the Mini-Cex grid by a structured objective clinical examination (OSCE), a long-case clinical examination (LCE) and a global end-of-placement marking (GEPM). Following these evaluations, students and teachers fulfilled an open questionnaire.

RESULTS

In 2021, 41 students and 16 teachers participated in the study. Physical examination was evaluated in 0%, 97% et 76% of cases during OSCE, LCE and GEPM, respectively; teaching skills were assessed for 100, 42 et 49% of students in OSCE, LCE and GEPM, respectively. As compared to OSCE, there was a perceived superiority of LCE regarding its formative value (P=0.07 and P=0.03) and its summative value (P=0.0007 and P=0.02), for students and teachers, respectively. Qualitative analysis highlights the breadth of clinical skills that could be assessed during OSCE stations. Integration into a team was an additional skill that could specifically be assessed during GEPM. GEPM could also take into account the progress made during placement.

CONCLUSION

Despite its subjectivity, LCE seemed to be the preferred modality for an end-of-rotation assessment.

Informatics in Undergraduate Medical Education: Analysis of Competency Frameworks and Practices Across North America

BACKGROUND

With the advent of competency-based medical education, as well as Canadian efforts to include clinical informatics within undergraduate medical education, competency frameworks in the United States have not emphasized the skills associated with clinical informatics pertinent to the broader practice of medicine.

OBJECTIVE

By examining the competency frameworks with which undergraduate medical education in clinical informatics has been developed in Canada and the United States, we hypothesized that there is a gap: the lack of a unified competency set and frame for clinical informatics education across North America.

METHODS

We performed directional competency mapping between Canadian and American graduate clinical informatics competencies and general graduate medical education competencies. Directional competency mapping was performed between Canadian roles and American common program requirements using keyword matching at the subcompetency and enabling competency levels. In addition, for general graduate medical education competencies, the Physician Competency Reference Set developed for the Liaison Committee on Medical Education was used as a direct means of computing the ontological overlap between competency frameworks.

RESULTS

Upon mapping Canadian roles to American competencies via both undergraduate and graduate medical education competency frameworks, the difference in focus between the 2 countries can be thematically described as a difference between the concepts of clinical and management reasoning.

CONCLUSIONS

We suggest that the development or deployment of informatics competencies in undergraduate medical education should focus on 3 items: the teaching of diagnostic reasoning, such that the information tasks that comprise both clinical and management reasoning can be discussed; precision medical education, where informatics can provide for more fine-grained evaluation; and assessment methods to support traditional pedagogical efforts (both at the bedside and beyond). Assessment using cases or structured assessments (eg, Objective Structured Clinical Examinations) would help students draw parallels between clinical informatics and fundamental clinical subjects and would better emphasize the cognitive techniques taught through informatics.

A Call to Action From the California Consortium for the Assessment of Clinical Competence: Making the Case for Regional Collaboration

The discontinuation of the United States Medical Licensing Examination Step 2 Clinical Skills Examination emphasizes the need for other reliable standardized assessments of medical student clinical skills. For 30 years, the California Consortium for the Assessment of Clinical Competence (CCACC) has collaborated in the development of clinical skills assessments and has become a valuable resource for clinicians, standardized patient educators, psychometricians, and medical educators. There are many merits to strong multi-institutional partnerships, including the integration of data across multiple schools to provide feedback to both students and curricula, pooled test performance statistics for analysis and quality assurance, shared best practices and resources, individual professional development, and opportunities for research and scholarship. The infrastructure of the CCACC allows member schools to adapt to a changing medical landscape, from emerging trends in clinical medicine to the limitations imposed by a global pandemic. In the absence of a national examination, there is now a greater need for all medical schools to develop a comprehensive, dynamic, and psychometrically sound assessment that accurately evaluates clinical skills. Medical schools working together in regional consortia have the opportunity to create and implement innovative and robust assessments that evaluate a wide range of clinical skills, ensure that medical students have met an expected level of clinical competency before graduation, and provide a framework that contributes to ongoing efforts for the development of new national clinical skills standards.

Comparing Entrustment Decision-Making Outcomes of the Core Entrustable Professional Activities Pilot, 2019-2020

IMPORTANCE

Gaps in readiness for indirect supervision have been identified for essential responsibilities encountered early in residency, presenting risks to patient safety. Core Entrustable Professional Activities (EPAs) for entering residency have been proposed as a framework to address these gaps and strengthen the transition from medical school to residency.

OBJECTIVE

To assess progress in developing an entrustment process in the Core EPAs framework.

DESIGN, SETTING, AND PARTICIPANTS

In this quality improvement study in the Core EPAs for Entering Residency Pilot, trained faculty made theoretical entrustment determinations and recorded the number of workplace-based assessments (WBAs) available for each determination in 2019 and 2020. Four participating schools attempted entrustment decision-making for all graduating students or a randomly selected subset of students. Deidentified, individual-level data were merged into a multischool database.

INTERVENTIONS

Schools implemented EPA-related curriculum, WBAs, and faculty development; developed systems to compile and display data; and convened groups to make theoretical summative entrustment determinations.

MAIN OUTCOMES AND MEASURES

On an EPA-specific basis, the percentage of students for whom an entrustment determination could be made, the percentage of students ready for indirect supervision, and the volume of WBAs available were recorded.

RESULTS

Four participating schools made 4525 EPA-specific readiness determinations (2296 determinations in 2019 and 2229 determinations in 2020) for 732 graduating students (349 students in 2019 and 383 students in 2020). Across all EPAs, the proportion of determinations of "ready for indirect supervision" increased from 2019 to 2020 (997 determinations [43.4%] vs 1340 determinations [60.1%]; 16.7 percentage point increase; 95% CI, 13.8-19.6 percentage points; P < .001), as did the proportion of determinations for which there were 4 or more WBAs (456 of 2295 determinations with WBA data [19.9%] vs 938 [42.1%]; 22.2 percentage point increase; 95% CI, 19.6-24.8 percentage points; P < .001). The proportion of EPA-specific data sets considered for which an entrustment determination could be made increased from 1731 determinations (75.4%) in 2019 to 2010 determinations (90.2%) in 2020 (14.8 percentage point increase; 95% CI, 12.6-16.9 percentage points; P < .001). On an EPA-specific basis, there were 5 EPAs (EPA 4 [orders], EPA 8 [handovers], EPA 10 [urgent care], EPA 11 [informed consent], and EPA 13 [patient safety]) for which few students were deemed ready for indirect supervision and for which there were few WBAs available per student in either year. For example, for EPA 13, 0 of 125 students were deemed ready in 2019 and 0 of 127 students were deemed ready in 2020, while 0 determinations in either year included 4 or more WBAs.

CONCLUSIONS AND RELEVANCE

These findings suggest that there was progress in WBA data collected, the extent to which entrustment determinations could be made, and proportions of entrustment determinations reported as ready for indirect supervision. However, important gaps remained, particularly for a subset of Core EPAs.

A pilot study on the use of the nominal group technique to refine core pharmacy roles and to determine what competencies may be missing from pharmacy education

INTRODUCTION

To describe the use of the nominal group technique (NGT) to refine pharmacy core roles and to compare these roles with current pharmacy outcomes and other literature to highlight potential deficiencies.

METHODS

The NGT process was used for this proposal review. The process was conducted in four key stages: silent generation, round-robin, clarification, and voting. A convenience sampling of five pharmacy faculty and administrators that have researched the areas of practice-readiness and pharmacy competencies formed the panel of participants for the NGT process.

RESULTS

Study findings offer seven core roles that define pharmacists' scope of practice: knowledge, patient care skills, professional, scholar, system-based practice/manager, collaborator, and advocate/health promoter. Development of these core roles revealed several missing pharmacy competencies or ones only covered in optional learning objectives: conflict management, professional advocacy, scholarship, empathy, personal health, transitions of care, health outcomes, quality improvement, and health insurance.

CONCLUSIONS

The development of pharmacy roles is one way to ensure students are adequately prepared for pharmacy practice following graduation. Mapping of competencies to core professional roles would allow schools/colleges of pharmacy to have one cohesive document to guide pedagogical and assessment practice. More research and consensus building will be needed before these core roles could be disseminated more broadly.

Can Incoming Interns Be Entrusted to Recognize Medical Emergencies? Implementation of a Vignette-Based Cross-Cover Assessment

BACKGROUND

Interns must recognize urgent clinical situations and know when to seek assistance. However, assessing this skill is challenging.

OBJECTIVE

We explored whether graduating medical students could determine urgency of medical cross-cover scenarios and what factors were associated with this ability.

METHODS

Sixty senior medical students enrolled in an internal medicine residency preparation course, and 28 experts were invited to take an assessment using 4 clinical vignette handoffs, each with 5 to 6 cross-cover scenarios. Respondents were asked whether they would evaluate the patient at bedside and notify their supervising resident. They were asked to rate their comfort managing the scenario, rate the urgency (1=low, 2=moderate, 3=high), and take a medical knowledge quiz. Student performance was categorized based on stratification of clinical urgency-those who underestimated (fourth quartile), accurately estimated (second and third quartile), and overestimated (first quartile) urgency. We examined differences between groups in medical knowledge, action, and confidence using analysis of variance and post-hoc Tukey Honestly Significant Difference test.

RESULTS

Fifty-eight students (96.7%) and 22 experts (78.6%) participated. Clear differentiation emerged between students' ability to estimate urgency on the 3-point urgency scale (lowest quartile: 2.15±0.11; mid-quartiles: 2.38±0.07; upper quartile: 2.61±0.10, respectively). Students who underestimated urgency were less likely to notify their supervising resident (P=.001) and less likely to evaluate a patient at bedside (P=.01). There was no difference in quiz score or comfort level.

CONCLUSIONS

Incoming interns vary in their abilities to recognize urgent scenarios, independent of medical knowledge and confidence.

The education passport: connecting programmatic assessment across learning and practice

Competency-based medical education (CBME) shifts us from static assessment of learning to developmental assessment for learning. However, implementation challenges associated with CBME remain a major hurdle, especially after training and into practice. The full benefit of developmental assessment for learning over time requires collaboration, cooperation, and trust among learners, regulators, and the public that transcends each individual phase. The authors introduce the concept of an "Education Passport" that provides evidence of readiness to travel across the boundaries between undergraduate medical education, graduate medical education, and the expanse of practice. The Education Passport uses programmatic assessment, a process of collecting numerous low stakes assessments from multiple sources over time, judging these data using criterion-referencing, and enhancing this with coaching and competency committees to understand, process, and accelerate growth without end. Information in the Passport is housed on a cloud-based server controlled by the student/physician over the course of training and practice. These data are mapped to various educational frameworks such Entrustable Professional Activities or milestones for ease of longitudinal performance tracking. At each stage of education and practice the student/physician grants Passport access to all entities that can provide data on performance. Database managers use learning analytics to connect and display information over time that are then used by the student/physician, their assigned or chosen coaches, and review committees to maintain or improve performance. Global information is also collected and analyzed to improve the entire system of learning and care. Developing a true continuum that embraces performance and growth will be a long-term adaptive challenge across many organizations and jurisdictions and will require coordination from regulatory and national agencies. An Education Passport could also serve as an organizing tool and will require research and high-value communication strategies to maximize public trust in the work.

Evaluating the Association of a Core EPA-Oriented Patient Handover Curriculum on Medical Students' Self-reported Frequency of Observation and Skill Acquisition

INTRODUCTION

"Giving or receiving a patient handover to transition patient care responsibility" is one of the thirteen Core Entrustable Professional Activities (Core EPAs) for Entering Residency. However, implementing a patient handover curriculum in undergraduate medical education (UME) remains challenging. Educational leaders in the multi-institutional Core EPA8 pilot workgroup developed a longitudinal patient handover UME curriculum that was implemented at two pilot institutions.

MATERIALS AND METHODS

We utilized multi-school graduation questionnaire data to assess the association of our patient handover curriculum on self-reported frequency of observation/feedback and skill acquisition by comparing data from the shared curriculum schools to data from other Core EPA pilot schools (three schools with school-specific curriculum; five without a dedicated handover curriculum). Questionnaire data from 1,278 graduating medical students of the class of 2020 from all ten Core EPA pilot schools were analyzed.

RESULTS

Graduates from the two medical schools that implemented the shared patient handover curriculum reported significantly greater frequency of handover observation/feedback compared to graduates at the other schools (school-specific curriculum (p < .05) and those without a handover curriculum (p < .05)). Graduates from the two shared approach schools also more strongly agreed that they possessed the skill to perform handovers compared to graduates from the other eight pilot schools that did not implement this curriculum.

CONCLUSION

The findings of this study suggest that the implementation of a multi-institutional Core EPA-based curricular model for teaching and assessing patient handovers was successful and could be implemented at other UME institutions.