Reimagining Core Entrustable Professional Activities for Undergraduate Medical Education in the Era of Artificial Intelligence

The proliferation of generative artificial intelligence (AI) and its extensive potential for integration into many aspects of health care signal a transformational shift within the health care environment. In this context, medical education must evolve to ensure that medical trainees are adequately prepared to navigate the rapidly changing health care landscape. Medical education has moved toward a competency-based education paradigm, leading the Association of American Medical Colleges (AAMC) to define a set of Entrustable Professional Activities (EPAs) as its practical operational framework in undergraduate medical education. The AAMC's 13 core EPAs for entering residencies have been implemented with varying levels of success across medical schools. In this paper, we critically assess the existing core EPAs in the context of rapid AI integration in medicine. We identify EPAs that require refinement, redefinition, or comprehensive change to align with the emerging trends in health care. Moreover, this perspective proposes a set of "emerging" EPAs, informed by the changing landscape and capabilities presented by generative AI technologies. We provide a practical evaluation of the EPAs, alongside actionable recommendations on how medical education, viewed through the lens of the AAMC EPAs, can adapt and remain relevant amid rapid technological advancements. By leveraging the transformative potential of AI, we can reshape medical education to align with an AI-integrated future of medicine. This approach will help equip future health care professionals with technological competence and adaptive skills to meet the dynamic and evolving demands in health care.

Early career outcomes of a large four-year MD/ MPH program: Results of a cross sectional survey of three cohorts of graduates

The University of Miami Miller School of Medicine started a four-year MD/ MPH program in 2011 with a mission to graduate public health physician leaders to address the public health needs of the 21st century, with emphasis on three areas: leadership, research, and public health. A prospective cross-sectional survey of early graduates was conducted to understand how they incorporate public health training into their careers. There were two study questions: What are the self-described early career activities of the graduates of the first three cohorts in the areas of leadership, research, and public health and what are the perceptions regarding the influence of the public health training on their careers? In the summer of 2020, a survey was sent to graduates from the classes of 2015, 2016, and 2017. In addition to several multiple-choice questions, the survey included an open-ended question on the impact of public health training in their careers. Inductive content analysis was used to analyze the responses to the open-ended question. Eighty-two of the 141 eligible graduates (63%) completed the survey; 80 of whom had participated or was currently participating in residency training. Forty-nine joined a residency in a primary care field. Many graduates had leadership roles in their early careers, including 35 who were selected as chief residents. Fifty-seven participated in research, most commonly in quality improvement (40), clinical (34) and community based (19). Over one third (30) chose to do work in public health during residency. Themes that emerged regarding the influence of public health training on their careers were: 1) Shifts in perspective, 2) Value of specific skills related to public health, 3) Steppingstone for professional opportunities 4) Focus on health disparities, social determinants, and inadequacies of the healthcare system, 5) Perception as leaders and mentors for peers, and 6) Preparedness for the pandemic. Graduates self-reported involvement in leadership, research, and public health activities as well as a commitment towards addressing some of our most pressing public health needs. Although long-term career outcomes need to be determined over time, graduates currently report considerable benefits of their public health training for their professional outcomes.

EPMO: A novel medical student assessment tool that integrates entrustable professional activities, prime, and the modified Ottawa coactivity scale

PURPOSE

Alignment of workplace-based assessments (WPBA) with core entrustable professional activities (EPAs) for entering residency may provide opportunities to monitor student progress across the continuum of undergraduate medical education. Core EPAs, however, reflect tasks of varying degrees of difficulty and faculty assessors are not accustomed to rating students based on entrustability. Expectations of student progress should vary depending on the complexity of the tasks associated with the EPAs. An assessment tool that orients evaluators to the developmental progression of specific EPA tasks will be critical to fairly evaluate learners.

METHODS

The authors developed an EPA assessment tool combining the frameworks of Professionalism, Reporter, Interpreter, Manager, Educator (PRIME), and Modified Ottawa coactivity scales. Only those EPAs that could be repeatedly observed and assessed across clinical clerkships were included. From July 2019 to March 2020, third-year medical students across multiple clerkships were assessed using this tool. The authors hypothesized that if the tool was applied correctly, ratings of learner independence would be lower with higher complexity tasks and that such ratings would increase over the course of year with ongoing clinical learning.

RESULTS

Assessment data for 247 medical students were similar across clerkships suggesting that evaluators in diverse clinical contexts were able to use this tool to assign scores reflective of developing entrustability in the workplace. Faculty rated student entrustability highest in skills emphasized in the pre-clerkship curriculum (professionalism and reporter) and progressively lower in more advanced skills (interpreter and manager). Students' ratings increased over time with more clinical exposure.

CONCLUSIONS

The authors developed a composite WBPA tool that combines the frameworks of EPAs, PRIME, and Modified Ottawa Co- Activity and demonstrated the usability of applying it for learner assessments in clinical settings. Further multicenter studies with cohorts of pre- and post-clerkship students may provide additional validity evidence for the tool.

Design, Implementation, and Evaluation of an Intensive Course on Issues in Women's Health and Gender-Based Medicine

OBJECTIVES

Sex and gender have profound effects on disease prevalence, presentation, and outcome, but these issues are not covered in depth in standard medical school curricula. To improve understanding of women's health, an intensive 1-month class was offered to fourth-year medical students.

METHODS

The class combined background lectures on the biological and social determinants of women's health with presentations on specific medical conditions by practicing clinicians and students. Students' anonymous responses to end-of-class evaluation used by Stony Brook University School of Medicine as well as pre- and post-class answers to the question "why are women twice as likely to go to the doctor" were analyzed using quantitative, descriptive, and qualitative approaches.

RESULTS

The class was given between 2017 and 2022 to a total of 154 students. Course evaluations were submitted by 133 students. Over 80% of responders ranked the class as good or excellent and many expressed surprise about how much sex and gender influence health. Furthermore, before taking the class responders favored gender stereotypes (82%) and OB/GYN visits (56%) as the main reasons why women utilize healthcare more often than men, whereas only 31% of post-class answers included these factors (p < .0001), which were replaced by others including misdiagnosis, high rate of adverse effects of medications, implicit bias, and longevity.

CONCLUSION

A dedicated class given to students at the end of their undergraduate medical training increased awareness and knowledge of the effects of sex and gender on women's health.

Challenges and opportunities for educating health professionals after the COVID-19 pandemic

The education of health professionals substantially changed before, during, and after the COVID-19 pandemic. A 2010 Lancet Commission examined the 100-year history of health-professional education, beginning with the 1910 Flexner report. Since the publication of the Lancet Commission, several transformative developments have happened, including in competency-based education, interprofessional education, and the large-scale application of information technology to education. Although the COVID-19 pandemic did not initiate these developments, it increased their implementation, and they are likely to have a long-term effect on health-professional education. They converge with other societal changes, such as globalisation of health care and increasing concerns of health disparities across the world, that were exacerbated by the pandemic. In this Health Policy, we list institutional and instructional reforms to assess what has happened to health-professional education since the publication of the Lancet Commission and how the COVID-19 pandemic altered the education process.

Pediatric Schwartz Rounds: Influencing Provider Insights and Emotional Connectedness

BACKGROUND AND OBJECTIVES

Schwartz Rounds (SR) is an interdisciplinary program that focuses on compassionate care by allowing the formation of an interprofessional community around the human and emotional testimonies of caregivers. The purpose of this study was to examine the impact of implementing departmental SR on pediatric care providers at a tertiary care children's hospital in New York.

METHODS

We applied the logic outcomes model for program evaluation to examine the impact of SR on pediatric providers. The standard evaluation form provided by the Schwartz Center was used to collect data after every SR. Descriptive statistics and qualitative data content analysis methods were used to analyze the evaluation data from the SR.

RESULTS

A total of 820 standard evaluation forms were collected from 17 of the 23 SR sessions offered (response rate: 74.8%). Most participants felt that, during the SR sessions, challenging social and emotional aspects of patient care were discussed and that they gained better perspectives of their coworkers and their patients/families. They reported less isolation and more openness to express their feelings about patient care to their coworkers. The analysis of 299 written comments identified 5 themes: understanding other people's perspectives, the importance of communication, empathy and compassion, awareness of personal biases, and maintaining boundaries.

CONCLUSIONS

Schwartz Rounds can provide an effective venue for pediatric care providers to gain insights into coworker and patient/family perspectives and process emotional experiences while providing patient care in a variety of circumstances.

Evidence-Based Diagnostic Test Accuracy of History, Physical Examination, and Imaging for Intussusception: A Systematic Review and Meta-analysis

INTRODUCTION

Intussusception is the most common cause of pediatric small bowel obstruction. Timely and accurate diagnosis may reduce the risk of bowel ischemia. We quantified the diagnostic test accuracy of history, physical examination, abdominal radiographs, and point-of-care ultrasound.

METHOD

We conducted a systematic review for diagnostic test accuracy of history, physical examination, and imaging concerning for intussusception. Our literature search was completed in June 2019. Databases included Medline via Ovid, Embase, Scopus, and Wiley Cochrane Library. We conducted a second review of the literature up to June 2019 for any additional studies. Inclusion criteria were younger than 18 years and presenting to the emergency department for abdominal complaints, consistent with intussusception. We performed data analysis using mada, version 0.5.8. We conducted univariate and bivariate analysis (random effects model) with DerSimonian-Laird and Reitsma model, respectively. QUADAS-2 was used for bias assessment.

RESULTS

The literature search identified 2639 articles, of which 13 primary studies met our inclusion criteria. Abdominal pain, vomiting, and bloody stools had positive likelihood ratios LR(+) between 1 and 2, whereas the negative likelihood ratio, LR(-), ranged between 0.4 and 0.8. Abnormal abdominal radiograph had LR(+) of 2.5 and LR(-) of 0.20, whereas its diagnostic odds ratio was 13. Lastly, point-of-care ultrasound had LR(+) of 19.7 and LR(-) of 0.10. The diagnostic odds ratio was 213.

CONCLUSIONS

History and physical examination had low diagnostic test accuracy. Abdominal radiographs had low diagnostic test accuracy, despite moderate discriminatory characteristics. Point-of-care ultrasound had the highest diagnostic test accuracy to rule in or rule out intussusception.

Does Delaying the United States Medical Licensing Examination Step 1 to After Clerkships Affect Student Performance on Clerkship Subject Examinations?

Phenomenon: Schools are considering the optimal timing of Step 1 of the United States Medical Licensing Examination (USMLE). Two primary reasons for moving Step 1 after the core clerkships are to promote deeper, more integrated basic science learning in clinical contexts and to better prepare students for the increasingly clinical focus of Step 1. Positioning Step 1 after the core clerkships leverages a major national assessment to drive learning, encouraging students to deepen their basic science knowledge while in the clinical setting. Previous studies demonstrated small increases in Step 1 scores, reductions in failure rates, and similar Step 2 Clinical Knowledge scores when Step 1 was after the clerkships. Some schools that have moved Step 1 reported declines in clinical subject examination (CSE) performance. This may be due to shortened pre-clerkship curricula, the absence of the Step 1 study period for knowledge consolidation, or exposure to fewer National Board of Medical Examiners type questions prior to taking CSEs. This multi-institutional study aimed to determine whether student performance on CSEs was affected by moving Step 1 after the core clerkships. Approach: CSE scores for students from eight schools that moved Step 1 after core clerkships between 2012 and 2016 were analyzed in a pre-post format. Hierarchical linear modeling was used to quantify the effect of the curriculum on CSE performance. Additional analysis determined if clerkship order impacted clinical subject exam performance and whether the curriculum change resulted in more students scoring in the lowest percentiles (as defined as below the national fifth percentile) before and after the curricular change. Findings: After moving Step 1 to after the clerkships, collectively these eight schools demonstrated statistically significant lower performance on four CSEs (Medicine, Neurology, Pediatrics, and Surgery) but not Obstetrics/Gynecology or Psychiatry. Comparing performance within the three years pre and post Step 1 change, differences across all clerkships ranged from 0.3 to -2.0 points, with an average difference of -1.1. CSE performance in clerkships taken early in the sequence was more affected by the curricular change, and differences gradually disappeared with subsequent examinations. Medicine and Neurology showed the largest average differences between curricular-group when taken early in the clinical year. Finally, there was a slightly higher chance of scoring below the national fifth percentile in four of the clinical subject exams (Medicine, Neurology, Pediatrics, and Psychiatry) for the cohort with Step 1 after the clerkships. Insights: Moving Step 1 after core clerkships had a small impact on CSE scores overall, with decreased scores for exams early in the clerkship sequence and an increased number of students below the fifth percentile. Score differences have minor effects on clerkship grades, but overall the size of the effect is unlikely to be educationally meaningful. Schools can use a variety of mitigation strategies to address CSE performance and Step 1 preparation in the clerkship phase.

Enhancing pediatricians' engagement on social media: the role of board style questions

This article was migrated. The article was marked as recommended. Background: Social Media is used among medical professionals for collaborative education. Little is known about how case discussions prompt engagement. Objective: To determine the association between item characteristics of board exam-style questions to social media engagement. Methods: This was a prospective cohort study through the American Academy of Pediatrics (AAP) PediaLink FaceBook page, conducted in 2018 over 9 months. Items from the 2017 PREP® questions were ranked in difficulty, then rated in relevance to general pediatrics through content-expert consensus. Thirty-six questions were randomly posted on FaceBook and Twitter weekly. Independent variables included item difficulty rank, difficulty level (easy vs hard), relevance to general pediatrics, and word count. Outcome variables included percent correct responses and total comments under the post. Results: More difficult questions were associated with fewer comments (rho=0.63, p<0.001) and lower correct response percentages (rho=0.39, p=0.02). Easy questions garnered more comments than hard questions (median 18 IQR 13-23 vs median 9.5 IQR 5-14, p=0.001). Correct response percentage was lower for hard questions (90% IQR 85-95% vs. 77% IQR 60-94%, p=0.04). Relevance to general pediatrics and word count did not affect engagement (p > 0.1). Conclusion: Easier practice test items attracted more responses from pediatricians on social media, increasing engagement.

Healthcare workforce transformation: implementing patient-centered medical home standards in an academic medical center

BACKGROUND

Large scale implementation of new strategies and healthcare delivery standards in academic medical centers (AMCs) requires training of healthcare workforce at different stages of their medical career. The patient-centered medical home (PCMH) model for healthcare delivery involves adoption by all members of the healthcare workforce, including seasoned professionals and trainees. Though widely known, the PCMH model has been implemented sporadically at large AMCs and methods to implement the model across healthcare workforce have not been well-documented.

METHODS

To meet all PCMH standards and achieve sustainable level 3 recognition, the authors implemented in 2014-2015 a multi-pronged approach that capitalized on existing educational infrastructure among faculty, residents, and medical students. Within 18 months, the authors applied new interdisciplinary practices and policies, redesigned residency training in continuity practices and extensively modified medical school curricula.

RESULTS

These innovative transformational education efforts addressed the six PCMH standards for faculty, residents, and undergraduate medical students. Faculty played a major role as system change agents and facilitators of learning. Residents learned to better understand patients' cultural needs, identify 'at-risk' patients, ensure continuity of care, and assess and improve quality of care. Medical students were exposed to PCMH core standards throughout their training via simulations, training in the community and with patients, and evaluation tasks. By implementing these changes across the healthcare workforce, the AMC achieved PCMH status in a short time, changed practice culture and improved care for patients and the community. Since then, the AMC has been able to maintain PCMH recognition annually with minimal effort.

CONCLUSIONS

Successful strategies that capitalize on existing strengths in infrastructure complemented by innovative educational offerings and inter-professional partnerships can be adapted by other organizations pursuing similar transformation efforts. This widespread transformation across the healthcare workforce facilitate a deep-rooted change that enabled our academic medical center to sustain PCMH recognition.

The Optimal Timing of Step 1 in Medical Education Following the Transition to Pass/Fail: A Unique Perspective from Post-clerkship Step 1 Schools

The National Board of Medical Examiners' decision to change Step 1 of the United States Medical Licensing Examination (USMLE) from a three-digit score to Pass/Fail (P/F) represents a disruptive change for students, faculty, and leaders in the academic community. In the context of this change, some schools may re-consider the optimal timing of Step 1 as they strive to align their assessment practices with sound educational principles. Currently, over 20 schools administer USMLE Step 1 after the core clerkships. In this commentary, we review the educational rationale for a post-clerkship Step 1, highlighting how adult learning theories support this placement. We discuss some short-term challenges post-clerkship Step 1 schools may encounter due to the proposed timing of the change in scoring, which creates three unique scenarios for learners that can introduce inequity in the system and provoke anxiety. We review outcomes of potentially heightened importance when Step 1 is P/F, including lower clinical subject exam scores in some clerkships, lower failure rates on Step 1 and stable Step 2 Clinical Knowledge scores with implications for the residency match. We outline the future potential for performance-based time-variable Step 1 study periods that are facilitated by post-clerkship placement of the exam. Finally, we discuss opportunities to achieve the goal of enhancing student well-being, which was a major rationale for eliminating the three-digit score.

Technology Can Augment, but Not Replace, Critical Human Skills Needed for Patient Care

The practice of medicine is changing rapidly as a consequence of electronic health record adoption, new technologies for patient care, disruptive innovations that breakdown professional hierarchies, and evolving societal norms. Collectively, these have resulted in the modification of the physician's role as the gatekeeper for health care, increased shift-based care, and amplified interprofessional team-based care. Technological innovations present opportunities as well as challenges. Artificial intelligence, which has great potential, has already transformed some tasks, particularly those involving image interpretation. Ubiquitous access to information via the Internet by physicians and patients alike presents benefits as well as drawbacks: patients and providers have ready access to virtually all of human knowledge, but some websites are contaminated with misinformation and many people have difficulty differentiating between solid, evidence-based data and untruths. The role of the future physician will shift as complexity in health care increases and as artificial intelligence and other technologies advance. These technological advances demand new skills of physicians; memory and knowledge accumulation will diminish in importance while information management skills will become more important. In parallel, medical educators must enhance their teaching and assessment of critical human skills (e.g., clear communication, empathy) in the delivery of patient care. The authors emphasize the enduring role of critical human skills in safe and effective patient care even as medical practice is increasingly guided by artificial intelligence and related technology, and they suggest new and longitudinal ways of assessing essential noncognitive skills to meet the demands of the future. The authors envision practical and achievable benefits accruing to patients and providers if practitioners leverage technological advancements to facilitate the development of their critical human skills.

Medical students' perception of behaviors in the clinical learning environment evolve with increasing clinical exposure as measured with situational video vignettes

This cross-sectional study involved matriculating, mid-level and graduating medical students (n = 723) who participated in specific transition courses in our medical school curriculum between August 2016 and March 2019. We used a mixed-methods approach (survey and analysis of narrative comments) to study the evolution in perception of the learning environment by medical students with increasing clinical exposure using four different video vignettes. Differences in student perceptions of mistreatment exists at various levels of training. Compared to graduating students, matriculating students were more likely to perceive reprimanding a student on being late as appropriate behavior (80.5% vs 53.3%, p = 0.001), whereas a significantly higher proportion of the graduating students (15.3%, p = 0.001) perceived it as mistreatment. A majority of the matriculating students (84%, p = 0.001) considered the case of an eager student as mistreatment, while a significantly higher proportion of the graduating students (59.5%, p = 0.001) did not think it was mistreatment. Qualitative analysis of comments from students at different stages of training displayed an increased appreciation of their professional responsibilities and nuanced appreciation of body language and tone as contributing factors in determining whether a situation represented inappropriate behavior. Our results demonstrate that students' perceptions of inappropriate behaviors evolve with increased clinical exposure.

Use of student feedback to drive quality improvement (QI) in a preclinical U.S. medical school course

Medical educators are continually looking for ways to enhance integrated learning and help students see how the material taught in their various courses is inter-related. . At Stony Brook School of Medicine, we embarked on a school-wide new curriculum called the Learning focused, Experiential, Adaptive, Rigorous and Novel (LEARN) curriculum and developed several integrated courses that were not based in specific departments. As part of this process, the pre-clinical (Phase-1) curriculum was shortened to 17 months to accommodate an expanded set of clinical offerings. The new structure called for teachers from different departments to lead and conduct the integrated blocks of pre-clinical courses. In this paper, we describe our discouraging experience with the first iteration of an integrated course in Cardiology, Pulmonology and Renal organ systems (CPR), and its transformation into a highly successful second iteration. This involved a systematic course quality improvement (QI) process within the context of a larger school wide curricular reform. As a result, student overall satisfaction with the course increased from 22% (28 of 127 responders) to 83% (111 of 134 responders); the mean score on a standardized NBME content exam increased by 6.7%. We report the systematic process we used to collect data from students and faculty that helped facilitate quality improvement in a key course in Phase-1 of our LEARN curriculum.

How are clinician-educators evaluated for educational excellence? A survey of promotion and tenure committee members in the United States

Background: In recent years, educational leaders have proposed domains of educational excellence and corresponding metrics to objectively measure contributions of clinician-educators for promotion and tenure (P&T). The purpose of this study was to explore whether P&T committees in United States (US) have incorporated these recommendations into practice. Method: The authors conducted a survey of P&T leaders across institutions in US. Items included questions related to institutional tracks for P&T, domains included in promotional packets, metrics for their measurement, and use of an Educator's Portfolio (EP). Results: Respondents from 55 institutions completed the survey. The presence of a teaching academy/society was associated with the presence of a promotion track for clinician-educators (p = 0.04). Only teaching activities (91%), assessment of learners (55%), and educational scholarship (51%) were required by a majority of institutions. Few institutions used objective methods for measuring impact and less than half (47%) required an EP. Discussion: These results highlight both progress in the recognition of clinician-educators while also suggesting discordance in the perspective of educational leaders and the practice of P&T committees. The authors advocate for establishing a national community of expert medical educators who may assist P&T committees in adopting consensus-based criteria and metrics to evaluate clinician-educators' contributions.

Curriculum reform and evolution: Innovative content and processes at one US medical school

AIM

Curriculum reform in medical schools continues to be an ever-present and challenging activity in medical education. This paper describes one school's experiences with specific curricular innovations that were developed or adapted and targeted to meet a clear set of curricular goals during the curriculum reform process. Those goals included: (a) promoting active learning and learner engagement; (b) establishing early professional identity; and (c) developing physician competencies in an integrated and contextual manner while allowing for individualized learning experiences for the millennial student.

METHODS

Six specific innovations championed by the school are described in detail. These included Themes in Medical Education, Translational Pillars, Stony Brook Teaching Families, Transition Courses, Educational Continuous Quality Improvement Processes, and our Career Advising Program. Development of the ideas and design of the innovations were done by faculty and student teams.

RESULTS

We discuss successes and ongoing challenges with these innovations which are currently in the fourth year of implementation.

CONCLUSIONS

Our curriculum reform has emphasized the iterative process of curriculum building. Based on our experience, we discuss general and practical guidelines for curriculum innovation in its three phases: setting the stage, implementation, and monitoring for the achievement of intended goals.

How Mentor Identity Evolves: Findings From a 10-Year Follow-up Study of a National Professional Development Program

PURPOSE

Despite academic medicine's endorsement of professional development and mentoring, little is known about what junior faculty learn about mentoring in implicit curricula of professional development programs, and how their mentor identity evolves in this context. The authors explored what faculty-participants in the Educational Scholars Program implicitly learned about mentoring and how the implicit curriculum affected mentor identity transformation.

METHOD

Semistructured interviews with 19 of 36 former faculty-participants were conducted in 2016. Consistent with constructivist grounded theory, data collection and analysis overlapped. The authors created initial codes informed by Ibarra's model for identity transformation, iteratively revised codes based on incoming data patterns, and created visual representations of relationships amongst codes to gain a holistic, shared understanding of the data.

RESULTS

In the implicit curriculum, faculty-participants learned the importance of having multiple mentors, the value of peer mentors, and the incremental process of becoming a mentor. The authors used Ibarra's model to understand how the implicit curriculum worked to transform mentor identity: Faculty-participants reported observing mentors, experimenting with different ways to mentor and to be a mentor, and evaluating themselves as mentors.

CONCLUSIONS

The Educational Scholars Program's implicit curriculum facilitated faculty-participants taking on mentor identity via opportunities it afforded to watch mentors, experiment with mentoring, and evaluate self as mentor, key ingredients for identity construction. Leaders of professional development programs can develop faculty as mentors by capitalizing on what faculty-participants learn in the implicit curriculum and deliberately structuring postgraduation mentoring opportunities.

Impact of a Home Hospice Visit Program on Third-Year Medical Students: A qualitative analysis of student reflections

Aim

This study evaluates the impact of an interprofessional home hospice visit (HHV) on third-year medical students’ attitudes toward, and understanding of, end-of-life care and the visit's effect on students’ views of their emerging professional roles and identities.

Methods

All third-year medical students at Stony Brook School of Medicine in Stony Brook, New York, USA, participated in an HHV. A didactic session preceded the HHV. Subsequently, students were required to submit a piece of reflective writing detailing the impact of the visit. We conducted a qualitative analysis of a random sample drawn from the 467 submitted reflections.

Results

Six themes emerged from the student reflections: three were related to the students’ direct observations during the HHV, and three were related to the reflective learning of the students based on their HHV experience.

Conclusion

The qualitative analysis of the reflective writings showed that the students gained a deep appreciation of the human identity of hospice patients and a humanistic understanding of their own role as future physicians.

Case Studies for Recognizing Appropriate and Inappropriate Behaviors in the Clinical Learning Environment

INTRODUCTION

The Association of American Medical Colleges surveys graduating medical students regarding the persistent prevalence of learning environment concerns. This training module is designed to increase awareness of appropriate and inappropriate behaviors in the clinical learning environment among medical professionals and trainees.

METHODS

An introductory PowerPoint presents the types of inappropriate behaviors that may be observed in the clinical learning environment along with institution-specific mechanisms for reporting such behaviors. We have also created six vignettes depicting various scenarios that trainees may encounter. The vignettes are presented in both text and video format and may be used in any combination. The entire module consisting of the PowerPoint presentation and the case studies can be delivered in 90 minutes to a large group of learners. Learners are divided into smaller groups of six to eight for discussions. The presentation and discussion can be done by a single or multiple facilitators. The target audience is primarily medical professionals and trainees at various levels of clinical exposure.

RESULTS

Since implementation of this training module at our institution, awareness of what constitutes mistreatment and how to report it has increased to nearly 100%. Representative institutional responses are provided for each vignette.

DISCUSSION

This training module can be presented to medical students, residents, and faculty at different stages of their professional development. We have enhanced learner awareness of what constitutes mistreatment and how to report these events. We offer these educational materials for other institutions to adapt and use in their specific institutional contexts.

The Impact of a National Faculty Development Program Embedded Within an Academic Professional Organization

A sizeable literature describes the effectiveness of institution-based faculty development programs in nurturing faculty educators as scholars, but national programs are less common and seldom evaluated. To fill this role, the Educational Scholars Program (ESP) was created within the Academic Pediatric Association (APA) in 2006. It is a national, three-year, cohort-based certification program focused on fostering educational scholarship. This article describes the development and outcomes of an innovative program embedded within the framework of a national professional organization, and offers a model for potential adaptation by similar organizations to enhance their support of educators.After 10 years, 171 scholars have enrolled in the ESP, and 50 faculty have participated. Scholars are assigned a faculty advisor and participate in three full-day sessions at a national meeting; online, interactive learning modules; and a mentored, scholarly project. The program receives support from the APA in four organizational frames: structural, human resource, political, and symbolic. The self-perceived scholarly proficiency of the scholars in Cohort 1 increased significantly over time, and their productivity and collaborations increased during and after the program. Scholars wrote enthusiastically about their experience in yearly and postprogram evaluations. In interviews, eight past APA presidents explained that the ESP strengthened the APA's mission, created new leaders, and provided a new model for other APA programs. Outcomes of the ESP suggest that a longitudinal faculty development program embedded within a national professional organization can create a social enterprise not only within the organization but also within the broader national community of educator-scholars.

A model of influences on the clinical learning environment: the case for change at one U.S. medical school

BACKGROUND

The learning environment within a school of medicine influences medical students' values and their professional development. Despite national requirements to monitor the learning environment, mistreatment of medical students persists.

METHODS

We designed a program called WE SMILE: We can Eradicate Student Mistreatment In the Learning Environment with a vision to enhance trainee and faculty awareness and ultimately eliminate medical student mistreatment. We provide a description of our program and early outcomes.

RESULTS

The program has enhanced student awareness of what constitutes mistreatment and how to report it. Faculty members are also aware of the formal processes and procedures for review of such incidents. Our proposed model of influences on the learning environment and the clinical workforce informs the quality of trainee education and safety of patient care. Institutional leadership and culture play a prominent role in this model. Our integrated institutional response to learning environment concerns is offered as a strategy to improve policy awareness, reporting and management of student mistreatment concerns.

CONCLUSIONS

Our WE SMILE program was developed to enhance education and awareness of what constitutes mistreatment and to provide multiple pathways for student reporting, with clear responsibilities for review, adjudication and enforcement. The program is demonstrating several signs of early success and is offered as a strategy for other schools to adopt or adapt. We have recognized a delicate balance between preserving student anonymity and informing them of specific actions taken. Providing students and other stakeholders with clear evidence of institutional response and accountability remains a key challenge. Multiple methods of reporting have been advantageous in eliciting information on learning environment infringements. These routes and types of reporting have enhanced our understanding of student perceptions and the specific contexts in which mistreatment occurs, allowing for targeted interventions. A common platform across the healthcare professions to report and review concerns has afforded us opportunities to deal with interprofessional issues in a respectful and trustworthy manner. We offer a model of learning environment influences with leadership and institutional culture at the helm, as a way to frame a comprehensive perspective on this challenging and complex concern.

Building Sustainable Professional Development Programs: Applying Strategies From Implementation Science to Translate Evidence Into Practice

INTRODUCTION

Multisite and national professional development (PD) programs for educators are challenging to establish. Use of implementation science (IS) frameworks designed to convert evidence-based intervention methods into effective health care practice may help PD developers translate proven educational methods and models into successful, well-run programs. Implementation of the national Educational Scholars Program (ESP) is used to illustrate the value of the IS model.

METHODS

Four adaptable elements of IS are described: (1) replication of an evidence-based model, (2) systematic stages of implementation, (3) management of implementation using three implementation drivers, and (4) demonstration of program success through measures of fidelity to proven models and sustainability.

RESULTS

Implementation of the ESP was grounded on five established principles and methods for successful PD. The process was conducted in four IS stages over 10 years: Exploration, Installation, Initial Implementation, and Full Implementation. To ensure effective and efficient processes, attention to IS implementation drivers helped to manage organizational relationships, build competence in faculty and scholars, and address leadership challenges. We describe the ESP's fidelity to evidence-based structures and methods, and offer three examples of sustainability efforts that enabled achievement of targeted program outcomes, including academic productivity, strong networking, and career advancement of scholars.

DISCUSSION

Application of IS frameworks to program implementation may help other PD programs to translate evidence-based methods into interventions with enhanced impact. A PD program can follow systematic developmental stages and be operationalized by practical implementation drivers, thereby creating successful and sustainable interventions that promote the academic vitality of health professions educators.

Can Individualized Learning Plans in an advanced clinical experience course for fourth year medical students foster Self-Directed Learning?

BACKGROUND

Residency programs have utilized Individualized Learning Plans (ILPs) to customize resident education while undergraduate medical education has not done so in a meaningful way. We discuss the use of ILPs within a fourth year medical school course to facilitate self-directed learning (SDL).

METHODS

At Stony Brook University School of Medicine, an ILP component was added to the Advanced Clinical Experience (ACE) course for fourth year students. Each completed an ILP outlining personal learning goals and strategies to achieve them. An adaptation of the Motivated Strategies for Learning Questionnaire (MSLQ) (Duncan T and McKeachie W, Educ Psych 40(2):117-128, 2005 and Cook DA et al., Med Ed 45:1230-1240, 2011) was used to measure success of ILPs in improving SDL. Qualitative data analysis was conducted on the ILPs and self-reflections.

RESULTS

Forty-eight students participated. Two of the four SDL sub-domains identified on the MSLQ showed improvement; self-efficacy (p = .001) and self-regulation (p = .002). 'Medical Knowledge' was the competency most frequently identified as an area of concentration (90 %) and professionalism was selected least frequently (4 %). A higher percentage (83 %) of students who reported complete achievement of their ILP goals also reported feeling better prepared for entering residency.

CONCLUSIONS

ILPs improve SDL strategies among medical students and may serve as useful tools to help shape future learning goals as they transition to residency training.

Drivers of Dashboard Development (3-D): A Curricular Continuous Quality Improvement Approach

PROBLEM

Undergraduate medical education (UME) programs are seeking systematic ways to monitor and manage their educational performance metrics and document their achievement of external goals (e.g., Liaison Committee on Medical Education [LCME] accreditation requirements) and internal objectives (institution-specific metrics). In other continuous quality improvement (CQI) settings, summary dashboard reports have been used to evaluate and improve performance.

APPROACH

The Stony Brook University School of Medicine UME leadership team developed and implemented summary dashboard performance reports in 2009 to document LCME standards/criteria compliance, evaluate medical student performance, and identify progress in attaining institutional curricular goals and objectives. Key performance indicators (KPIs) and benchmarks were established and have been routinely monitored as part of the novel Drivers of Dashboard Development (3-D) approach to curricular CQI.

OUTCOMES

The systematic 3-D approach has had positive CQI impacts. Substantial improvements over time have been documented in KPIs including timeliness of clerkship grades, midclerkship feedback, student mistreatment policy awareness, and student satisfaction. Stakeholder feedback indicates that the dashboards have provided useful information guiding data-driven curricular changes, such as integrating clinician-scientists as lecturers in basic science courses to clarify the clinical relevance of specific topics. Gaining stakeholder acceptance of the 3-D approach required clear communication of preestablished targets and annual meetings with department leaders and course/clerkship directors.

NEXT STEPS

The 3-D approach may be considered by UME programs as a template for providing faculty and leadership with a CQI framework to establish shared goals, document compliance, report accomplishments, enrich communications, facilitate decisions, and improve performance.

Reforming the 4th-Year Curriculum as a Springboard to Graduate Medical Training: One School's Experiences and Lessons Learned

PROBLEM

Concerns regarding the quality of training in the 4th year of medical school and preparation of graduates to enter residency education persist and are borne out in the literature.

INTERVENTION

We reviewed the published literature regarding Year 4 concerns as well as institutional efforts to improve the 4th-year curriculum from several schools. Based on input from key stakeholders, we established 4 goals for our Year 4 curriculum reform: (a) standardize the curricular structure, (b) allow flexibility and individualization, (c) improve the preparation for residency, and (d) improve student satisfaction. After the reform, we evaluated the outcomes using results from the Association of American Medical Colleges Questionnaire, student focus groups, and program director surveys.

CONTEXT

This article describes the context, process, and outcomes of the reform of the Year 4 curriculum at Stony Brook University School of Medicine.

OUTCOME

We were able to achieve all four stated goals for the reform. The significant components of the change included a flexible adaptable curriculum based on individual needs and preferences, standardized learning objectives across the year, standardized competency-based evaluations regardless of discipline, reinforcement of clinical skills, and training for the transition to the workplace as an intern. The reform resulted in increased student satisfaction, increased elective time, and increased preparedness for residency training as perceived by the graduates. The Program Director survey showed significant changes in ability to perform a medical history and exam, management of common medical conditions and emergencies, clinical reasoning and problem-solving skills, working and communication with the healthcare team, and overall professionalism in meeting obligations inherent in the practice of medicine.

LESSONS LEARNED

Lessons learned from our 4th-year reform process are discussed. Listening to the needs of the stakeholders was an important step in ensuring buy-in, having an institutional champion with an organizational perspective on the overall institutional mission was helpful in building the guiding coalition for change, building highly interactive collaborative interdisciplinary teams to work together addressed departmental silos and tunnel vision early on, and planning a curriculum is exciting but planning the details of the implementation can be quite tedious.

Routine, empathic and compassionate patient care: definitions, development, obstacles, education and beneficiaries

RATIONALE

We believe that this study represents an innovative approach to clarifying the definitions of routine, empathic and compassionate health care, as well as of sympathy. We emphasize the importance of affective empathy and its intensification in the context of patient suffering (compassion), without abandoning the ideal of clinical equanimity.

METHODS

We develop a pedagogical model for clinicians and trainees who are weaker in their empathic skills that includes four levels of growth. We clarify representative obstacles to empathic and compassionate care in education and clinical practice. We summarize the four beneficiaries of empathic and compassionate care (clinicians, patients, trainees, institutions). We suggest areas for future research, including the development of a compassion scale and conclude with a statement on how the conceptual and professional confusion we address adversely impacts patients and trainees. The article represents the consensus work of a group of health care professionals and students at Stony Brook University Hospital and School of Medicine who have been engaged in this project for several years through the Center for Medical Humanities, Compassionate Care, and Bioethics, established in August of 2008.

CONCLUSIONS

We discern a shift away from concepts of clinical empathy and compassionate care that deny a significant place for an affective component and that idealize 'detachment'.

Evaluating educators using a novel toolbox: applying rigorous criteria flexibly across institutions

Valuing faculty as educators is essential for medical schools to fulfill their unique mission of educating physicians. The 2006 Consensus Conference on Educational Scholarship, sponsored by the Association of American Medical Colleges (AAMC) Group on Educational Affairs, provided educators seeking academic promotion with a portfolio-based format for documenting activities in five domains, using evidence of quantity, quality, a scholarly approach, and educational scholarship. Yet, the lack of a rigorous, widely accepted system to assess educator portfolio submissions during the promotion and tenure process continues to impede the ability to fully value educators and educational scholars.The AAMC Task Force on Educator Evaluation was formed in 2010 to establish consensus guidelines for use by those responsible for the rigorous evaluation of the educational contributions of faculty. The task force delineated the educational contributions currently valued by institutions and then fulfilled its charge by creating the Toolbox for Evaluating Educators, a resource which contains explicit evidence-based criteria to evaluate faculty in each of the five domains of educator activity. Adoption of such criteria is now the rate-limiting step in using a fair process to recognize educators through academic promotion. To inform institutional review and implementation of these criteria, this article describes the iterative, evidence- and stakeholder-based process to establish the criteria. The authors advocate institutional adoption of these criteria so that faculty seeking academic promotion as educators, like their researcher colleagues, can be judged and valued using established standards for the assessment of their work.

Academic medicine change management: the power of the liaison committee on medical education accreditation process

Stony Brook University School of Medicine (SBU SOM) used a Liaison Committee on Medical Education (LCME) site visit to design a change management approach that engaged students, revitalized faculty, and enabled significant, positive institutional transformation while flexibly responding to concurrent leadership transitions. This "from-the-trenches" description of novel LCME site-visit-related processes may provide an educational program quality improvement template for other U.S. medical schools. The SBU SOM site visit processes were proactively organized within five phases: (1) planning (4 months), (2) data gathering (12 months), (3) documentation (6 months), (4) visit readiness (2 months), and (5) visit follow-up (16 months). The authors explain the key activities associated with each phase.The SBU SOM internal leadership team designed new LCME-driven educational performance reports to identify challenging aspects of the educational program (e.g., timeliness of grades submitted, midcourse feedback completeness, clerkship grading variability across affiliate sites, learning environment or student mistreatment incidents). This LCME process increased institutional awareness, identified the school's LCME vulnerabilities, organized corrective actions, engaged key stakeholders in communication, ensured leadership buy-in, and monitored successes. The authors' strategies for success included establishing a strong internal LCME leadership team, proactively setting deadlines for all phases of the LCME process, assessing and communicating vulnerabilities and action plans, building multidisciplinary working groups, leveraging information technology, educating key stakeholders through meetings, retreats, and consultants, and conducting a mock site visit. The urgency associated with an impending high-stakes LCME site visit can facilitate positive, local, educational program quality improvement.

Guidelines for evaluating the educational performance of medical school faculty: priming a national conversation

BACKGROUND

The academic community needs a sound framework for the promotion and advancement of educators. The Group on Educational Affairs of the Association of American Medical Colleges organized a consensus conference that affirmed the use of five domains for documenting the quantity and quality of scholarly engagement in educational activities: teaching, curriculum, advising/mentoring, educational leadership/administration, and learner assessment.

SUMMARY

In this article, we offer detailed guidelines to evaluate these five domains of educator performance and the essential elements of scholarly activity. The guidelines are adapted from our developmental educator portfolio template and educator portfolio analysis tool, previously published in MedEdPORTAL. A short tool for educator performance evaluation that summarizes items in the guidelines is proposed for discussion.

CONCLUSIONS

Our goal in this article is to itemize criteria for systematic faculty evaluation that can be applied in any institutional setting to assist promotion decision makers in their task of evaluating medical school faculty.

Evaluating the performance of medical educators: a novel analysis tool to demonstrate the quality and impact of educational activities

PURPOSE

Traditional promotion standards rely heavily on quantification of research grants and publications in the curriculum vitae. The promotion and retention of educators is challenged by the lack of accepted standards to evaluate the depth, breadth, quality, and impact of educational activities. The authors sought to develop a practical analysis tool for the evaluation of educator portfolios (EPs), based on measurable outcomes that allow reproducible analysis of the quality and impact of educational activities.

METHOD

The authors, 10 veteran educators and an external expert evaluator, used a scholarly, iterative consensus-building process to develop the tool and test it using real EPs from educational scholars who followed an EP template. They revised the template in parallel with the analysis tool to ensure that EP data enabled valid and reliable evaluation. The authors created the EP template and analysis tool for scholar and program evaluation in the Educational Scholars Program, a three-year national certification program of the Academic Pediatric Association.

RESULTS

The analysis tool combines 18 quantitative and 25 qualitative items, with specifications, for objective evaluation of educational activities and scholarship.

CONCLUSIONS

The authors offer this comprehensive, yet practical tool as a method to enhance opportunities for faculty promotions and advancement, based on well-defined and documented educational outcome measures. It is relevant for clinical educators across disciplines and across institutions. Future studies will test the interrater reliability of the tool, using data from EPs written using the revised template.