Better together: A multistakeholder approach to developing specialty-wide entrustable professional activities in emergency medicine

Purpose

Entrustable professional activities (EPAs) are a widely used framework for curriculum and assessment, yet the variability in emergency medicine (EM) training programs mandates the development of EPAs that meet the needs of the specialty as a whole. This requires eliciting and incorporating the perspectives of multiple stakeholders (i.e., faculty, residents, and patients) in the development of EPAs. Without a shared understanding of what a resident must be able to do upon graduation, we run the risk of advancing ill-prepared residents that may provide inconsistent care.

Methods

In an effort to address these challenges, beginning in February 2020, the authors assembled an advisory board of 25 EM faculty to draft and reach consensus on a final list of EPAs that can be used across all training programs within the specialty of EM. Using modified Delphi methodology, the authors came to consensus on an initial list of 22 EPAs. The authors presented these EPAs to faculty supervisors, residents, and patients for refinement. The authors collated and analyzed feedback from focus groups of residents and patients using thematic analysis. The EPAs were subsequently refined based on this feedback.

Results

Stakeholders in EM residency training endorsed a final revised list of 22 EPAs. Stakeholder focus groups highlighted two main thematic considerations that helped shape the finalized list of EM EPAs: attention to the meaningful nuances of EPA language and contextualizing the EPAs and viewing them developmentally.

Conclusions

To foreground all key stakeholders within the EPA process for EM, the authors chose within the development process to draft; come to consensus; and refine EPAs for EM in collaboration with relevant faculty, patient, and resident stakeholders. Each stakeholder group contributed meaningfully to the content and intended implementation of the EPAs. This process may serve as a model for others in developing stakeholder-responsive EPAs.

Development of entrustable professional activities for emergency medicine medical education fellowships: A modified Delphi study

Background

It is essential that medical education (MedEd) fellows achieve desired outcomes prior to graduation. Despite the increase in postgraduate MedEd fellowships in emergency medicine (EM), there is no consistently applied competency framework. We sought to develop entrustable professional activities (EPAs) for EM MedEd fellows.

Methods

From 2021 to 2022, we used a modified Delphi method to achieve consensus for EPAs. EM education experts generated an initial list of 173 EPAs after literature review. In each Delphi round, panelists were asked to make a binary choice of whether to include the EPA. We determined an inclusion threshold of 70% agreement a priori. After the first round, given the large number of EPAs meeting inclusion threshold, panelists were instructed to vote whether each EPA should be included in the "20 most important" EPAs for a MedEd fellowship. Modifications were made between rounds based on expert feedback. We calculated descriptive statistics.

Results

Seventeen experts completed four Delphi rounds each with 100% response. After Round 1, 87 EPAs were eliminated and two were combined. Following Round 2, 46 EPAs were eliminated, seven were combined, and three were included in the final list. After the third round, one EPA was eliminated and 13 were included. After the fourth round, 11 EPAs were eliminated. The final list consisted of 16 EPAs in domains of career development, education theory and methods, research and scholarship, and educational program administration.

Conclusions

We developed a list of 16 EPAs for EM MedEd fellowships, the first step in implementing competency-based MedEd.

Teaching residents to teach: A pilot study for an innovative online curriculum

Introduction

Resident-as-teacher (RAT) curricula have improved teaching behavior, ability, and confidence among resident participants. However, there are limited data on the appropriate format, length, and content. With teaching being a core residency competency and residents delivering one-third of student teaching in the clinical setting, properly training residents in clinical teaching is essential. We created a formal, scalable, asynchronous RAT curriculum. We report the pilot implementation of this curriculum along with feasibility, user acceptability, and preliminary knowledge outcomes.

Methods

In this pilot pre-post interventional study, senior emergency medicine residents completed a formalized online education curriculum during their "teach month." The curriculum consisted of three online modules completed weekly over a 4-week rotation. Topics included adult learning, assessment and feedback, and group teaching. Several surveys were sent to residents before and after curriculum implementation. The surveys rated satisfaction and asked several education-specific knowledge questions to assess learning. Ratings were analyzed using means and confidence intervals (95%). Knowledge questions were graded and then analyzed by ANOVA and Fisher's LSD test.

Results

After the online modules were completed, the intervention group residents' mean score on knowledge questions was significantly higher than that prior to the curriculum and significantly higher than that the control group (previous graduated residents; 6.00 vs. 2.70, p = 0.0001; and 6.00 vs. 3.00, p = 0.0003, respectively). This score was maintained 3 months after completing the online modules. Intervention group residents were more satisfied with the online education resources than the control group (p = 0.048).

Conclusions

Residents participating in a formalized online curriculum during their teach month demonstrate a high comprehension of education concepts and increased satisfaction with the provided educational resources and report high satisfaction with the teach month. Our pilot study suggests that a short online education-focused curriculum is an effective method of providing RAT training and may be applicable to clinical teachers across specialties and experience levels.

Five Key Articles on Curriculum Development for Graduate Medical Educators

Background Curriculum development is an essential domain for medical educators, yet specific training in this area is inconsistent. With competing demands for educators' time, a succinct resource for best practice is needed. Objective To create a curated list of the most essential articles on curriculum development to guide education scholars in graduate medical education. Methods We used a modified Delphi method, a systematic consensus strategy to increase content validity, to achieve consensus on the most essential curriculum development articles. We convened a panel of 8 experts from the United States in curricular development, with diverse career stages, institutions, gender, and specialty. We conducted a literature search across PubMed and Google Scholar with keywords, such as "curriculum development" and "curricular design," to identify relevant articles focusing on a general overview or approach to curriculum development. Articles were reviewed across 3 iterative Delphi rounds to narrow down those that should be included in a list of the most essential articles on curriculum development. Results Our literature search yielded 1708 articles, 90 of which were selected for full-text review, and 26 of which were identified as appropriate for the modified Delphi process. We had a 100% response rate for each Delphi round. The panelists narrowed the articles to a final list of 5 articles, with 4 focusing on the development of new curriculum and 1 on curriculum renewal. Conclusions We developed a curated list of 5 essential articles on curriculum development that is broadly applicable to graduate medical educators.

Postgraduate Selection in Medical Education: A Scoping Review of Current Priorities and Values

PURPOSE

The process of screening and selecting trainees for postgraduate training has evolved significantly in recent years, yet remains a daunting task. Postgraduate training directors seek ways to feasibly and defensibly select candidates, which has resulted in an explosion of literature seeking to identify root causes for the problems observed in postgraduate selection and generate viable solutions. The authors therefore conducted a scoping review to analyze the problems and priorities presented within the postgraduate selection literature to explore practical implications and present a research agenda.

METHOD

Between May 2021 and February 2022, the authors searched PubMed, EMBASE, Web of Science, ERIC, and Google Scholar for English language literature published after 2000. Articles that described postgraduate selection were eligible for inclusion. 2,273 articles were ultimately eligible for inclusion. Thematic analysis was performed on a subset of 100 articles examining priorities and problems within postgraduate selection. Articles were sampled to ensure broad thematic and geographical variation across the breadth of articles that were eligible for inclusion.

RESULTS

Five distinct perspectives or value statements were identified in the thematic analysis: (1) Using available metrics to predict performance in postgraduate training; (2) identifying the best applicants via competitive comparison; (3) seeking alignment between applicant and program in the selection process; (4) ensuring diversity, mitigation of bias, and equity in the selection process; and (5) optimizing the logistics or mechanics of the selection process.

CONCLUSIONS

This review provides insight into the framing and value statements authors use to describe postgraduate selection within the literature. The identified value statements provide a window into the assumptions and subsequent implications of viewing postgraduate selection through each of these lenses. Future research must consider the outcomes and consequences of the value statement chosen and the impact on current and future approaches to postgraduate selection.

Preparing for the PICU: A Qualitative Study of Residents as They Prepare for Their First Pediatric Critical Care Rotation

Learning critical care medicine in the pediatric intensive care unit (PICU) can be stressful. Through semistructured interviews ( n  = 16), this study explored the emotions, perceptions, and motivations of pediatric medicine (PM) and emergency medicine (EM) residents, as they prepared for their first PICU rotation. Qualitative data were collected and analyzed using the grounded theory method. Three resultant themes emerged: (1) residents entered the PICU with a range of intense emotions and heightened expectations; (2) they experienced prior history of psychologically traumatic learning events (adverse learning experiences or ALEs); and (3) informed by ALEs, residents prepared for their rotation by focusing heavily on their most basic level of physiological needs and adopting a survival mindset prior to the start of the rotation. These three themes led to a substantive, or working, theory that ALE-associated events may affect how residents approach upcoming learning opportunities. Consequently, adapting a trauma-informed approach as a component of medical education may improve resident learning experiences in the PICU and beyond.

Top emergency medicine faculty development papers since 2000: A Delphi study

Objectives

Faculty development is essential for academic emergency physicians to maintain clinical skills and succeed in administrative and leadership roles and for career advancement and satisfaction. Faculty developers in emergency medicine (EM) may struggle to find shared resources to guide faculty development efforts in a way that builds on existing knowledge. We aimed to review the EM-specific faculty development literature since 2000 and come to a consensus about the most useful for EM faculty developers.

Methods

A database search was conducted on the topic of faculty development in EM from 2000 to 2020. After identifying relevant articles, we performed a modified Delphi process in three rounds, using a team of educators with a range of experiences with faculty development and education research, to identify articles that would be most useful to a broad audience of faculty developers.

Results

We identified 287 potentially relevant articles on the topic of EM faculty development, 244 from the initial literature search, 42 from manual review of the references of the papers meeting inclusion criteria, and one by our study group's recommendation. Thirty-six papers met final inclusion criteria and underwent full-text review by our team. The Delphi process yielded six articles that were deemed most highly relevant over the three rounds. Each of these articles is described here, along with summaries and implications for faculty developers.

Conclusions

We present the most useful EM papers from the past two decades for faculty developers seeking to develop, implement, or revise faculty development interventions.

Faculty Development and the Emergency Medicine Educator: A National Needs Assessment

BACKGROUND

Faculty development (FD) encompasses structured programming that aims to enhance educator knowledge, skill, and behavior. No uniform framework for faculty development exists, and academic institutions vary in their faculty development programming, ability to overcome barriers, resource utilization, and achievement of consistent outcomes.

OBJECTIVE

The authors aimed to assess current FD needs among emergency medicine educators from six geographically and clinically distinct academic institutions to inform overall faculty development advancement in emergency medicine (EM).

METHODS

This cross-sectional study assessed FD needs among EM educators. A survey was developed, piloted, and distributed to faculty via each academic institution's internal e-mail listserv. Respondents were asked to rate their comfort level with and interest in several domains of FD. Respondents were also asked to identify their previous experience, satisfaction with the FD they have received, and barriers to receiving FD.

RESULTS

Across six sites, 136/471 faculty completed the survey in late 2020 (response rate of 29%): 69.1% of respondents reported being satisfied overall with the FD they have participated in, and 50.7% reported being satisfied with education FD specifically. Faculty report higher comfort levels and interest in several domains when satisfied with the education-specific FD they have received compared with those who report not being satisfied.

CONCLUSIONS

EM faculty report generally high satisfaction with the overall faculty development they have received, although only half express satisfaction with their education-related faculty development. Faculty developers in EM may incorporate these results to inform future faculty development programs and frameworks.

Promotion and tenure letters: A guide for faculty

Promotion and tenure (P&T) letters are a key component of the academic advancement portfolio. Despite their importance, many faculty are not trained to write these letters and there is limited literature describing the approach and key components. This paper reviews the role of P&T letters and provides general guidelines for writers. We present a step-by-step guide, which includes how to respond to requests, the role of institutional guidelines, providing context to the letter, evaluating candidates, and delivering an overall recommendation. Finally, we discuss current controversies in P&T letters. This paper is intended to help novice and more experienced writers to enhance their P&T letters, while also helping applicants for promotion understand what is being asked of their letter writers.

Beyond the CLAIM: A comprehensive needs assessment strategy for creating an Advanced Medical Education Research Training Program (ARMED-MedEd)

BACKGROUND

The health professions education (HPE) landscape has shifted substantively with increasing professionalization of research and scholarship. Clinician educators seeking to become competitive in this domain often pursue fellowships and master's degrees in HPE, but there are few resources for the continuing professional development (CPD) of those who seek to continue developing their scholarly practice within HPE. Acknowledging the multiple players in this landscape, the authors sought to design a new "beyond beginners" HPE research program using a novel needs assessment planning process.

METHODS

The authors developed and conducted a new three-phase, five-step process that sets forth a programmatic approach to conducting a needs assessment for a CPD course in HPE research. The five steps of the CLAIM method are: Competitive analysis, Literature review with thematic analysis, Ask stakeholders, Internal review by experts, and Mapping of a curriculum. These steps are organized into three phases (Discovery, Convergence, and Synthesis).

RESULTS

Over a 12-month period, the authors completed a comprehensive needs assessment. The CLAIM process revealed that longitudinal digital connection, diverse and in depth exposure to HPE research methods, skills around scholarly publishing, and leadership and management of research would be beneficial to our design.

CONCLUSIONS

The CLAIM method provided scaffolding to help the authors create a robust curriculum that adopts a scholarly approach for developing a HPE research course. This needs assessment methodology may be useful in other CPD contexts.

Planning for diverse, equitable, inclusive research in health professions education: An integral thread in the ARMED MedEd research course

Racism in medicine affects patients, trainees, and practitioners and contributes to health care inequities. An effective strategy to actively oppose the structural racism ingrained in the fabric of medicine is to intentionally and systematically address diversity, equity, and inclusion (DEI) in medical education and research. As part of ARMED MedEd, a new longitudinal cohort course in advanced research methods in medical education, sponsored by the Society for Academic Emergency Medicine, the leadership team deliberately included a nested DEI curriculum. The goal of the DEI curriculum is to reduce bias in development, recruitment, and implementation of education research studies to promote equity and inclusion in medical education, research, and ultimately, patient care. A team of medical educators with expertise in DEI developed curricular elements focusing on DEI in education research. The two major components are a didactic curriculum (including implicit bias training) to teach researchers to consider equity as they design studies and a consultative service to refine research protocols to address lingering unintended bias. A dedicated focus on DEI can be incorporated into an advanced education research methodology course to raise awareness and provide tools to avoid bias in research design and implementation of interventions. Over time, the network of education researchers who are trained in DEI awareness will grow and provide equitable offerings to their learners to mitigate health inequities.

Leveling the field: Development of reliable scoring rubrics for quantitative and qualitative medical education research abstracts

BACKGROUND

Research abstracts are submitted for presentation at scientific conferences; however, criteria for judging abstracts are variable. We sought to develop two rigorous abstract scoring rubrics for education research submissions reporting (1) quantitative data and (2) qualitative data and then to collect validity evidence to support score interpretation.

METHODS

We used a modified Delphi method to achieve expert consensus for scoring rubric items to optimize content validity. Eight education research experts participated in two separate modified Delphi processes, one to generate quantitative research items and one for qualitative. Modifications were made between rounds based on item scores and expert feedback. Homogeneity of ratings in the Delphi process was calculated using Cronbach's alpha, with increasing homogeneity considered an indication of consensus. Rubrics were piloted by scoring abstracts from 22 quantitative publications from AEM Education and Training "Critical Appraisal of Emergency Medicine Education Research" (11 highlighted for excellent methodology and 11 that were not) and 10 qualitative publications (five highlighted for excellent methodology and five that were not). Intraclass correlation coefficient (ICC) estimates of reliability were calculated.

RESULTS

Each rubric required three rounds of a modified Delphi process. The resulting quantitative rubric contained nine items: quality of objectives, appropriateness of methods, outcomes, data analysis, generalizability, importance to medical education, innovation, quality of writing, and strength of conclusions (Cronbach's α for the third round = 0.922, ICC for total scores during piloting = 0.893). The resulting qualitative rubric contained seven items: quality of study aims, general methods, data collection, sampling, data analysis, writing quality, and strength of conclusions (Cronbach's α for the third round = 0.913, ICC for the total scores during piloting = 0.788).

CONCLUSION

We developed scoring rubrics to assess quality in quantitative and qualitative medical education research abstracts to aid in selection for presentation at scientific meetings. Our tools demonstrated high reliability.

Moving beyond personal factors: A national study of wellness interventions in emergency medicine residency programs

BACKGROUND

In 2017, the Accreditation Council for Graduate Medical Education (ACGME) revised its Common Program Requirements to support trainees and faculty by mandating programs to provide dedicated wellness resources and education. Emergency medicine may benefit from this change due to high burnout rates within the specialty. However, the current state of wellness interventions in emergency medicine (EM) residency programs has not yet been well described. Understanding current practices is necessary to assess unmet needs and inform the development and evaluation of future interventions that aim to improve trainee wellness.

OBJECTIVES

The goal of this study was to describe currently implemented wellness interventions in EM residency programs.

METHODS

This descriptive study surveyed 250 ACGME-accredited EM residency programs between March 1 and June 1, 2020, regarding wellness interventions. The survey included demographic questions; structured multiple-choice questions about cost, frequency, and champions; and free-text response options to briefly describe interventions. Respondents were also asked to classify the interventions according to the seven factors described in the National Academy of Medicine Model of Clinician Well-Being and Resilience.

RESULTS

Ninety of 250 (36% response rate) residency programs participated, describing 162 unique wellness interventions. Respondents classified the majority of interventions (n = 136) as targeting personal factors according to the National Academy of Medicine model. Qualitative analysis revealed five major themes describing the interventions: program culture, program factors, environmental and clinical factors, wellness activities and practices, and wellness resources.

CONCLUSIONS

Results of this survey may help to inform a national needs assessment addressing the current state of wellness interventions in EM residency programs. Our results highlight the need for more interventions targeting external factors impacting resident wellness.

Coaching educators: Impact of a novel national faculty development program for didactic presentation skills

BACKGROUND

Didactic lectures remain common in medical education. Many faculty physicians do not receive formal training on public presentations or leading instructional sessions. Coaching has emerged in medical education with the potential to positively impact skills. We sought to evaluate a novel, national faculty peer-coaching program created to improve lecture presentation skills and foster career development.

METHODS

This was a mixed-methods study of participant and faculty perceptions after completing the Council of Residency Directors in Emergency Medicine Academy Coaching Program. Participants completed an online evaluative survey consisting of multiple choice and Likert-type items. Program coaches participated in semistructured interviews. Descriptive statistics were reported for survey data. Thematic qualitative analysis by two independent reviewers was performed on interview data.

RESULTS

During 2012 to 2017, a total of 30 participants and 11 coaches from 37 residency programs across the United States engaged in the program. Twenty-four (80%) participants completed the survey. Eight (73%) coaches participated in semistructured interviews. Data were collected between October and December 2018. The mean ± SD numbers of national presentations participants had given before and after the coaching program were 6.92 ± 7.68 and 16.42 ± 15.43, respectively. Since their coaching, most participants (87.5%) have been invited to give a lecture at another institution. Many participants felt that the program improved their lecture evaluations, public speaking, ability to engage an audience, and professional development. Almost all (92%) would recommend the program to a colleague. The coaches perceived multiple benefits including improved skills, self-reflection, networking, career advancement, and personal fulfillment. Suggestions for improvement included improved administrative processes, more clear expectations, increased marketing, and increased participant and coach engagement.

CONCLUSION

Participants and coaches perceived multiple benefits from this novel, national faculty coaching program. With identification of the success, challenges, and suggestions for improvement, others may benefit as they develop coaching programs in medical education.

Faculty development using a virtual community of practice: Three-year outcomes of the Academic Life in Emergency Medicine Faculty Incubator program

INTRODUCTION

The Academic Life in Emergency Medicine (ALiEM) Faculty Incubator program is a longitudinal, 1-year, virtual faculty development program for early- and mid-career faculty members that crosses specialties and institutions. This study sought to evaluate the outcomes among 3 years of participants.

METHODS

This cross-sectional survey study evaluated postcourse and 1-year outcomes from three graduated classes of the ALiEM Faculty Incubator program. The program evaluation survey was designed to collect outcomes across multiple Kirkpatrick levels using pre/post surveys and tracking of abstracts, publications, speaking opportunities, new leadership positions, and new curricula.

RESULTS

Over 3 years, 89 clinician educators participated in the program. Of those, 59 (66%) completed the initial survey and 33 (37%) completed the 1-year survey. Participants reported a significant increase in knowledge (4.1/9.0 vs. 7.0/9.0). The number of abstracts, publications, and invited presentations significantly increased after course completion and continued postcourse. A total of 37 of 59 (62.7%) developed a new curriculum during the course and 19 of 33 (57.6%) developed another new curriculum after the course. A total of 29 of 59 (49.2%) began a new leadership position upon course completion with 15 of 33 (45.5%) beginning another new leadership position 1 year later.

DISCUSSION

The ALiEM Faculty Incubator program demonstrated an increase in perceived knowledge and documented academic productivity among early- and mid-career medical educators.

Bias in assessment: name, reframe, and check in

Cognitive bias permeates almost every learner assessment in medical education. Assessment bias has the potential to affect a learner's education, future career and sense of self-worth. Decades of data show that there is little educators can do to overcome bias in learner assessments. Using in-group favouritism as an example, we offer an evidence-based, three-step solution to understand and move forward with cognitive bias in assessment: (1) Name: a simple admission about the presence of inherent bias in assessment, (2) Reframe: a rephrasing of assessment language to shed light on the assessor's subjectivity and (3) Check-in: a chance to ensure learner understanding and open lines of bidirectional communication. This process is theory-informed and based on decades of educational, sociological and psychological literature; we offer it as a logical first step towards a much-needed paradigm shift towards addressing bias in learner assessment.

Wellness Interventions in Emergency Medicine Residency Programs: Review of the Literature Since 2017

Introduction

Recent research demonstrates burnout prevalence rates as high as 76% in emergency medicine (EM) residents. In 2017 the Accreditation Council for Graduate Medical Education (ACGME) required that all training programs provide dedicated wellness education for their trainees as a requirement for accreditation. We aimed to conduct a systematic review of published wellness interventions conducted in EM residency programs following the implementation of the 2017 ACGME Common Program Requirements change in order to characterized published intervention and evaluate their effectiveness.

Methods

We applied a published approach to conducting systematic reviews of the medical education literature. We performed a search of the literature from January 1, 2017–February 1, 2020. Studies were included for final review if they described a specific intervention and reported outcomes with the primary goal of improving EM resident wellness. Outcomes were characterized using the Kirkpatrick training evaluation model.

Results

Eight of 35 identified studies met inclusion criteria. Most described small convenience samples of EM residents from single training programs and used the satisfaction rates of participants as primary outcome data. Only quantitative assessment methods were used. Studies addressed only a limited number of factors affecting resident wellness. The majority of interventions focused on personal factors, although a few also included sociocultural factors and the learning and practice environment.

Conclusion

There is a relative dearth of literature in the area of research focused on interventions designed to improve EM resident wellness. Furthermore, the studies we identified are narrow in scope, involve relatively few participants, and describe programmatic changes of limited variety. Future directions include an increase and emphasis on multi-institutional studies, randomized controlled trials, qualitative methodology, and opportunities for funded research.

Reimagining Residency Selection: Part 3-A Practical Guide to Ranking Applicants in the Post-COVID-19 Era

Collapse

Emergency Medicine Education Research Since the 2012 Consensus Conference: How Far Have We Come and What's Next?

In 2012, the Society for Academic Emergency Medicine convened a consensus conference on the state of medical education research with goals of defining and advancing a future research agenda. Since that time, emergency medicine (EM) education research has grown significantly. A task force of EM education experts was assembled and sought to understand the current state of EM education research and future directions. Among the advances are increases in medical education fellowships, advanced degree and certification programs, faculty development programs, publication venues, and funding. These findings are discussed in light of the prior objectives from the 2012 consensus conference, and recommendations for future directions are provided.

Comparison of Male and Female Resident Milestone Assessments During Emergency Medicine Residency Training: A National Study

PURPOSE

A previous study found that milestone ratings at the end of training were higher for male than for female residents in emergency medicine (EM). However, that study was restricted to a sample of 8 EM residency programs and used individual faculty ratings from milestone reporting forms that were designed for use by the program's Clinical Competency Committee (CCC). The objective of this study was to investigate whether similar results would be found when examining the entire national cohort of EM milestone ratings reported by programs after CCC consensus review.

METHOD

This study examined longitudinal milestone ratings for all EM residents (n = 1,363; 125 programs) reported to the Accreditation Council for Graduate Medical Education every 6 months from 2014 to 2017. A multilevel linear regression model was used to estimate differences in slope for all subcompetencies, and predicted marginal means between genders were compared at time of graduation.

RESULTS

There were small but statistically significant differences between males' and females' increase in ratings from initial rating to graduation on 6 of the 22 subcompetencies. Marginal mean comparisons at time of graduation demonstrated gender effects for 4 patient care subcompetencies. For these subcompetencies, males were rated as performing better than females; differences ranged from 0.048 to 0.074 milestone ratings.

CONCLUSIONS

In this national dataset of EM resident milestone assessments by CCCs, males and females were rated similarly at the end of their training for the majority of subcompetencies. Statistically significant but small absolute differences were noted in 4 patient care subcompetencies.

Defining Trainee Competence: Value Is in the Eye of the Stakeholder

In this issue of Academic Medicine, Lundsgaard and colleagues present "Embracing Multiple Stakeholder Perspectives in Defining Trainee Competence," a qualitative exploration of how various stakeholders contribute to the understanding of trainee competence. Drawing on stakeholder theory from business management, the authors of that report explore how the perceptions of key stakeholders (leaders/administrators, nurses/nurse practitioners, trainees, and patients) either confirm, enhance, or complicate the picture of competence that emerges from the perspectives of senior physician supervisors.In this Invited Commentary, the author considers the potential effects of applying stakeholder theory to educational assessment and elaborates on Lundsgaard and colleagues' findings that additional stakeholder perceptions may be redundant in some instances and present conflicting understandings of competence in others. She also emphasizes the importance of a process of defining competence, and ultimately developing assessment programs, that includes measuring and illuminating conflicting perceptions of what constitutes competence. Finally, the author considers potential applications of Lundsgaard and colleagues' results for educators and assessors with comprehensive assessment programs already in place.There are limitations and risks to applying a conceptual framework that aims to increase value for a company to the training of physicians. However, with thoughtful application, incorporating multiple stakeholders into instrument development and workplace-based assessment may provide educators with a more nuanced picture of competence that incorporates the values of those ultimately impacted by trainee performance.

Critical Appraisal of Emergency Medicine Educational Research: The Best Publications of 2016

OBJECTIVES

The objectives were to critically appraise the emergency medicine (EM) medical education literature published in 2016 and review the highest-quality quantitative and qualitative studies.

METHODS

A search of the English language literature in 2016 querying MEDLINE, Scopus, Education Resources Information Center (ERIC), and PsychInfo identified 510 papers related to medical education in EM. Two reviewers independently screened all of the publications using previously established exclusion criteria. The 25 top-scoring quantitative studies based on methodology and all six qualitative studies were scored by all reviewers using selected scoring criteria that have been adapted from previous installments. The top-scoring articles were highlighted and trends in medical education research were described.

RESULTS

Seventy-five manuscripts met inclusion criteria and were scored. Eleven quantitative and one qualitative papers were the highest scoring and are summarized in this article.

CONCLUSION

This annual critical appraisal series highlights the best EM education research articles published in 2016.

The ALiEM Faculty Incubator: A Novel Online Approach to Faculty Development in Education Scholarship

PROBLEM

Early- and midcareer clinician educators often lack a local discipline-specific community of practice (CoP) that encourages scholarly activity. As a result, these faculty members may feel disconnected from other scholars.

APPROACH

Academic Life in Emergency Medicine (ALiEM) piloted the Faculty Incubator. This longitudinal, asynchronous, online curriculum focused on developing a virtual CoP among 30 early- to midcareer medical educators (the "incubatees"), 8 core faculty mentors, and 10 guest mentors. The yearlong curriculum included 12 monthly modules focusing on core concepts in medical education scholarship. The initiative connected the incubatees with a virtual community of peers and mentors, with whom they completed multiple scholarly projects, sought mentorship, and engaged professionally. The authors used an online, closed, social media platform (Slack) to facilitate the exchange of ideas.

OUTCOMES

In the inaugural year (March 2016-February 2017), the mentorship team facilitated exceptional levels of online engagement among incubatees. All participants (incubatees, core mentors, and guest mentors) shared 1,081 files and exchanged a total of 22,665 messages (approximately 62 per day). Of these, 3,036 (13.4%) were via open channels, 5,483 (24.2%) via small groups, and 14,146 (62.4%) via direct messages.

NEXT STEPS

The ALiEM Faculty Incubator represents a proof of concept, and initial outcomes show that it is possible to engage an international group of early- to midcareer medical educators to create a vibrant online CoP. The Faculty Incubator leaders plan to determine whether this engaged group of health professions educators will increase their scholarly output as a result of this initiative.

Identifying the Emergency Medicine Personality: A Multisite Exploratory Pilot Study

OBJECTIVES

This study aimed to understand the personality characteristics of emergency medicine (EM) residents and assess consistency and variations among residency programs.

METHODS

In this cross-sectional study, a convenience sample of residents (N = 140) at five EM residency programs in the United States completed three personality assessments: the Hogan Personality Inventory (HPI)-describing usual tendencies; the Hogan Development Survey (HDS)-describing tendencies under stress or fatigue; and the Motives, Values, and Preferences Inventory (MVPI)-describing motivators. Differences between EM residents and a normative population of U.S. physicians were examined with one-sample t-tests. Differences between EM residents by program were analyzed using one-way analysis of variance tests.

RESULTS

One-hundred forty (100%), 124 (88.6%), and 121 (86.4%) residents completed the HPI, HDS, and MVPI, respectively. For the HPI, residents scored lower than the norms on the adjustment, ambition, learning approach, inquisitive, and prudence scales. For the HDS, residents scored higher than the norms on the cautious, excitable, reserved, and leisurely scales, but lower on bold, diligent, and imaginative scales. For the MVPI, residents scored higher than the physician population norms on altruistic, hedonistic, and aesthetics scales, although lower on the security and tradition scales. Residents at the five programs were similar on 22 of 28 scales, differing on one of 11 scales of the HPI (interpersonal sensitivity), two of 11 scales of the HDS (leisurely, bold), and three of 10 scales of the MVPI (aesthetics, commerce, and recognition).

CONCLUSIONS

Our findings suggest that the personality characteristics of EM residents differ considerably from the norm for physicians, which may have implications for medical students' choice of specialty. Additionally, results indicated that EM residents at different programs are comparable in many areas, but moderate variation in personality characteristics exists. These results may help to inform future research incorporating personality assessment into the resident selection process and the training environment.

A Descriptive Analysis of the Use of Twitter by Emergency Medicine Residency Programs

BACKGROUND

Twitter is increasingly recognized as an instructional tool by the emergency medicine (EM) community. In 2012, the Council of Residency Directors in Emergency Medicine (CORD) recommended that EM residency programs' Twitter accounts be managed solely by faculty. To date, little has been published regarding the patterns of Twitter use by EM residency programs.

OBJECTIVE

We analyzed current patterns in Twitter use among EM residency programs with accounts and assessed conformance with CORD recommendations.

METHODS

In this mixed methods study, a 6-question, anonymous survey was distributed via e-mail using SurveyMonkey. In addition, a Twitter-based search was conducted, and the public profiles of EM residency programs' Twitter accounts were analyzed. We calculated descriptive statistics and performed a qualitative analysis on the data.

RESULTS

Of 168 Accreditation Council for Graduate Medical Education-accredited EM programs, 88 programs (52%) responded. Of those programs, 58% (51 of 88) reported having a program-level Twitter account. Residents served as content managers for those accounts in the majority of survey respondents (61%, 28 of 46). Most programs did not publicly disclose the identity or position of their Twitter content manager. We found a wide variety of applications for Twitter, with EM programs most frequently using Twitter for educational and promotional purposes. There is significant variability in the numbers of followers for EM programs' Twitter accounts.

CONCLUSIONS

Applications and usage among EM residency programs are varied, and are frequently not consistent with current CORD recommendations.

The View From the Top: Academic Emergency Department Chairs' Perspectives on Education Scholarship

Education scholarship continues to grow within emergency medicine (EM) and in academic medicine in general. Despite a growing interest, would-be education scholars often struggle to find adequate mentorship, research training, funding, and protected time to produce rigorous scholarship. The ways in which individual academic EM departments can support this mission remains an area in need of description.

OBJECTIVES

We sought to describe academic EM department chairs' perceptions of education scholarship and facilitators and barriers to producing high-quality education scholarship.

METHODS

We conducted a qualitative study using a grounded theory-derived approach. Participants were solicited directly, and semistructured interviews were conducted via telephone. Interviews were transcribed verbatim and were analyzed by three study investigators using a coding matrix. Discrepancies in coding were resolved via in depth discussion.

RESULTS

We interviewed seven EM chairs from academic departments throughout North America (six in geographically diverse regions of the United States and one in western Canada). Chairs described education scholarship as lacking clearly defined and measurable outcomes, as well as methodologic rigor. They identified that education faculty within their departments need training and incentives to pursue scholarly work in a system that primarily expects teaching from educators. Chairs acknowledged a lack of access to education research expertise and mentorship within their own departments, but identified potential resources within their local medical schools and universities. They also voiced willingness to support career development opportunities and scholarly work among faculty seeking to perform education research.

CONCLUSIONS

Academic EM chairs endorse a need for methodologic training, mentorship, and access to expertise specific to education scholarship. While such resources are often rare within academic EM departments, they may exist within local universities and schools of medicine. Academic EM chairs described themselves as willing and able to support faculty who wish to pursue this type of work.

Critical Appraisal of Emergency Medicine Educational Research: The Best Publications of 2015

OBJECTIVE

The objectives were to critically appraise the medical education research literature of 2015 and review the highest-quality quantitative and qualitative examples.

METHODS

A total of 434 emergency medicine (EM)-related articles were discovered upon a search of ERIC, PsychINFO, PubMED, and SCOPUS. These were both quantitative and qualitative in nature. All were screened by two of the authors using previously published exclusion criteria, and the remaining were appraised by all authors using a previously published scoring system. The highest scoring articles were then reviewed.

RESULTS

Sixty-one manuscripts were scored, and 10 quantitative and two qualitative papers were the highest scoring and are reviewed and summarized in this article.

CONCLUSIONS

This installment in this critical appraisal series reviews 12 of the highest-quality EM-related medical education research manuscripts published in 2015.

Critical Appraisal of Emergency Medicine Educational Research: The Best Publications of 2015

OBJECTIVE

The objectives were to critically appraise the medical education research literature of 2015 and review the highest-quality quantitative and qualitative examples.

METHODS

A total of 434 emergency medicine (EM)-related articles were discovered upon a search of ERIC, PsychINFO, PubMED, and SCOPUS. These were both quantitative and qualitative in nature. All were screened by two of the authors using previously published exclusion criteria, and the remaining were appraised by all authors using a previously published scoring system. The highest scoring articles were then reviewed.

RESULTS

Sixty-one manuscripts were scored, and 10 quantitative and two qualitative papers were the highest scoring and are reviewed and summarized in this article.

CONCLUSIONS

This installment in this critical appraisal series reviews 12 of the highest-quality EM-related medical education research manuscripts published in 2015.

Taking Your Qualitative Research to the Next Level: A Guide for the Medical Educator

Qualitative research is a focused and deliberate approach to understanding the human condition and can identify areas that warrant further investigation through hypothesis-testing (quantitative) research. In this overview article, we discuss reasons to undertake a qualitative study; introduce three research paradigms whose frameworks are commonly used by medical education researchers (grounded theory, ethnography, phenomenology); describe the four most commonly used data gathering techniques (interviews, focus groups, observation, document tracing); and discuss the nature of the sampling/data gathering process. Examples of research questions that employ each paradigm and data gathering technique as well as a list of published sample articles to guide researchers are provided.

Creating a Cadre of Fellowship-Trained Medical Educators, Part II: A Formal Needs Assessment to Structure Postgraduate Fellowships in Medical Education Scholarship and Leadership

PURPOSE

Education leaders at the 2012 Academic Emergency Medicine Consensus Conference on education research proposed that dedicated postgraduate education scholarship fellowships (ESFs) might provide an effective model for developing future faculty as scholars. A formal needs assessment was performed to understand the training gap and inform the development of ESFs.

METHOD

A mixed-methods needs assessment was conducted of four emergency medicine national stakeholder groups in 2013: department chairs; faculty education/research leaders; existing education fellowship directors; and current education fellows/graduates. Descriptive statistics were reported for quantitative data. Qualitative data from semistructured interviews and free-text responses were analyzed using a thematic approach.

RESULTS

Participants were 11/15 (73%) education fellowship directors, 13/20 (65%) fellows/graduates, 106/239 (44%) faculty education/research leaders, and a convenience sample of 26 department chairs. Department chairs expected new education faculty to design didactics (85%) and teach clinically (96%). Faculty education/research leaders thought new faculty were inadequately prepared for job tasks (83.7%) and that ESFs would improve the overall quality of education research (91.1%). Fellowship directors noted that ESFs provide skills, mentorship, and protected time for graduates to become productive academicians. Current fellows/graduates reported pursing an ESF to develop skills in teaching and research methodology.

CONCLUSIONS

Stakeholder groups uniformly perceived a need for training in education theory, clinical teaching, and education research. These findings support dedicated, deliberate training in these areas. Establishment of a structure for scholarly pursuits prior to assuming a full-time position will effectively prepare new faculty. These findings may inform the development, implementation, and curricula of ESFs.

Developing and Implementing a Multisource Feedback Tool to Assess Competencies of Emergency Medicine Residents in the United States

OBJECTIVES

Multisource feedback (MSF) has potential value in learner assessment, but has not been broadly implemented nor studied in emergency medicine (EM). This study aimed to adapt existing MSF instruments for emergency department implementation, measure feasibility, and collect initial validity evidence to support score interpretation for learner assessment.

METHODS

Residents from eight U.S. EM residency programs completed a self-assessment and were assessed by eight physicians, eight nonphysician colleagues, and 25 patients using unique instruments. Instruments included a five-point rating scale to assess interpersonal and communication skills, professionalism, systems-based practice, practice-based learning and improvement, and patient care. MSF feasibility was measured by percentage of residents who collected the target number of instruments. To develop internal structure validity evidence, Cronbach's alpha was calculated as a measure of internal consistency.

RESULTS

A total of 125 residents collected a mean of 7.0 physician assessments (n = 752), 6.7 nonphysician assessments (n = 775), and 17.8 patient assessments (n = 2,100) with respective response rates of 67.2, 75.2, and 77.5%. Cronbach's alpha values for physicians, nonphysicians, patients, and self were 0.97, 0.97, 0.96, and 0.96, respectively.

CONCLUSIONS

This study demonstrated that MSF implementation is feasible, although challenging. The tool and its scale demonstrated excellent internal consistency. EM educators may find the adaptation process and tools applicable to their learners.

Constructing a Shared Mental Model for Faculty Development for the Core Entrustable Professional Activities for Entering Residency

In 2014, the Association of American Medical Colleges identified 13 Core Entrustable Professional Activities for Entering Residency (Core EPAs), which are activities that entering residents might be expected to perform without direct supervision. This work included the creation of an interinstitutional concept group focused on faculty development efforts, as the processes and tools for teaching and assessing entrustability in undergraduate medical education (UME) are still evolving. In this article, the authors describe a conceptual framework for entrustment that they developed to better prepare all educators involved in entrustment decision making in UME. This framework applies to faculty with limited or longitudinal contact with medical students and to those who contribute to entrustment development or render summative entrustment decisions.The authors describe a shared mental model for entrustment that they developed, based on a critical synthesis of the EPA literature, to serve as a guide for UME faculty development efforts. This model includes four dimensions for Core EPA faculty development: (1) observation skills in authentic settings (workplace-based assessments), (2) coaching and feedback skills, (3) self-assessment and reflection skills, and (4) peer guidance skills developed through a community of practice. These dimensions form a conceptual foundation for meaningful faculty participation in entrustment decision making.The authors also differentiate between the UME learning environment and the graduate medical education learning environment to highlight distinct challenges and opportunities for faculty development in UME settings. They conclude with recommendations and research questions for future Core EPA faculty development efforts.