Which Applicant Factors Predict Success in Emergency Medicine Training Programs? A Scoping Review

A Practical Guide to Implementing Holistic Review during Surgery Resident Selection

Provider workforce diversity is a key component of improving healthcare quality and addressing healthcare disparities. Furthermore, the traditional approach of "score-centered" application metrics do not consistently correlate with meeting milestones in surgery, nor do they adequately predict a surgical resident's clinical strength and operative abilities. We present here an adaptable process by which surgical residency programs can identify their values and incorporate holistic review into their resident selection process to improve resident selection and physician workforce diversity.

Residency Program Directors' Views on Research Conducted During Medical School: A National Survey

PURPOSE

With the United States Medical Licensing Examination Step 1 transition to pass/fail in 2022, uncertainty exists regarding how other residency application components, including research conducted during medical school, will inform interview and ranking decisions. The authors explore program director (PD) views on medical student research, the importance of disseminating that work, and the translatable skill set of research participation.

METHOD

Surveys were distributed to all U.S. residency PDs and remained open from August to November 2021 to query the importance of research participation in assessing applicants, whether certain types of research were more valued, productivity measures that reflect meaningful research participation, and traits for which research serves as a proxy. The survey also queried whether research would be more important without a numeric Step 1 score and the importance of research vs other application components.

RESULTS

A total of 885 responses from 393 institutions were received. Ten PDs indicated that research is not considered when reviewing applicants, leaving 875 responses for analysis. Among 873 PDs (2 nonrespondents), 358 (41.0%) replied that meaningful research participation will be more important in offering interviews. A total of 164 of 304 most competitive specialties (53.9%) reported increased research importance compared with 99 of 282 competitive (35.1%) and 95 of 287 least competitive (33.1%) specialties. PDs reported that meaningful research participation demonstrated intellectual curiosity (545 [62.3%]), critical and analytical thinking skills (482 [55.1%]), and self-directed learning skills (455 [52.0%]). PDs from the most competitive specialties were significantly more likely to indicate that they value basic science research vs PDs from the least competitive specialties.

CONCLUSIONS

This study demonstrates how PDs value research in their review of applicants, what they perceive research represents in an applicant, and how these views are shifting as the Step 1 exam transitions to pass/fail.

Medical School Clinical Knowledge Exam Scores, Not Demographic or Other Factors, Associated With Residency In-Training Exam Performance

BACKGROUND

The anesthesiology in-training exam (ITE) is a 200-item multiple-choice assessment completed annually by physician residents. Because all matriculated U.S. Department of Defense (DoD) anesthesiology residents are "hired" by the DoD after residency graduation, it is important to ensure that ITE performance, as a proxy for core competencies achievement, is maximized.

METHODS

Graduated resident program files from 2013 to 2020 were queried for age, sex, matriculant status (medical student vs. other), medical school (Uniformed Services University vs. other), military service (Army vs. Air Force), preresidency military service (yes vs. no), U.S. Medical Licensing Exam (USMLE) Step 2 Clinical Knowledge (CK) score, and the American Board of Anesthesiologists ITE Score from the third clinical anesthesia year (CA-3 year).

RESULTS

For every 1-point increase in USMLE Step 2 CK true z-score, the CA-3 ITE z-score increased by 0.59 points. Age was not associated with CA-3 ITE z-score in any dataset regression. Categorical covariates of sex, application status, medical school, service, and preresidency military service were not significantly associated with CA-3 ITE z-score (all P >.05), as shown by estimated adjusted marginal means. The estimated adjusted grand mean of CA-3 ITE z-scores was 0.48 (standard error ± 0.14).

CONCLUSION

Resident physicians enter residency with varying degrees of past academic success, and it is important to develop early strategies to support them in acquiring the requisite knowledge base.

Predictors of future academic productivity in pediatric otolaryngology fellowship applicants

OBJECTIVES

To investigate which components of pediatric otolaryngology fellowship applications are more closely predictive of future academic productivity in applicants who go on to complete their fellowship training.

METHODS

Applications to our institution's ACGME accredited pediatric otolaryngology fellowship program through the SF Match program for the years 2011-2016 were reviewed. Applicant files on record were utilized to extract independent variables including sex, mean USMLE score, residency program Doximity ranking, military experience, number of national honors/awards, AOA status, total number of publications listed on application, number of first author publications listed on application, and AAOHNS Committee involvement. Academic productivity was determined by number of PubMed indexed publications per year, practice setting, and H-index (Scopus). Statistical analysis consisted of multivariate and univariate regression models, with p < 0.05 being considered statistically significant.

RESULTS

Multivariate regression showed that USMLE Step 1 and 2 mean score and number of publications listed on application exhibited statistically significant correlations with a higher number of future post fellowship publications per year. Residency program Doximity rank, applicant number of awards and honors, AOA status, and number of first author publications were not predictive of future academic productivity. No statistically significant associations were found between any variables and the faculty position outcome variable.

CONCLUSIONS

Quantifiable criteria in pediatric otolaryngology fellowship applications, such as number of listed publications and mean USMLE scores are strongly correlated with future academic productivity metrics.

Responding to Recommended Changes to the 2020-2021 Residency Recruitment Process From a Diversity, Equity, and Inclusion Perspective

In May 2020, the Coalition for Physician Accountability's Work Group on Medical Students in the Class of 2021 Moving Across Institutions for Post Graduate Training (WG) released its final report and recommendations. These recommendations pertain to away rotations, virtual interviews, Electronic Residency Application Service opening for programs and the overall residency timeline, and general communications and attempt to provide clarity and level the playing field during the 2020-2021 residency application cycle. The WG's aims include promoting professional accountability by improving the quality, efficiency, and continuity of the education, training, and assessment of physicians. The authors argue the first 3 WG recommendations may disproportionately impact candidates from historically excluded and underrepresented groups in medicine (HEURGMs) and may affect an institution's ability to ensure equity in the selection of residency applicants and, thus, warrant further consideration. The authors examine these recommendations from a diversity, equity, and inclusion (DEI) perspective. For each of the first 3 WG recommendations, the authors highlight new opportunities created by the recommendations and detail challenges that programs must carefully navigate to ensure equity for all candidates. The authors also recommend solutions to guide programs as they address these challenges, meet new common program requirements, and attempt to promote equity for HEURGMs. Finally, the authors recommend that after the 2020-2021 recruitment cycle, the medical education community evaluate DEI-related outcomes of both the WG's and the authors' recommendations and incorporate the findings into future application cycles.

Metrics of Resident Achievement for Defining Program Aims

Introduction

Resident achievement data is a powerful but underutilized means of program evaluation, allowing programs to empirically measure whether they are meeting their program aims, facilitate refinement of curricula and improve resident recruitment efforts. The goal was to provide an overview of available metrics of resident achievement and how these metrics can be used to inform program aims.

Methods

A literature search was performed using PubMed and Google Scholar between May and November of 2020. Publications were eligible for inclusion if they discussed or assessed “excellence” or “success” during residency training. A narrative review structure was chosen due to the intention to provide an examination of the literature on available resident achievement metrics.

Results

57 publications met inclusion criteria and were included in the review. Metrics of excellence were grouped into larger categories, including success defined by program factors, academics, national competencies, employer factors, and possible new metrics.

Conclusions

Programs can best evaluate whether they are meeting their program aims by creating a list of important resident-level metrics based on their stated goals and values using one or more of the published definitions as a foundation. Each program must define which metrics align best with their individual program aims and mission.

Implementation of holistic review into emergency medicine residency application screening to improve recruitment of underrepresented in medicine applicants

INTRODUCTION

Increasing the diversity of the emergency medicine (EM) workforce is imperative, with more diverse teams showing improved patient care and increased innovation. Holistic review, adapted from the Association of American Medical Colleges (AAMC), focuses on screening applicants with a balanced method, valuing their experiences, attributes, and academic metrics equally. A core tenet to holistic review is that diversity is essential to excellence.

OBJECTIVE

Implementation of holistic review into the residency application screening process is effective at improving exposure to underrepresented in medicine (URiM) applicants.

METHODS

After adjustment of our residency application screening rubric, improving our balance across the experience, attributes, and metrics domains, we conducted a retrospective cohort study comparing the representation of URiM applicants invited to interview, interviewed, and ranked by composite score compared to our previous primarily metric-based process.

RESULTS

A total of 8,343 applicants were included in the study. Following implementation of holistic review, we saw an increase in the absolute percent of URiM applicants invited to interview (+11%, 95% confidence interview [CI] = 6.9% to 15.4%, p < 0.01), interviewed (+7.9%, 95% CI = 3.6% to 12.2%, p < 0.01), and represented in the top 75 through top 200 cutpoints based on composite score rank. The mean composite score for URiM applicants increased significantly compared to non-URiM applicants (+9.7, 95% CI = 8.2 to 11.2, p < 0.01 vs. +4.7, 95% CI = 3.5 to 5.9, p < 0.01).

CONCLUSION

Holistic review can be used as a systematic and equitable tool to increase the exposure and recruitment of URiM applicants in EM training programs.

USMLE Step-1 is Going to Pass/Fail, Now What Do We Do?

The Winter 2021 Webinar Audio Series (WAS) of the International Association of Medical Science Educators (IAMSE), titled, "USMLE Step-1 is Going to Pass/Fail, Now what do we do?" was broadcast live to audiences at academic institutions worldwide in five weekly webinars from January 7, 2021, to February 4, 2021. Recognized experts from various stakeholder groups discussed the impact of the decision to score the United States Medical Licensing Examination (USMLE) Step 1 exam Pass/Fail (P/F). The speakers identified challenges to their respective programs and explored creative ways to address potential consequences. Sessions included the perspectives of allopathic and osteopathic residency program directors, basic science faculty, undergraduate medical education curriculum designers, clinical educators, and programs for international medical students matriculating to the United States. On February 25, 2021, a bonus session provided cutting-edge updates from a co-chair of the Coalition for Physician Accountability Undergraduate Medical Education (UME) to Graduate Medical Education (GME) Review Committee (UGRC).

Curated collection for clinician educators: Six key papers on residency recruitment

INTRODUCTION

All emergency medicine (EM) residency programs must recruit new medical school graduates each year. The process is often overwhelming, with each program receiving far more applicants than available positions. We searched for evidence-based best practices to guide residency programs in screening, interviewing, and ranking applicants to ensure a high-performing and diverse residency class.

METHODS

A literature search was conducted on the topic of residency recruitment, utilizing a call on social media as well as multiple databases. After identifying relevant articles, we performed a modified Delphi process in three rounds, utilizing junior educators as well as more senior faculty.

RESULTS

We identified 51 relevant articles on the topic of residency recruitment. The Delphi process yielded six articles that were deemed most highly relevant over the three rounds. Transparency with selection criteria, holistic application review, standardized letters of evaluation, and blinding applicant files for interviewers were among noted best practices.

CONCLUSIONS

Well-supported evidence-based practices exist for residency recruitment, and programs may benefit from understanding which common recruitment practices offer the most value. The articles discussed here provide a foundation for faculty looking to improve their program's recruiting practices.