US Medical Licensing Exam Performance and American Board of Surgery Qualifying and Certifying Examinations

John A, Kavic SM. A scoping review of successful strategies for passing the American Board of Surgery certifying examination

Background

This scoping review identifies existing literature that investigates what factors contribute to success on the American Board of Surgery (ABS) Certifying Exam (CE) to provide practical, evidence-based recommendations.

Methods

A Pubmed search was completed utilizing the preferred reporting items for systematic reviews and meta-analysis extension for scoping review (PRISMA-ScR) method.

Results

Of 4368 articles identified, 45 articles met criteria for review. Manuscripts were placed into one of five categories: predictors from medical school, program interventions, modifiable candidate factors, the effect of mock oral exams, and those factors shown not to provide benefit for CE preparation.

Conclusions

A variety of factors have either been shown to provide benefit for or be predictive of CE performance. Acknowledgement of these factors can provide benefit to both surgery residents as well as surgery programs. Despite these findings, research into these factors is generally of low quality, prompting the need for ongoing, high-quality investigations.

Resident factors associated with American board of internal medicine certification exam failure

INTRODUCTION

Performance on the certifying examinations such as the American Board of Internal Medicine Certification Exam (ABIM-CE) is of great interest to residents and their residency programs. Identification of factors associated with certification exam result may allow residency programs to recognize and intervene for residents at risk of failing. Despite this, residency programs have few evidence-based predictors of certification exam outcome. The change to pass-or-fail score reporting of the USA Medical Licensing Exam (USMLE) Step 1 removes one such predictor.

MATERIALS AND METHODS

We performed a retrospective study of residents from a medium-sized internal medicine residency program who graduated from 1998 through 2017. We used univariate tests of associations between ABIM-CE result and various demographic and scholastic factors.

RESULTS

Of 166 graduates, 14 (8.4%) failed the ABIM-CE on the first attempt. Failing the first attempt of the ABIM-CE was associated with older median age on entering residency (29 vs 27 years; P = 0.01); lower percentile rank on the Internal Medicine In-Training Examination (IM-ITE) in each of the first, second, and third years of training (P < 0.001 for all); and lower scores on the USMLE Steps 1, 2 Clinical Knowledge, and 3 (P < 0.05 for all). No association was seen between a variety of other scholastic or demographic factors and first-attempt ABIM-CE result.

DISCUSSION

Although USMLE step 1 has changed to a pass-or-fail reporting structure, there are still other characteristics that allow residency programs to identify residents at risk of ABIM-CE first time failure and who may benefit from intervention.

The importance of USMLE step 2 on the screening and selection of applicants for general surgery residency positions

Background

As announced by the Federation of State Medical Boards (FSMB) and National Board of Medical Examiners (NBME), the United States Medical Licensing Examination (USMLE) Step 1 score reporting has transitioned to pass/fail outcomes instead of the traditional numeric score after January 26, 2022. USMLE Step 1 scores have been used widely as a crucial tool in screening and selecting applicants for residency programs. This study aims to determine the role of USMLE Step 2 in the selection of applicants for general surgery residency.

Methods

A retrospective study was conducted over six recruiting cycles from 2016 to 2021. The data from 334 interviewed applicants from one general surgery residency program were assessed. Data analyzed included USMLE Step 1 and Step 2 scores, applicant gender, Alpha Omega Alpha (AOA) status, letters of recommendation (LOR), and research/publications (RS).

Results

Of the 334 interviewed applicants, 209 (62.6%) were male. The mean [SD] USMLE Step 1 and USMLE Step 2 C K (Clinical Knowledge) scores were 239.6 [±10.4] and 249.2 [±11.4], respectively. The mean (SD) LOR and RS scores were 4.24 [±0.4] and 3.9 [±0.7], respectively. A positive correlation was observed between USMLE Step 1 and USMLE Step 2 C K (Clinical Knowledge) scores (r = 0.60, p < .001), LOR scores (r = 0.24, p = .008), and AOA status (r = 0.19, p = .038). There was a negligible correlation between USMLE scores and applicant gender.

Conclusion

Transitioning USMLE Step 1 to pass/fail will make the initial screening and selection process of applications challenging for residency programs. In the short term, USMLE Step 2 scores, LOR, and AOA status are important as screening assessments. Valid measures to ensure appropriate, equitable, and fair assessments are needed.

Simulation as Soothsayer: Simulated Surgical Skills MMIs During Residency Interviews are Associated With First Year Residency Performance

OBJECTIVE

The main consideration during residency recruitment is identifying applicants who will succeed during residency. However, few studies have identified applicant characteristics that are associated with competency development during residency, such as the Accreditation Council for Graduate Medical Education milestones. As mini multiple interviews (MMIs) can be used to assess various competencies, we aimed to determine if simulated surgical skills MMI scores during a general surgery residency interview were associated with Accreditation Council for Graduate Medical Education milestone ratings at the conclusion of intern year.

DESIGN

Retrospective cohort study. Interns' Step 1 and 2 clinical knowledge (CK) scores, interview day simulated surgical skills MMI overall score, traditional faculty interview scores, average overall milestone ratings in the spring of residency, and intern American Board of Surgery In-Training Examination (ABSITE) percentile scores were gathered. Two multiple linear regression were performed analyzing the association between Step 1, Step 2 CK, MMI, and traditional faculty interview scores with (1) average overall milestone rating and (2) ABSITE percentile scores, controlling for categorical/preliminary intern classification.

SETTING

One academic medical center PARTICIPANTS: General surgery interns matriculating in 2020-2021 RESULTS: Nineteen interns were included. Multiple linear regression revealed that higher overall simulated surgical skills MMI score was associated with higher average milestone ratings (β = .45, p = 0.03) and higher ABSITE score (β = .43, p = 0.02) while neither Step 1, Step 2 CK, nor faculty interview scores were significantly associated with average milestone ratings.

CONCLUSIONS

Surgical residency programs invest a tremendous amount of effort into training residents, thus metrics for predicting applicants that will succeed are needed. Higher scores on a simulated surgical skills MMIs are associated with higher milestone ratings 1 year into residency and higher intern ABSITE percentiles. These results indicate a noteworthy method, simulated surgical skills MMIs, as an additional metric that may select residents that will have early success in residency.

Considerations for a Holistic Model in Evaluating Medical Students for Cardiothoracic Surgical Residency

Program directors are tasked with selecting whom they think will be the best fit for residency and the next leaders of the field. While numerical metrics have played a vital role in this process, recent changes to student evaluation are reducing the availability of these metrics. This poses unique challenges for both applicants and program directors. Here we discuss how this will likely shift the focus on other parts of the application and the consequences (good and bad) of doing so.

A disturbing trend: An analysis of the decline in surgical critical care (SCC) fellowship training of Black and Hispanic surgeons

BACKGROUND

Underrepresented minorities in medicine (URiM) are disproportionally represented in surgery training programs. Rates of URiM applying to and completing General Surgery residency remain low. We hypothesized that the patterns of URiM disparities would persist into Surgical Critical Care (SCC) fellowship applicants, matriculants and graduates.

METHODS

We performed a retrospective analysis of SCC applicants, matriculants and graduates from 2005-2020 using the Graduate Medical Education (GME) resident survey and analyzed applicant characteristics using the Surgical critical care and Acute care surgery Fellowship Application Service (SAFAS) from 2018-2020. The data were stratified by race/ethnicity and gender. Indicator variables were created for Asian, Hispanic, White and Black trainees. Yearly proportions for each race/ethnicity and gender categories completing or enrolling in a program were calculated and plotted over time with Loess smoothing lines and overlying 95% confidence bands. The yearly rate and statistical significance of change over time were tested with linear regression models with race/ethnicity and gender proportion as the dependent variables and year as the explanatory variable.

RESULTS

From 2005-2020, there were a total of 2,481 graduates. Black men accounted for 4.7% of male graduates with a significant decline of 0.3% per year for the study period of those completing the fellowship (p = 0.02). Black women comprised 6.4% of female graduates and had a 0.6% decline each year (p < 0.01). A similar trend was seen with Hispanic men, who comprised 3.2% of male graduates and had a 0.3% annual decline (p = 0.02). White men had a significant increase in both matriculation to and graduation from SCC fellowships during the same interval. Similarly, Black and Hispanic applicants declined from 2019 to 2020, while the percentage of White applicants increased.

CONCLUSIONS

Disparities in URiM representation remain omnipresent in surgery and extend from residency training to SCC fellowship. Efforts to enhance the recruitment and retention of URiM in SCC training are warranted.

LEVEL OF EVIDENCE

Level IV - Therapeutic/Care Management.