USMLE Step-1 is Going to Pass/Fail, Now What Do We Do?

Journey of International Medical Graduates Toward Orthopaedic Surgery Residency in the United States

International medical graduates (IMGs) are integral to the US healthcare system but are scarce in orthopaedic residency classes. Positive contributions of IMGs to the field of orthopaedic surgery in the United States are well documented, but successfully matching into an orthopaedic residency position as an IMG remains very challenging. The purpose of this study was to review current processes, strategies, and potential barriers of IMGs applying for orthopaedic surgery residency in the United States as an IMG.

USMLE Performance, Subsequent Standardized Testing, and ABIM Certification Exam Preparation for Internal Medicine Residency Programs: A Narrative Review

Standardized examinations measure progress throughout medical education. Successful completion of the American Board of Internal Medicine Certification Examination (ABIM-CE) benchmarks completion of internal medicine (IM) residency training. Recent declines in initial ABIM-CE pass rates may prompt residency programs to examine strategies to improve learner performance. We synthesized published literature on associations between the United States Medical Licensing Examination (USMLE), in-training examination (ITE), and board preparation to support residents for ABIM-CE. We searched MEDLINE for test performance and preparation strategies for IM board certification during training. Relevant articles published until March 15, 2024, were screened using pre-defined criteria for narrative review, then codified into three domains (USMLE, ITE, curriculum/program strategies). Findings were grouped by theme into considerations for training programs seeking guidance on learning augmentation plans to improve resident performance on ABIM-CE. Themes drawn from articles focused on USMLE include validity in predicting CE performance, noting (1) failing USMLE Step 1 is associated with failing ABIM-CE, (2) any USMLE score < 220 increases failure probability, and (3) a mean USMLE ≥ 250 approximates ~ 100% pass rates on board examination. Inferences from ITE-focused articles support use as a predictive tool; specifically, a score < 35th percentile signals a resident at risk for failing the ABIM-CE while > 70th percentile is predictive of passing. Lastly, inferences from curriculum- and program-focused articles suggest standard contents (conferences) do not correlate with CE passage, while targeted clinical reasoning and remediation plans do. IM residency programs should consider adopting learning augmentation strategies targeted to at-risk residents to support CE passage.

Family Medicine Clerkship Directors' Perspectives on USMLE Pass/Fail Scoring: A CERA Study

INTRODUCTION

Reports on the effects of changing the United States Medical Licensing Exam (USMLE) Step 1 examination scoring to pass/fail are evolving in the medical literature. This Council of Academic Family Medicine Educational Research Alliance family medicine clerkship directors' study seeks to describe family medicine clerkship directors' perceptions on the impact of incorporation of Step 1 pass/fail score reporting on students' family medicine clerkship performance.

METHODS

Ninety-six clerkship directors responded (56.8% response rate). After exclusion of Canadian schools, we analyzed 88 clerkship directors' responses from US schools. We used descriptive statistics for demographics and responses to survey questions. We used ꭓ2 analysis to determine statistically significant associations between survey items.

RESULTS

Clerkship directors did not observe changes in students' overall clinical performance after Step 1 pass/fail scoring (60.8%). Fifty percent of clerkship directors reported changes in Step 1 timing recommendations in the past 3 years. Reasons included curriculum redesign (30.5%), COVID (4.5%), change in Step 1 to pass/fail (11.0%), and other reasons (3.7%). Forty-five percent of these clerkship directors did not observe a change in students' clinical medical knowledge after Step 1 went to pass/fail. Eighty-four percent of these clerkship directors did not compare student performance on clerkship standardized exams before and after Step 1 score changes. We found no significant relationship between Step 1 timing and student performance.

CONCLUSIONS

This study represents an early description of family medicine clerkship directors' perceived observations of the impact of Step 1 scoring changes on student performance. Continued investigation of the effects of USMLE Step 1 pass/fail scoring should occur.

A Longitudinal Evaluation of Collaboration in Plastic Surgery Clinical Research

Background

Few dedicated, funded clinical research fellowship positions exist in plastic surgery. This study provides insights from an established clinical research fellowship, highlighting its educational impact and confirming the impact of positive institutional support and a collaborative-first approach.

Methods

All research fellows within the program from 2008 to 2020 were examined during their year of employment and subsequent 2 years. Internal and external collaboration trends were assessed using PubMed affiliations. Correlation between external collaborations and research impact were examined. Research impact was characterized by publication count, journal impact factor, and journal diversity.

Results

Thirty-one research fellows were identified, with a 100% match rate. Four phases of development were identified, and a three-period cycle of productivity was noted to occur every 3 years. A shift toward more external collaborations occurred after 2016 (P = 0.008). A positive correlation was observed between external collaborations and academic output (r = 0.72, P = 0.004), journal diversity (r = 0.74, P = 0.004), and journal impact (r = 0.63, P < 0.05). Significant growth was observed in the collaborative networks (P = 0.002), publications (P = 0.003), journal diversity (P < 0.001), and research personnel (P = 0.002).

Conclusions

As a result of our strategic decision to engage collaborators across multiple disciplines, there is discernible improvement in measurable impact, contributing to the growth of our program. Dedicating resources to foster deeper collaborations can enrich the field of plastic surgery research, recognizing that this investment fuels the cycle of productivity, offering promising returns to the future.

The Transition from Letter Grading to Modified Pass-Fail Grading at a College of Veterinary Medicine: A Narrative Inquiry of Student Experiences

Compared with traditional letter grading, pass/fail grading is an assessment approach that can alter the educational environment and enhance student wellbeing without compromising education quality. Little is known about the experiences of students during the transition from traditional grading to pass/fail grading. The onset of the COVID-19 pandemic resulted in an abrupt move to modified pass-fail (MPF) grading at North Carolina State University College of Veterinary Medicine (NCSU-CVM), followed by a decision to permanently adopt MPF grading for the entire core pre-clinical doctor of veterinary medicine (DVM) curriculum. This study employed a narrative inquiry of surveys and interviews to facilitate deep understanding of student perspectives during the transition to MPF grading. Focus was placed on understanding what this transition meant for DVM students in terms of life and learning quality. Our analysis identified seven key themes that captured student experiences, which were generally positive, during this transition: education culture, student perceptions of instructor impact, shift from extrinsic to intrinsic valuation of curricular content, competitiveness for external merit-based opportunities, use of letter grading and MPF in a single semester, student recommendations, and wellbeing. Through exploration of these themes and presentation of concerns identified in students' stories, this study provides guidance for other programs considering revision of their own assessment frameworks.

Advising Orthopaedic Hopefuls in the Wake of a Pass/Fail US Medical Licensing Examination Step 1: A Proposed Method for Predicting US Medical Licensing Examination Step 2 Clinical Knowledge Outcomes

INTRODUCTION

Matching into orthopaedic residency has become difficult, and the US Medical Licensing Examination Step 1 transition to pass/fail scoring has complicated the process. Advisors' ability to mentor students has decreased, and program directors may rely on Step 2 Clinical Knowledge (CK) scores in selecting which candidates to interview. This study aims to offer a method to predict Step 2 CK outcomes based on preadmission and preclinical performance.

METHODS

The study investigated 486 students from a US medical school who enrolled in 2017 and 2018. Data on demographics, preadmission, and preclinical performance were collected. Before model creation, it was found that sex, Medical College Admission Test scores, Comprehensive Basic Science Examination performance, and preclinical curriculum performance produced optimal models. Multivariate ordinal logistic regression models were built to predict probabilities of four outcome levels of Step 2 CK: <235, 235 to 249, 250 to 265, and >265. Finally, nomograms were created to visualize probability calculations.

RESULTS

Each model's odds ratios revealed that female sex, higher MCAT scores, and better Comprehensive Basic Science Examination and preclinical performance were associated with an increased likelihood of being in higher Step 2 CK scoring groups. Preclinical performance had a profound effect, especially for those in the top 1/3. Models were successful in assigning higher probabilities to students in higher Step 2 CK scoring groups in more than 80% of instances. Nomograms presented provide examples of how to apply these models to an individual student.

DISCUSSION

This study presents a novel method for predicting probabilities of Step 2 CK outcomes that can be used to mentor students at a time point when Step 1 previously filled this role. It may assist in identifying orthopaedic hopefuls at risk of performing poorly on Step 2 CK and can foster the development of individualized guidance and mitigation strategies.

A Hands-On Surgical Event to Improve Medical Student Operating Room Confidence

INTRODUCTION

The negative perceptions and lack of exposure to surgery and the operating room (OR) have been known to divert students away from surgical specialties. This study describes the impact of a surgical subspecialty exposure event (OR Essentials), combined with surgical faculty and M4 mentorship on preclinical medical students' confidence at an academic medical center.

METHODS

OR essentials event teaches surgical skills to preclinical medical students through hands-on skill-based workshops in a simulated OR setting. Pre and postevaluations were administered to measure program impact.

RESULTS

One hundred four preclinical medical students participated. Following OR essentials, students reported a significant increase in confidence in the OR (P < 0.0001) and in basic surgical skills (P < 0.0001).

CONCLUSIONS

Early surgical exposure events like OR essentials provide opportunities to improve medical student confidence in the OR, which will hopefully support recruitment of future surgeons.

Differences in the Academic Attributes of Matched and Unmatched Orthopaedic Surgery Residency Applicants are Narrowing

Orthopaedic surgery remains one of the most competitive residency specialties, with the number of applicants outpacing the availability of residency positions each year. The purpose of this study was to analyze present-day orthopaedic surgery match data, identify differences between matched and unmatched applicants, and compare our findings to previous trends.

Methods

Applicant data from the National Resident Matching Program from 2016 to 2022 were analyzed. The number of matched and unmatched US allopathic senior orthopaedic applicants relative to the number of available positions was used to determine respective match rates. Performance metrics and applicant characteristics were compared by match status. Trends were compared with those of previous analysis from 2006 to 2014.

Results

The number of applicants increased from 863 in 2016 to 1,068 in 2022. The match rate decreased from 75% in 2016 to 66% in 2022 (p < 0.0001). Matched applicants had a higher number of contiguous ranks (12.3 vs. 6.5; p < 0.001), United States Medical Licensing Examination (USMLE) Step-1 score (248 vs. 240; p < 0.001), USMLE Step-2 score (255 vs. 247; p < 0.001), Alpha Omega Alpha (AOA) membership (38% vs. 13%; p < 0.001), and enrollment at a top 40 National Institutes of Health (NIH)-funded medical school (34% vs. 24%, p < 0.001). Compared with 2006 to 2014 data, a smaller percentage of matched applicants were enrolled in a top 40 NIH-funded medical school (34% vs. 37%, p = 0.013). The mean differences in USMLE Step-1 score (16 vs. 8.25 points, p < 0.001) and USMLE Step-2 score (16 vs. 8.25 points, p = 0.002) in favor of matched applicants nearly halved compared with that in 2006 to 2014. In addition, there was no longer a significant difference in the number of research products (abstracts, presentations, posters, and publications) between matched and unmatched applicants (p = 0.309).

Conclusions

Differences in the academic attributes of matched and unmatched orthopaedic surgery applicants have become less profound over time, making it increasingly difficult to predict a successful match based on USMLE Step scores, AOA membership, research productivity, and medical school research reputation. Future studies should evaluate differences in subjective metrics (e.g., away rotation and interview performance and letters of recommendation) by match status.

One test to rule them all: A qualitative study of formal, informal, and hidden curricula as drivers of USMLE "exam mania"

High-stakes examinations are an integral part of medical education. To practice in the United States (U.S.), students must pass the U.S. Medical Licensing Examinations (USMLE). With the transition of USMLE Step 1 to pass/fail scoring on January 26, 2022, a worldwide debate regarding how residency program directors will view the Step 2 Clinical Knowledge (CK) exam emerged. Here, the authors explore the role of formal, informal, and hidden curricula related to USMLE, with broader implications for high-stakes examinations. Six focus groups of fourth-year students who recently took Step 2 CK and a supplemental curricular content analysis were conducted to explore students' decision-making and emotions regarding the exam, including how the formal, informal, and hidden curricula influence their perspectives. Participants highlighted how informal and hidden curricula drive the belief that high-stakes examinations are the single most important factor in medical school. Prior experience with Step 1 drives behaviors and attitudes when preparing for Step 2 CK. Pressures from these examinations have unintended consequences on burnout, professional identity, specialty choice, and interpersonal interactions. Both interpersonal interactions within medical education as well as subconscious, unintended messaging can influence medical student approaches to and perspectives about high-stakes examinations. Within the context of U.S. medical training, with the transition to a new era of a pass/fail Step 1 examination, careful consideration to prevent shifting the current "Step 1 mania" to a "Step 2 CK mania" is warranted. More broadly, medical educators must examine the unintended yet potentially damaging pressures institutions generate in their medical trainees in relation to high-stakes examinations.

The US Residency Selection Process After the United States Medical Licensing Examination Step 1 Pass/Fail Change: Overview for Applicants and Educators

The United States Medical Licensing Examination (USMLE) Step 1, arguably the most significant assessment in the USMLE examination series, changed from a 3-digit score to a pass/fail outcome in January 2022. Given the rapidly evolving body of literature on this subject, this paper aims to provide a comprehensive review of the historical context and impact of this change on various stakeholders involved in residency selection. For this, relevant keyword-based searches were performed in PubMed, Google Scholar, and Scopus to identify relevant literature. Given the unique history of USMLE Step 1 in the US residency selection process and the score's correlation with future performance in board-certifying examinations in different specialties, this scoring change is predicted to significantly impact US Doctor of Medicine students, US Doctor of Osteopathic Medicine students, international medical graduates, and residency program directors, among others. The significance and the rationale of the pass/fail change along with the implications for both residency applicants and educators are also summarized in this paper. Although medical programs, academic institutions, and residency organizing bodies across the United States have swiftly stepped up to ensure a seamless transition and have attempted to ensure equity for all, the conversion process carries considerable uncertainty for residency applicants. For educators, the increasing number of applications conflicts with holistic application screening, leading to the expected greater use of objective measures, with USMLE Step 2 Clinical Knowledge likely becoming the preferred screening tool in lieu of Step 1.

Potential Impact of Pass/Fail Scores on USMLE Step 1: Predictors of Excellence in Obstetrics and Gynecology Residency Training

AIM

The study aims to determine resident applicant metrics most predictive of academic and clinical performance as measured by the Council of Resident Education in Obstetrics and Gynecology (CREOG) examination scores and Accreditation Council for Graduate Medical Education (ACGME) clinical performance (Milestones) in the aftermath of United States Medical Licensing Examination Scores (USMLE) Step 1 becoming a pass/fail examination.

METHODS

In this retrospective study, electronic and paper documents for Wayne State University Obstetrics and Gynecology residents matriculated over a 5-year period ending July 2018 were collected. USMLE scores, clerkship grade, and wording on the letters of recommendation as well as Medical Student Performance Evaluation (MSPE) were extracted from the Electronic Residency Application Service (ERAS) and scored numerically. Semiannual Milestone evaluations and yearly CREOG scores were used as a marker of resident performance. Statistical analysis on residents (n = 75) was performed using R and SPSS and significance was set at P < .05.

RESULTS

Mean USMLE score correlated with CREOG performance and, of all 3 Steps, Step 1 had the tightest association. MSPE and class percentile also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with resident performance. Of all metrics provided by ERAS, none taken alone, were as useful as Step 1 scores at predicting performance in residency. Regression modeling demonstrated that the combination of Step 2 scores with MSPE wording restored the predictive ability lost by Step 1.

CONCLUSIONS

The change of USMLE Step 1 to pass/fail may alter resident selection strategies. Other objective markers are needed in order to evaluate an applicant's future performance in residency.