The USMLE Step 1 Pass/Fail Reporting Proposal: Another View

International Medical Graduates in Radiology Residencies: Demographics, Performance, and Visa Issues

RATIONALE AND OBJECTIVES

The number of international medical graduates (IMG) in radiology residencies has varied from year to year even as the number of candidates continues to grow. It is unclear from which countries the IMGs are arriving and what visas are being used to accommodate them.

MATERIALS AND METHODS

We sent a survey to 195 program directors (PD) in diagnostic radiology (DR) inquiring about the number and nationality of IMG residents in their program, their attitudes about IMG candidates, the performance of their IMG trainees, and the visas that are offered.

RESULTS

We received responses from 121 of 195 (62.1%) DR programs (121/149 =81.2% of actionable emails). 80/121 (66.1%) had at least one IMG in their DR residency program and the countries of origin included India (36), Iran (30), Saudi Arabia (24), Egypt (16), Canada (14), Brazil (14), and Pakistan (9), as the most common. While most programs (76/104, 73.1%) offered J1 visas, 23/99 (23.2%) provided H-1B visas to trainees. IMG DR residents overall performed as well as American graduates, with an equal number of PDs saying IMGs performed better and worse than American graduates. PDs' issues with IMGs centered on visas: (1) expense, (2) lack of familiarity, (3) Educational Commission for Foreign Medical Graduates regulations, and (4) time commitment in submitting paperwork.

CONCLUSION

Most radiology IMG residents originate from India and Middle Eastern countries. Once enrolled, IMG residents perform similarly to US graduates. However, adding IMG candidates to the training program requires overcoming bureaucratic and monetary hurdles around visas.

SUMMARY SENTENCE

Most DR international medical graduate residents are from India or the Middle East. Although their performance is the same as American graduates in general, PDs note the monetary and bureaucratic hassles accompanying their recruitment.

Radiology program director's perspective on a novel hands-on advanced elective

With the advent of the USMLE Step 1 exam moving to a pass/fail status, Radiology Program Directors (PDs) and Associate Program Directors (APDs) need alternative methods of identifying interested and engaged medical students who are applying to their program. Additionally, undergraduate radiology medical education in the United States varies widely from institution to institution with no universal mandatory radiology component. To address these problems, we implemented an advanced fourth year hands-on radiology elective where the students were treated as first year radiology residents (R1s), giving them resident-level access to the Picture Archive and Communication System (PACS) and dictation software, and allowing them to perform entry-level procedures with appropriate supervision. After implementation of the elective, a 5-question online survey was sent to two hundred and ninety-eight PDs and APDs via the Association of Program Directors in Radiology (APDR) listserv, of which seventy-two responses were compiled, yielding a response rate of 24%. The survey focused on how a hands-on medical student elective would help in assessing prospective candidates and predicting R1 performance. Most respondents felt interest in radiology, motivation, and interpersonal skills would be better assessed after such an elective and the vast majority felt hands-on Advanced Elective would be at least slightly predictive of first year resident performance. Based on this information, we believe implementing a hands-on advanced radiology elective would significantly help address the passive nature of traditional radiology electives, providing valuable information to PDs and APDs and giving the best possible radiology experience to our medical students.

The Pass/Fail Effect: A Longitudinal Study of United States Medical Licensing Examination (USMLE) Step 1 Performance Over a Decade

Objectives This study aimed to analyze the impact of the United States Medical Licensing Examination (USMLE) Step 1 transition to a pass/fail scoring system in 2022 on the performance of first-time test takers in three distinct groups: Doctor of Osteopathy (DO) and Doctor of Medicine (MD) examinees from US/Canadian schools and examinees from non-US/Canadian schools. The analysis spans a decade-long period from 2012 to 2022, offering insights into the implications of this pivotal change in medical education. Methods We analyzed the performance of first-time USMLE Step 1 examinees from US/Canadian MD and DO programs and non-US/Canadian schools from 2012 to 2022, including the transition year to a pass/fail scoring system. Data were obtained from USMLE performance data reports and organized into annual contingency tables. Descriptive statistics and comparative analysis were used to identify trends and differences in performance across the groups. Data visualization techniques were employed to illustrate these findings, and the results were contextualized within the broader changes in medical education. Results In 2021, first-time takers from US/Canadian MD and DO Degree programs had pass rates of 96% and 94%, respectively, while non-US/Canadian schools had a pass rate of 82%. However, in 2022, these rates dropped to 93%, 89%, and 74%, respectively. The most significant relative decline was observed among non-US/Canadian Schools' first-time takers, with an 8% decrease. Repeaters consistently had lower pass rates across all groups. Conclusion The study reveals a notable decline in pass rates following the transition to pass/fail scoring, although this is based on just one year of data. This underscores the importance of students not rushing into the exam and dedicating sufficient time for preparation. The potential impact of this research could be transformative for medical education, but more years of data post-transition will be needed to confirm these initial findings. These findings serve as a reminder that the change in scoring does not diminish the rigor of the exam, prompting students to approach their studies with diligence and patience and potentially paving the way for systemic improvements in medical education and healthcare delivery worldwide.

The Upcoming Pass/Fail USMLE Step 1 Score Reporting: An Impact Assessment From Medical School Deans

Background: The US Medical Licensing Examination (USMLE) Step 1 change to pass/fail has been met with mixed reviews, and the impact on medical student education and residency match is unknown. We surveyed medical school student affairs deans regarding their thoughts on the upcoming transition of Step 1 to pass/fail. Methods: A questionnaire was emailed to medical school deans. Deans were asked to rank the importance of the following after the Step 1 reporting change: Step 2 Clinical Knowledge (Step 2 CK), clerkship grades, letters of recommendation, personal statement, medical school reputation, class rank, Medical Student Performance Evaluation, and research. They were asked how the score change will affect curriculum, learning, diversity, and student mental health. Deans were asked to select 5 specialties they thought would be most affected. Results: Regarding perceived importance of residency applications following the scoring change, the most frequent number 1 choice was Step 2 CK. The majority of deans (93.5%, n = 43) felt that the change to pass/fail would benefit medical student education/learning environment; however, most (68.2%, n = 30) did not believe their school curriculum would change. Students applying to dermatology, neurosurgery, orthopedic surgery, ENT, and plastic surgery were felt to be most affected by the scoring change; 58.7% (n = 27) felt it would not adequately address future diversity. Conclusion: The majority of deans feel the USMLE Step 1 change to pass/fail would benefit medical student education. Deans feel that students applying to traditionally more competitive specialties (ie, programs with fewer overall residency positions available) will be most affected.

Interventions Associated With Racial and Ethnic Diversity in US Graduate Medical Education: A Scoping Review

IMPORTANCE

Racially and ethnically minoritized individuals remain underrepresented in graduate medical education relative to their proportion in the population. While many programs and initiatives have been developed to address this problem, there is little consensus regarding strategies that work to improve representation across specialties.

OBJECTIVE

To examine and synthesize evidence-based practices that have been used to increase the proportions of underrepresented in medicine (URiM) trainees at US residency and fellowship programs.

EVIDENCE REVIEW

The authors searched PubMed, Google Scholar, Embase, PsycInfo, ERIC, Cochrane Reviews, Cochrane Trials, CINAHL, Scopus, and PROSPERO electronic databases to identify relevant studies published through January 2022. They screened all titles and abstracts for relevance and read full-text articles to identify articles reporting reliable data describing the outcomes of interventions to improve racial and ethnic diversity among trainees.

FINDINGS

Twenty-seven articles were included in this review. Two studies reported on fellowship programs. The most common interventions included holistic review (48%), decreased emphasis on United States Medical Licensing Examination Step 1 scores (48%), and explicit institutional messaging regarding the importance of diversity (37%). A combination of interventions was associated with an increased number of URiM applicants, interviewees, and matriculants across various medical and surgical specialties.

CONCLUSIONS AND RELEVANCE

In this scoping review, approaches and interventions associated with increased diversity in residency and fellowship programs were identified. Continued efforts are necessary to sustain such efforts and assess long-term outcomes.

Medical Students and the Drive for a Single Right Answer: Teaching Complexity and Uncertainty

The arrival of Generation Z, the next generation of medical learners, has been accompanied by efforts to adapt teaching practices for this new group of students. Many have identified challenges associated with addressing the needs of modern medical learners. One particular trend we have observed is that medical students are increasingly requesting an “answer key” for all aspects of their medical education. Students often expect to have the correct answers readily available to them to ensure they have reached the correct conclusion and to determine the precise knowledge they need to master. Yet, for much of medicine, and particularly in the care of critically ill patients with multisystem disease in intensive care units, answers are uncertain, and the body of knowledge is ever-growing. Students’ regular requests for solutions to be provided to them threaten to undermine their development into critically thinking, self-sufficient physicians. We outline three potential contributors to this multifactorial problem and offer corresponding pedagogical solutions. Specifically, we address how prioritizing outcomes over process, discomfort with uncertainty, and fear of faltering can cause students to seek excessive levels of support that may ultimately do more harm than good. Addressing students’ concerns in these three key areas will not only serve students well during their undergraduate medical training but will also equip them with the skills needed to succeed in the clinical realm. To produce physicians capable of navigating an increasingly uncertain world, medical educators will need to help students appreciate that finding the answer is more complex than being provided an answer key.

Current Views on the New United States Medical Licensing Examination Step 1 Pass/Fail Format: A Review of the Literature

INTRODUCTION

Residency programs have historically used numerical Step 1 scores to screen applicants, making it a career-defining, high-stakes examination. Step 1 scores will be reported as pass/fail starting in January 2022, fundamentally reshaping the residency application review process. This review aimed to identify opinions of physicians and medical students about the new format, identify arguments in support of or against the change, and determine the implications of this change on the residency selection process.

METHODS

A comprehensive PubMed review was performed in May 2021 to identify articles that discussed the new Step 1 format. Non-English and duplicate articles were excluded. Data collected from each article included publication year, specialty, subjects, and key findings.

RESULTS

A total of 81 articles were included, 26 of which discussed the impact of the new format within surgical fields (32.1%). Remaining articles discussed the implications within the medical community as a whole (n = 33, 40.7%) and nonsurgical fields (n = 22, 27.2%). Studies suggest Program Directors will rely on Step 2 Clinical Knowledge (CK) scores, medical school reputation, applicant familiarity, Dean's letters, recommendation letters, and research in lieu of numerical Step 1 scores. In addition, concerns have been raised that the new format will disadvantage international, osteopathic, and minority applicants while increasing stress surrounding Step 2 CK.

CONCLUSIONS

Within the medical community, there are concerns that Step 2 CK will be used to substitute Step 1 and that resident diversity will diminish due to the new Step 1 format. Holistic candidate consideration will be increasingly important.

Pass/Fail USMLE Step 1 Scoring-A Radiology Program Director Survey

Rationale and Objectives

In February 2020, administrators for the US medical licensing examination (USMLE) announced that Step 1 reporting would change to pass/fail in hopes of reducing the overemphasis of USMLE performance on the residency selection system and improving medical student well-being. Our objective was to determine the perspectives of diagnostic radiology (DR), interventional radiology (IR), and nuclear medicine (NM) program directors (PDs) regarding pass/fail USMLE Step 1 scoring.

Materials and Methods

A survey composed of thirteen questions on a three-point Likert scale, five demographic questions, and a free-text question was distributed to 179 DR, 84 IR, and 34 NM PDs from ACGME-accredited residency programs.

Results

In total, 140 unique responses were obtained (response rate = 47.1%). The PD respondents had a male predominance of 79.1%, average age of 46 ± 7.2 years, and average tenure of 5.9 ± 5.2 years. A majority of PDs (69.6%) disagreed that the change is a good idea, and a minority (21.6%) believe the change will improve medical student well-being. Further, 90.7% of PDs believe a pass/fail format will make it more difficult to objectively compare applicants and most will place more emphasis on USMLE Step 2 scores and medical school reputation (89.3% and 72.7%, respectively).

Conclusion

The lasting impact of pass/fail Step 1 scoring are uncertain and many radiology PDs do not support this change. While the central motivations to reduce the overemphasis on USMLE Step 1 performance and improve medical student well-being are admirable, it remains to be seen if pass/fail scoring will accomplish these goals.

Surgical clerkship: Do examination scores correlate with clinical performance?

BACKGROUND

This study aims to determine if there are correlations between clinical performance and objective grading parameters for medical students in the third-year surgery clerkship.

METHODS

Clerkship grades were compiled from 2016 to 2020. Performance on clinical rotations, NBME shelf exam, oral exam, and weekly quizzes were reviewed. Students were divided into quartiles (Q1-Q4) based on clinical performance. Standard statistical analysis was performed.

RESULTS

There were 625 students included in the study. Students in Q1+Q2 were more likely than those in Q3+Q4 to score in the top quartile on the shelf exam (29% vs. 19%, p = 0.002), oral exam (24% vs. 17%, p = 0.032), and quizzes (22% vs. 15%, p = 0.024). However, there was negligible correlation between clinical performance and performance on objective measures: shelf exam (R² = 0.027, p < 0.001), oral exam (R² = 0.021, p < 0.001), and weekly quizzes (R² = 0.053, p = 0.092).

CONCLUSIONS

Clinical performance does not correlate with objective grading parameters for medical students in the third-year surgery clerkship.

Ethnic/Racial Bias in Medical School Performance Evaluation of General Surgery Residency Applicants

OBJECTIVE

Differential use of communal terms (caring/unselfish traits) versus agentic terms (goal-oriented/leadership/achievement traits) may reveal bias and has been extensively reported in letters of recommendation for residency. We evaluated bias in medical student performance evaluations (MSPE) of general surgery residency applicants.

DESIGN

This is a retrospective study evaluating ethnic/race bias, as measured by differential use of agentic and communal terms, in MSPEs of residency applicants. 50% of MSPEs were randomly selected. An ethnic bias calculator derived from an open-source online gender bias calculator was populated with a list of validated agentic and communal terms. Relative frequency of communal and agentic terms was used to estimate bias. Multivariable regression was used to assess the association between the terms and ethnicity/race.

PARTICIPANTS

US medical students applying for a categorical surgery residency position at a single academic institution for a single Match cycle.

RESULTS

A total of 339 MSPEs were reviewed from 119 US medical schools. Genders were equally represented (women, 51.6%); most participants were white and Asian applicants (79.1%). Overall, MSPEs were more agency biased (65.2%) than communal biased (16.2%) or neutral (18.6%). MSPEs for Black and Hispanic/Latinx applicants were more likely to contain communal rather than agentic terms (adjusted OR: 3.02, 95% CI: 1.52-6.02) when compared to white and Asian applicants. This finding was independent of MSPE writer's gender or rank.

CONCLUSIONS

Surgery residency applicants self-identifying as Black and Hispanic/Latinx were more likely to be described using communal traits compared to white and Asian applicants, suggesting ethnic/racial bias. Such differences in language utilized in MSPEs may impact residency opportunities for applicants who are under-represented in medicine. Educational efforts aimed at MSPE writers may help to reduce bias.

Factors Influential in the Selection of Radiology Residents in the Post-Step 1 World: A Discrete Choice Experiment

OBJECTIVES

Reporting of United States Medical Licensing Examination Step 1 results will transition from a numerical score to a pass or fail result. We sought an objective analysis to determine changes in the relative importance of resident application attributes when numerical Step 1 results are replaced.

METHODS

A discrete choice experiment was designed to model radiology resident selection and determine the relative weights of various application factors when paired with a numerical or pass or fail Step 1 result. Faculty involved in resident selection at 14 US radiology programs chose between hypothetical pairs of applicant profiles between August and November 2020. A conditional logistic regression model assessed the relative weights of the attributes, and odds ratios (ORs) were calculated.

RESULTS

There were 212 participants. When a numerical Step 1 score was provided, the most influential attributes were medical school (OR: 2.35, 95% confidence interval [CI]: 2.07-2.67), Black or Hispanic race or ethnicity (OR: 2.04, 95% CI: 1.79-2.38), and Step 1 score (OR: 1.8, 95% CI: 1.69-1.95). When Step 1 was reported as pass, the applicant's medical school grew in influence (OR: 2.78, 95% CI: 2.42-3.18), and there was a significant increase in influence of Step 2 scores (OR: 1.31, 95% CI: 1.23-1.40 versus OR 1.57, 95% CI: 1.46-1.69). There was little change in the relative influence of race or ethnicity, gender, class rank, or clerkship honors.

DISCUSSION

When Step 1 reporting transitions to pass or fail, medical school prestige gains outsized influence and Step 2 scores partly fill the gap left by Step 1 examination as a single metric of decisive importance in application decisions.

Radiology resident selection factors predict resident performance

PURPOSE

To determine selection factors that predict radiology resident performance.

METHODS

59 consecutive radiology residents from 2002 to 2015 were ranked on performance during residency. Correlations and multiple regression analyses were performed to predict resident performance from the following selection factors: United States Medical Licensing Exam (USMLE) Step 1 score, medical school rank, Alpha Omega Alpha (AOA) membership, honors in clinical rotations, Medical Student Performance Evaluation (MSPE), and interview score. Results were compared against predictions from Match rank position.

RESULTS

Five selection factors showed significant or marginally significant correlations with resident performance (r = 0.2 to 0.3). The interview score was not significantly correlated. A multiple regression model comprised of the USMLE Step 1 score, medical school rank, AOA membership, and interview score predicted resident performance, with an adjusted R² of 0.19. The interview score was included in the model but did not achieve statistical significance. Match rank did not predict resident performance, with an R² of 0.01.

CONCLUSIONS

A multiple regression model comprised of the USMLE Step 1 score, medical school rank, and AOA membership predicted radiology resident performance and may assist with resident selection.

Medical Student Attitudes toward USMLE Step 1 and Health Systems Science - A Multi-Institutional Survey

Phenomenon: Because of its importance in residency selection, the United States Medical Licensing Examination Step 1 occupies a critical position in medical education, stimulating national debate about appropriate score use, equitable selection criteria, and the goals of undergraduate medical education. Yet, student perspectives on these issues and their implications for engagement with health systems science-related curricular content are relatively underexplored. Approach: We conducted an online survey of medical students at 19 American allopathic medical schools from March-July, 2019. Survey items were designed to elicit student opinions on the Step 1 examination and the impact of the examination on their engagement with new, non-test curricular content related to health systems science. Findings: A total of 2856 students participated in the survey, representing 23.5% of those invited. While 87% of students agreed that doing well on the Step 1 exam was their top priority, 56% disagreed that studying for Step 1 had a positive impact on engagement in the medical school curriculum. Eighty-two percent of students disagreed that Step 1 scores should be the top item residency programs use to offer interviews. When asked whether Step 1 results should be reported pass/fail with no numeric score, 55% of students agreed, while 33% disagreed. The majority of medical students agreed that health systems science topics were important but disagreed that studying for Step 1 helped learn this content. Students reported being more motivated to study a topic if it was on the exam, part of a course grade, prioritized by residency program directors, or if it would make them a better physician in the future. Insights: These results confirm the primacy of the United States Medical Licensing Examination Step 1 exam in preclinical medical education and demonstrate the need to balance the objectives of medical licensure and residency selection with the goals of the broader medical profession. The survey responses suggest several potential solutions to increase student engagement in health systems science curricula which may be especially important after Step 1 examination results are reported as pass/fail.

Residency match interview scheduling: quantifying the applicant experience

BACKGROUND

The process of offering and scheduling residency interviews varies widely among programmes. Applicants report distress and have advocated for reform. However, there is a paucity of quantitative data to characterise applicant concerns.

OBJECTIVE

We quantified the interview scheduling experience for US allopathic medical students in the 2020 main residency match.

METHODS

An anonymous, 13 question survey was sent to student representatives from each Association of American Medical Colleges member institution. Recipients were asked to forward the survey to their entire fourth-year class.

RESULTS

Of 4314 applicants to whom the survey was sent, 786 (18.2%) responded. Overall, 20.4% reported missing the opportunity to interview at a programme because they did not have adequate time to respond to an invitation; applicants into surgical specialties were significantly more likely than their non-surgical peers to report this experience (26.4% vs 18.4%, p<0.05). Most (57.4%) respondents scheduled an interview knowing they would likely cancel it in the future. The most commonly cited reason for this behaviour was concern that applicants would not receive invitations from other programmes (85.6%). A majority (56.4%) of respondents did not believe the match interview process functions based on equity and merit.

CONCLUSIONS

About one in five respondents missed the opportunity to interview at a programme because they did not respond to an invitation in time. Most respondents scheduled interviews knowing that they were likely to cancel them in the future. Standardisation of the interview invitation timeline would address these concerns.

The New USMLE Step 1 Paradigm: An Opportunity to Cultivate Diversity of Excellence

The February 2020 announcement that United States Medical Licensing Examination (USMLE) Step 1 results will be reported as pass/fail instead of numerical scores has been controversial. Step 1 scores have played a key role in residency selection, including screening for interviews. Although Step 1 scores are viewed as an objective criterion, they have been shown to disadvantage female and underrepresented minority applicants, cause student anxiety and financial burden, and affect student well-being. Furthermore, Step 1 scores incompletely predict applicants' overall residency performance. With this paradigm shift in Step 1 score reporting, residency programs will have fewer objective, standardized metrics for selection decisions, which may lead to greater emphasis on USMLE Step 2 Clinical Knowledge scores or yield unintended consequences, including shifting weight to metrics such as medical school reputation.Yet, greater breadth in residency selection metrics will better serve both applicants and programs. Some students excel in coursework, others in research or leadership. All factors should be recognized, and broader metrics should be implemented to promote and recognize diversity of excellence. Given the need for metrics for residency selection as well as for a more holistic approach to evaluating residency applicants, assessment during medical school should be revisited and made more meaningful. Another opportunity may involve use of situational judgment tests to predict professionalism and performance on other competencies. It will be important to evaluate the impact of the new Step 1 paradigm and related initiatives going forward. Residency application overload must also be addressed.

Proposed Changes to the 2021 Residency Application Process in the Wake of COVID-19

The COVID-19 pandemic has brought unique challenges to the delivery of undergraduate medical education, particularly for current third-year medical students who are preparing to apply to residency. In mid-March, medical schools suspended all clinical rotations for the remainder of the 2019-2020 academic year. As such, third-year medical students may not be able to complete sufficient clinical experiences to make important career choices before they have to submit their residency applications. While the decision to suspend clinical rotations was necessary to protect students, specialty organizations and residency programs must mitigate the deficits in students' clinical education caused by the COVID-19 pandemic.In this Perspective, the authors identify potential challenges for third-year medical students and advocate for solutions to improve the residency application process for students and programs. First, they propose delaying the date that programs can access applicant data through the Electronic Residency Application Service, thereby affording students more time to complete clinical experiences, solidify their specialty decision, and strengthen their residency application. Second, the authors recommend a restriction on the number of visiting rotations that students are expected to complete to allow for a more equitable distribution of these important experiences. Third, they suggest that program directors from each specialty agree on a maximum number of applications per applicant (based on historical data) to curb an upsurge in applications that may stem from the unique circumstances created by COVID-19 without causing applicants undue stress. Lastly, the authors advocate that residency programs develop infrastructure to conduct video-based interviews and engage students through virtual networking events.Amidst the unique environment created by COVID-19, the authors urge governing bodies, specialty organizations, and residency programs to consider these recommendations to improve the efficiency and reduce the stress surrounding the 2021 Match.

Beyond High-Stakes Testing: Learner Trust, Educational Commodification, and the Loss of Medical School Professionalism

With ever-growing emphasis on high-stakes testing in medical education, such as the Medical College Admission Test and the United States Medical Licensing Examination Step 1, there has been a recent surge of concerns on the rise of a "Step 1 climate" within U.S. medical schools. The authors propose an alternative source of the "climate problem" in current institutions of medical education. Drawing on the intertwined concepts of trust and professionalism as organizational constructs, the authors propose that the core problem is not hijacking-by-exam but rather a hijackable learning environment weakened by a pernicious and under-recognized tide of commodification within the U.S. medical education system. The authors discuss several factors contributing to this weakening of medicine's control over its learning environments, including erosion of trust in medical school curricula as adequate preparation for entry into the profession, increasing reliance on external profit-driven sources of medical education, and the emergence of an internal medical education marketplace. They call attention to breaches in the core tenets of a profession-namely a logic that differentiates its work from market and managerial forces, along with related slippages in discretionary decision making. The authors suggest reducing reliance on external performance metrics (high-stakes exams and corporate rankings), identifying and investing in alternative metrics that matter, abandoning the marketization of medical education "products," and attending to the language of educational praxis and its potential corruption by market and managerial lexicons. These steps might salvage some self-governing independence implied in the term "profession" and make possible (if not probable) a recovery of a public trust becoming of the term and its training institutions.

The impact of United States Medical Licensing Exam (USMLE) step 1 cutoff scores on recruitment of underrepresented minorities in medicine: A retrospective cross-sectional study

BACKGROUND AND AIMS

United States Medical Licensing Exam (USMLE) scores are the single, most objective criteria for admission into residency programs in the country. Underrepresented minorities in medicine (URiM) are found to have lower USMLE scores compared to their White counterparts. The objective of this study is to examine how USMLE step 1 cutoff scores may exclude self-reported URiM from the residency interview process across various specialties.

METHODS

This was a retrospective cross-sectional study of 10 541 applicants to different residency programs at Zucker School of Medicine at Hofstra/Northwell Health between May 2014 and May 2015. We identified Blacks and Hispanics as URiM. The primary outcome is the percentage of applicants with USMLE step 1 score above different ranges of cutoff score, from 205 to 235 in five-point increments, by race/ethnicity and by URiM status. Secondary outcome is percentages of URiM vs non-URiM above and below mean USMLE step 1 scores by different specialties (internal medicine, obstetrics/gynecology, pediatrics, and psychiatry).

RESULTS

The study sample included 2707 White, 722 Black, 805 Hispanic, 5006 Asian, and 562 Other Race/Ethnicity applicants. Overall, 50.2% were male, 21.3% URiM, 7.4% had limited English proficiency, 67.6% attended international medical schools, and 2.4% are Alpha Omega Alpha Honor Medical Society (AOA) members. The mean (±SD) USMLE step 1 score was significantly greater among non-URiM applicants as compared to URiM applicants (223.7 ± 19.4 vs 216.1 ± 18.4, P < .01, two-sample t-test). Non-URiM applicants were younger, and the percentage of male and AOA applicants was greater among non-URiM applicants as compared to URiM applicants (50.5% vs 47.7%, P = .02, Chi-Square test; 2.9% vs 1.2%, P < .01, Chi-Square test, respectively).

CONCLUSION

Using a USMLE step 1 cutoff score as an initial filter for applicant recruitment and selection could jeopardize the benefits of a diverse residency program. Practical implications are discussed.

The Relationship Between US Medical Licensing Examination Step Scores and ABR Core Examination Outcome and Performance: A Multi-institutional Study

PURPOSE

We analyzed multi-institutional data to understand the relationship of US Medical Licensing Examination (USMLE) Step scores to ABR Core examination performance to identify Step score tiers that stratify radiology residents into different Core performance groups.

METHODS

We collected USMLE Step scores and ABR Core examination outcomes and scores for anonymized residents from 13 different diagnostic radiology residency programs taking the ABR Core examination between 2013 and 2019. USMLE scores were grouped into noniles using z scores and then aggregated into three tiers based on similar Core examination pass-or-fail outcomes. Core performance was grouped using standard deviation from the mean and then measured by the percent of residents with scores below the mean. Differences between Step tiers for Core outcome and Core performance were statistically evaluated (P < .05 considered significant).

RESULTS

Differences in Step 1 terciles Core failure rates (45.9%, 11.9%, and 3.0%, from lowest to highest Step tiers; n = 416) and below-mean Core performance (83.8%, 54.1%, and 21.1%, respectively; n = 402) were significant. Differences in Step 2 groups Core failure rates (30.0%, 10.6%, and 2.0%, from lowest to highest Step tiers; n = 387) and below-mean Core performance (80.0%, 43.7%, and 14.0%, respectively; n = 380) were significant. Step 2 results modified Core outcome and performance predictions for residents in Step 1 terciles of varying statistical significance.

CONCLUSIONS

Tiered scoring of USMLE Step results has value in predicting radiology resident performance on the ABR Core examination; effective stratification of radiology resident applicants can be done without reporting numerical Step scores.