United States Medical Licensing Examination Step 1 and 2 Scores Predict Neuroradiology Fellowship Success

Radiology resident selection factors predict resident performance

PURPOSE

To determine selection factors that predict radiology resident performance.

METHODS

59 consecutive radiology residents from 2002 to 2015 were ranked on performance during residency. Correlations and multiple regression analyses were performed to predict resident performance from the following selection factors: United States Medical Licensing Exam (USMLE) Step 1 score, medical school rank, Alpha Omega Alpha (AOA) membership, honors in clinical rotations, Medical Student Performance Evaluation (MSPE), and interview score. Results were compared against predictions from Match rank position.

RESULTS

Five selection factors showed significant or marginally significant correlations with resident performance (r = 0.2 to 0.3). The interview score was not significantly correlated. A multiple regression model comprised of the USMLE Step 1 score, medical school rank, AOA membership, and interview score predicted resident performance, with an adjusted R² of 0.19. The interview score was included in the model but did not achieve statistical significance. Match rank did not predict resident performance, with an R² of 0.01.

CONCLUSIONS

A multiple regression model comprised of the USMLE Step 1 score, medical school rank, and AOA membership predicted radiology resident performance and may assist with resident selection.

A Novel USMLE Step 1 Projection Model Using a Single Comprehensive Basic Science Self-Assessment Taken During a Brief Intense Study Period

BACKGROUND

Comprehensive Basic Science Self-Assessments (CBSSAs) offered by the National Board of Medical Examiners (NBME) are used by students to gauge preparedness for the United States Medical Licensing (USMLE) Step 1. Because residency programs value Step 1 scores, students expend many resources attempting to score highly on this exam. We sought to generate a predicted Step 1 score from a single CBSSA taken several days out from a planned exam date to inform student testing and study plans.

METHODS

2016 and 2017 Step 1 test takers at one US medical school were surveyed. The average daily score improvement from CBSSA to Step 1 during the 2016 study period was calculated and used to generate a predicted Step 1 score as well as mean absolute prediction errors (MAPEs). The predictive model was validated on 2017 data.

RESULTS

In total, 43 of 61 respondents totaling 141 CBSSAs in 2016 and 37 of 43 respondents totaling 122 CBSSAs in 2017 were included. The final prediction model was [Predicted Step 1 = 292 - (292 - CBSSA score) * 0.987527 ^ (number of days out)]. In 2016, the average difference between predicted and actual scores was -0.81 (10.2) and the MAPE was 7.8. In 2017, 88 (72.1%) and 118 (96.7%) of true Step 1 scores fell within one and two standard deviations of a student's predicted score. There was a MAPE of 7.7. Practice form used (p = 0.19, 0.07) and how far out from actual Step 1 it was taken (p = 0.82, 0.38) were not significant in either year of study.

CONCLUSION

This projection model is reasonable for students to use to gauge their readiness for Step 1 while it remains a scored exam and provides a framework for future predictive model generation as the landscape of standardized testing changes in medical education.

Educational and Personal Opportunity Costs of Medical Student Preparation for the United States Medical Licensing Examination Step 1 Exam: A Single-Center Study

Purpose

To assess the degree to which medical students choose to disengage from their regular preclinical curriculum and extracurricular activities in order to focus on United States Medical Licensing Examination (USMLE) Step 1 exam preparation, as well as learner-perceived effects of Step 1 preparation on their physical, social, and mental health.

Method

Online survey of medical students who have taken the USMLE Step 1 exam at a single large Midwestern academic medical center.

Results

The response rate was 54%. Students often reported absenteeism from a variety of preclinical curricular activities, including lectures (44%) and didactics focusing on medical ethics (37%), clinical skills (28%), and encounters with actual and standardized patients (9%) in order to study for USMLE Step 1. Many students also forewent extracurricular opportunities including research (53%), elective patient care opportunities (45%), community service (39%), and healthcare advocacy experiences (38%) in order to study for USMLE Step 1. Majorities of students identified Step 1 preparation as a cause of burnout (79%) or significant anxiety or depression (61%), for which nearly a third sought mental healthcare; students also reported Step 1 preparation as a cause of engaging in dangerous behaviors such as illicit prescription stimulant use as well as driving or providing patient care while impaired by fatigue. In narrative comments, students frequently described Step 1 to be a barrier to their development into effective clinicians, the traditional medical school curriculum to be a barrier to performance on Step 1, or both.

Conclusions

Medical students often prioritize Step 1 exam preparation over engaging with the standard preclinical curriculum, extracurricular opportunities, and activities to promote wellbeing. These findings have implications for the emphasis residency program directors place on single high-stakes standardized exams in the resident recruitment process.

Trends in Neuroradiology Fellowship Match in the United States: Analysis of the 13-Year National Resident Matching Program Data

The neuroradiology fellowship match is conducted by the National Resident Matching Program (NRMP) every year. The purpose of this study is to report the trends in neuroradiology match results from 2008 to 2020 in the United States (US) using data available from the NRMP. The fellowship match data was obtained from the NRMP archives. Data specific to programs (number of programs and filled positions) were obtained. Using the NRMP's "Charting Outcomes in the Match: Specialty Matching Service'' report, a detailed analysis of applicant characteristics broken down for each specialty, was also obtained for neuroradiology for the year 2018. Neuroradiology programs in the NRMP Match increased from 71 to 77 and the number of positions increased from 158 to 270 between 2008 and 2020. The fill rate of fellowship spots has remained steady at 75.9% in 2008 and 74.4% in 2020. The proportion of US allopathic medical graduates (US MD) amongst the filled spots decreased from 79.2% (2008) to 64.2% (2020), while the proportion of international medical graduates (IMGs) increased from 11.7% to 25.4%. US MD and IMG fill trends did not reach statistical significance (P = 0.78 and P = 0.92, respectively). US MDs had a higher number of research experiences and publications (23.2 and 26, respectively) when compared to US IMGs (5 and 7) and non-US IMGs (6.8 and 14.2). To conclude, over the last 13 years, the neuroradiology fellowship programs in the US have increased in number and expanded in size, with an almost stable fill rate of around 75%.

The Effects of Pass/Fail USMLE Step 1 Scoring on the Otolaryngology Residency Application Process

OBJECTIVES

To investigate how the decision to report United States Medical Licensing Examination (USMLE) Step 1 score as pass/fail will influence future otolaryngology residency application and match processes.

STUDY DESIGN

Survey study.

METHODS

An anonymous and voluntary survey approved by the Otolaryngology Program Directors Organization was administered to academic faculty members from April 24, 2020 through May 19, 2020.

RESULTS

Two hundred fifty-seven surveys were received from department chairs (17.5%), program directors (24.1%), associate program directors (12.5%), and department faculty (45.9%). USMLE Step 1 has been the most heavily weighted metric for offering interviews (44.0%), and it has correlated with residents' medical knowledge (77.0%) and in-service performance (79.8%) but not with surgical skills (57.6%) or patient care (47.1%). In total, 68.1% disagreed with the decision to make USMLE Step 1 pass/fail. This change is anticipated to lead to an increase in significance of USMLE Step 2 CK (89.1%), core clerkship grades (80.9%), elective rotation at the respective institutions (65.7%), Alpha Omega Alpha and other awards (64.6%), and letters of recommendation (63.8%). The new scoring is also anticipated to especially benefit students from top-ranked schools (70.8%), increase medical students' anxiety/uncertainty regarding obtaining interview invites (59.1%), and negatively affect international (51.4%), doctor of osteopathic medicine (45.9%), and underrepresented students (36.9%). Indication that USMLE Step 2 CK will significantly increase in weight varied according to department position (P = .049), geographic region (P = .047), years of practice (P < .001), and residency program size (P = .002).

CONCLUSION

Most academic otolaryngologists disagreed with changing USMLE Step 1 scoring to pass/fail and believe that it will increase other objective/subjective metrics' weight and put certain student populations at a disadvantage.

LEVEL OF EVIDENCE

N/A. Laryngoscope, 131:E738-E743, 2021.

Characteristics of internal medicine residents who successfully match into cardiology fellowships

BACKGROUND

The unique traits of residents who matriculate into subspecialty fellowships are poorly understood. We sought to identify characteristics of internal medicine (IM) residents who match into cardiovascular (CV) fellowships.

METHODS

We conducted a retrospective cohort study of 8 classes of IM residents who matriculated into residency from 2007 to 2014. The primary outcome was successful match to a CV fellowship within 1 year of completing IM residency. Independent variables included residents' licensing exam scores, research publications, medical school reputation, Alpha Omega Alpha (AOA) membership, declaration of intent to pursue CV in the residency application personal statement, clinical evaluation scores, mini-clinical evaluation exercise scores, in-training examination (ITE) performance, and exposure to CV during residency.

RESULTS

Of the 339 included residents (59% male; mean age 27) from 120 medical schools, 73 (22%) matched to CV fellowship. At the time of residency application, 104 (31%) had ≥1 publication, 38 (11%) declared intention to pursue CV in their residency application personal statement, and 104 (31%) were members of AOA. Prior to fellowship application, 111 (33%) completed a CV elective rotation. At the completion of residency training, 108 (32%) had ≥3 publications. In an adjusted logistic regression analysis, declaration of intention to pursue CV (OR 6.4, 99% CI 1.7-23.4; p < 0.001), completion of a CV elective (OR 7.3, 99% CI 2.8-19.0; p < 0.001), score on the CV portion of the PGY-2 ITE (OR 1.05, 99% CI 1.02-1.08; p < 0.001), and publication of ≥3 manuscripts (OR 4.7, 99% CI 1.1-20.5; p = 0.007) were positively associated with matching to a CV fellowship. Overall PGY-2 ITE score was negatively associated (OR 0.93, 99% CI 0.90-0.97; p < 0.001) with matching to a CV fellowship.

CONCLUSIONS

Residents' matriculation into CV fellowships was associated with declaration of CV career intent, completion of a CV elective rotation, CV medical knowledge, and research publications during residency. These findings may be useful when advising residents about pursuing careers in CV. They may also help residents understand factors associated with a successful match to a CV fellowship. The negative association between matching into CV fellowship and overall ITE score may indicate excessive subspecialty focus during IM residency.

Factors Predictive of Performance on USMLE Step 1: Do Commercial Study Aids Improve Scores?

BACKGROUND

The United States Medical Licensing Examination (USMLE) Step 1 was the most cited factor in granting medical student applicants' residency interviews in a 2016 NRMP survey. Medical students utilize numerous commercial resources to achieve the highest score possible. This study sought to examine preparatory resources and their association with USMLE Step 1 performance.

METHODS

In 2016 and 2017, 170 medical students from a public, Midwestern medical school completed a voluntary survey after completing USMLE Step 1. The survey asked about study aids, anticipated performance, test preparation time, and practice exam performance. Actual scores and academic performance (overall percent for years 1 and 2 of medical school) were matched to survey responses. Data was analyzed using descriptive and inferential statistical analyses.

RESULTS

USMLE Step 1 performance was associated with academic performance (r = 0.719, p < 0.01). Post hoc tests indicated a positive difference with the UWorld Qbank and exam performance (F _4,134 = 10.29, p < 0.05). Amount of days dedicated to test preparation was negatively correlated with performance in the 2017 student cohort (r = - 0.304, p < 0.01). The use of other commercially available products and the number of practice tests completed were not correlated with performance.

DISCUSSION

Cumulative pre-clinical grades had the strongest association with performance on the USMLE Step 1. Completing UWorld Qbank at least once was positively related with Step 1 scores. However, the use of other study aids, completing multiple practice exams, and delaying exam dates were not statistically significant. This study indicates performance in the pre-clinical curriculum and increased usage of UWorld are strongly associated with USMLE Step 1 performance.

USMLE Step 2 CK: Best Predictor of Multimodal Performance in an Internal Medicine Residency

BACKGROUND

Internal medicine (IM) residency programs receive information about applicants via academic transcripts, but studies demonstrate wide variability in satisfaction with and usefulness of this information. In addition, many studies compare application materials to only 1 or 2 assessment metrics, usually standardized test scores and work-based observational faculty assessments.

OBJECTIVE

We sought to determine which application materials best predict performance across a broad array of residency assessment outcomes generated by standardized testing and a yearlong IM residency ambulatory long block.

METHODS

In 2019, we analyzed available Electronic Residency Application Service data for 167 categorical IM residents, including advanced degree status, research experience, failures during medical school, undergraduate medical education award status, and United States Medical Licensing Examination (USMLE) scores. We compared these with post-match residency multimodal performance, including standardized test scores and faculty member, peer, allied health professional, and patient-level assessment measures.

RESULTS

In multivariate analyses, USMLE Step 2 Clinical Knowledge (CK) scores were most predictive of performance across all residency performance domains measured. Having an advanced degree was associated with higher patient-level assessments (eg, physician listens, physician explains, etc). USMLE Step 1 scores were associated with in-training examination scores only. None of the other measured application materials predicted performance.

CONCLUSIONS

USMLE Step 2 CK scores were the highest predictors of residency performance across a broad array of performance measurements generated by standardized testing and an IM residency ambulatory long block.

Factors Determining Scientific Paper Productivity by Neuroradiology Fellows

RATIONALE AND OBJECTIVES

We sought to determine (1) whether United States Medical Licensing Examination (USMLE) scores predict academic productivity in neuroradiology fellows as measured by publications and citations, and (2) what factors predict such productivity.

MATERIALS AND METHODS

We reviewed the USMLE scores, gender, medical school location attended, publication record before and during fellowship, fellowship evaluation ratings and subsequent practice site (private vs academic) of neuroradiology fellows from 2004 to 2014 to determine relationships with publications and citations after fellowship. Spearman's correlation and Poisson regression analyses were performed to assess the association between these factors and quantity of publications and citations per year after fellowship.

RESULTS

USMLE scores and fellowship evaluation scores correlated inversely with radiology publications and citations. There were strong correlations between publication records before or during fellowship and after fellowship. Fellows from international medical schools, with PhD degrees, and those fellows proceeding to academic practice had more publications before or during and after neuroradiology fellowship.

CONCLUSIONS

The best predictors of whether a graduating neuroradiology fellow will publish and have high citation rates is prior publication record, a PhD degree, and staying in academics. USMLE scores and evaluations during the fellowship were inversely correlated with publication measures of academic productivity.