Grading Changes for USMLE Step 1 - A Golden Opportunity to Recalibrate Medical Education Priorities

Otolaryngology Program Director, House-Staff, and Student Opinions: Step 1 Pass/Fail Score Reporting

OBJECTIVE(S)

To compare otolaryngology program director, house-staff, and medical student perspectives on the score reporting change of USMLE Step 1 to pass/fail.

METHODS

Separate electronic surveys were sent to program directors of ACGME-accredited otolaryngology programs (Cronbach's alpha = .87), otolaryngology house-staff (Cronbach's alpha = .91), and medical students interested in otolaryngology (Cronbach's alpha = .76).

RESULTS

Among the 51 otolaryngology program directors that completed the survey (response rate of 46.8%), 17.6% favored reporting USMLE Step 1 as pass/fail. A majority believed the reporting change would make it more difficult to screen (74.5%) and objectively compare applicants (82.4%). Step 2 CK scores will be more important to most program directors due to the reporting change (83.7%). Of the 93 house-staff that completed surveys, most did not favor the reporting change (61.3%). Over half (54.0%) of the 87 medical students that completed surveys did not support the scoring change, and most (65.5%) did not feel that it would decrease anxiety around residency applications (65.5%). Most house-staff and medical students felt that the scoring change would put non-U.S. MD students at a disadvantage (65.6% of house-staff, 69.8% of medical students).

CONCLUSION

Most survey respondents do not agree with the decision to report Step 1 as pass/fail. Despite its intended goals, most do not believe pass/fail Step 1 reporting will improve medical student well-being and believe it will put certain student populations at a greater disadvantage.

Assessing open-book examination in medical education: The time is now

As a result of the coronavirus pandemic, the feasibility of holding secure closed-book examinations in medical education is compromised. In this Personal View, we compare the underlying reasoning for using open-book and closed-book exams. We rethink the role of open-book assessment and offer ways in which we believe they can complement closed-book exams. We highlight the gap in research, highlight future directions, and call on medical educators to seize our current golden opportunity to explore the impact of open-book exams - on their own or combined to closed-book tests, as a blended approach - on learners, educators, and licensing bodies.

Race and Gender Differences in Medical Student Perspectives on Social Determinants of Health Education: A Single-Institution Survey Study

PURPOSE

The field of medicine is becoming increasingly aware of the role that social determinants of health (SDH) play in shaping health and health outcomes. Organized medicine - including prominent physician groups and accreditation bodies - has endorsed SDH education as an integral component of medical school curricula. This study sought to describe medical student perspectives on the current state of SDH in preclinical curricula.

METHODS

The authors developed a 9-item survey to assess time being spent on SDH and attitudes toward the current level of SDH content in preclinical curricula. All medical students at both campuses of a large public medical school were invited to participate between December 2019 and February 2020.

RESULTS

Of 1010 medical students invited to participate, 515 (51.0%) responded. Of the 515 respondents, 480 (93.2%) reported spending at least 40 hours per week on medical school, and of those, 405 (84.4%) said they spend 0-2 hours on SDH. The majority of all respondents (62.1%; 320/515) felt the current level of focus on SDH is "not enough", while only eleven students (2.1%; 11/515) felt it is "too much". In a multiple logistic model, Black students were over four times as likely as white students (aOR 4.19; 95% CI 1.37-18.38) to feel the current level of focus on SDH is "not enough". Likewise, women were 2.3-times (aOR 2.30; 95% CI 1.52-3.49) as likely as men to feel the level of focus on SDH is "not enough".

CONCLUSION

In practice, medical students are spending considerably less time learning SDH than is advised by consensus of expert educators and administrators. Over sixty percent of medical students do not feel the current level of focus on SDH is sufficient. Further study is needed to determine why women and racial minority students are significantly more likely to feel this way.

Medical Student Attitudes toward USMLE Step 1 and Health Systems Science - A Multi-Institutional Survey

Phenomenon: Because of its importance in residency selection, the United States Medical Licensing Examination Step 1 occupies a critical position in medical education, stimulating national debate about appropriate score use, equitable selection criteria, and the goals of undergraduate medical education. Yet, student perspectives on these issues and their implications for engagement with health systems science-related curricular content are relatively underexplored. Approach: We conducted an online survey of medical students at 19 American allopathic medical schools from March-July, 2019. Survey items were designed to elicit student opinions on the Step 1 examination and the impact of the examination on their engagement with new, non-test curricular content related to health systems science. Findings: A total of 2856 students participated in the survey, representing 23.5% of those invited. While 87% of students agreed that doing well on the Step 1 exam was their top priority, 56% disagreed that studying for Step 1 had a positive impact on engagement in the medical school curriculum. Eighty-two percent of students disagreed that Step 1 scores should be the top item residency programs use to offer interviews. When asked whether Step 1 results should be reported pass/fail with no numeric score, 55% of students agreed, while 33% disagreed. The majority of medical students agreed that health systems science topics were important but disagreed that studying for Step 1 helped learn this content. Students reported being more motivated to study a topic if it was on the exam, part of a course grade, prioritized by residency program directors, or if it would make them a better physician in the future. Insights: These results confirm the primacy of the United States Medical Licensing Examination Step 1 exam in preclinical medical education and demonstrate the need to balance the objectives of medical licensure and residency selection with the goals of the broader medical profession. The survey responses suggest several potential solutions to increase student engagement in health systems science curricula which may be especially important after Step 1 examination results are reported as pass/fail.

The Effects of Pass/Fail USMLE Step 1 Scoring on the Otolaryngology Residency Application Process

OBJECTIVES

To investigate how the decision to report United States Medical Licensing Examination (USMLE) Step 1 score as pass/fail will influence future otolaryngology residency application and match processes.

STUDY DESIGN

Survey study.

METHODS

An anonymous and voluntary survey approved by the Otolaryngology Program Directors Organization was administered to academic faculty members from April 24, 2020 through May 19, 2020.

RESULTS

Two hundred fifty-seven surveys were received from department chairs (17.5%), program directors (24.1%), associate program directors (12.5%), and department faculty (45.9%). USMLE Step 1 has been the most heavily weighted metric for offering interviews (44.0%), and it has correlated with residents' medical knowledge (77.0%) and in-service performance (79.8%) but not with surgical skills (57.6%) or patient care (47.1%). In total, 68.1% disagreed with the decision to make USMLE Step 1 pass/fail. This change is anticipated to lead to an increase in significance of USMLE Step 2 CK (89.1%), core clerkship grades (80.9%), elective rotation at the respective institutions (65.7%), Alpha Omega Alpha and other awards (64.6%), and letters of recommendation (63.8%). The new scoring is also anticipated to especially benefit students from top-ranked schools (70.8%), increase medical students' anxiety/uncertainty regarding obtaining interview invites (59.1%), and negatively affect international (51.4%), doctor of osteopathic medicine (45.9%), and underrepresented students (36.9%). Indication that USMLE Step 2 CK will significantly increase in weight varied according to department position (P = .049), geographic region (P = .047), years of practice (P < .001), and residency program size (P = .002).

CONCLUSION

Most academic otolaryngologists disagreed with changing USMLE Step 1 scoring to pass/fail and believe that it will increase other objective/subjective metrics' weight and put certain student populations at a disadvantage.

LEVEL OF EVIDENCE

N/A. Laryngoscope, 131:E738-E743, 2021.