Analysis of Current Orthopedic Surgery Residents and Their Prior Medical Education: Does Medical School Ranking Matter in Orthopedic Surgery Match?

Going Above and Beyond With SJTs: Impact of Applicant Characteristics on Open Response SJT Participation

BACKGROUND

Situational judgment tests (SJT) have gained popularity as a standardized assessment of nontechnical competencies for applicants to medical school and residency. SJT formats range from rating the effectiveness of potential response options to solely open response. We investigated differences in test-taking patterns between responders and nonresponders to optional open response SJT questions during the application process.

METHODS

This was a prospective multi-institutional study of general surgery applicants to seven residency programs. Applicants completed a 32-item SJT designed to measure ten core competencies: adaptability, attention to detail, communication, dependability, feedback receptivity, integrity, professionalism, resilience, self-directed learning, and team orientation. Each SJT item included an optional, nonscored, open response space for applicants to provide a behavioral response if they desired. Trends in applicant gender, race, ethnicity, medical school ranking, and USMLE scores were examined between the responder versus nonresponder group.

RESULTS

In total, 1491 general surgery applicants were invited to complete the surgery-specific SJT. Of these, 1454 (97.5%) candidates completed the assessment and 1177 (78.9%) provided additional responses to at least one of the 32 SJT scenario sets. There were no differences in overall SJT performance, USMLE scores (Step 1: 235, SD 14, Step 2: 250, SD 11), race and/or ethnicity between the responder and nonresponder groups. Responders were more likely to be from a top 25 medical school (p < 0.05) compared to the nonresponder group. Among applicants who completed any open response questions, women completed a significantly higher number of questions compared to men (7.21 vs 6.07, p = 0.003). The number of open responses provided correlated with higher scores on SJT items measuring dependability (r = 0.07, p = 0.007).

CONCLUSIONS

SJT design and format has the potential to impact test-taker response patterns. SJT developers and adopters should ensure test format and design have no unintended consequences prior to implementation.

"Top 20" residency-trained academic orthopaedic surgeons have higher academic productivity and funding in their careers

"Top 20" status on Doximity, an online networking service for medical professionals, is an indicator of the reputation of a residency program. The study assesses how training at a Top 20 (T20) orthopaedic residency program impacts career productivity and funding. Fellowship and Residency Electronic Interactive Database was used in 2022 to identify active orthopaedic residency programs. Demographic and training data was collected for each orthopaedic surgeon using institutional websites and Doximity. The Residency Navigator feature on Doximity was used to rank residency programs by "reputation." Programs were categorized as either T20 or non-T20. The relative citation ratio (RCR) was calculated using the NIH iCite tool and Hirsch index (h-index) was calculated using Scopus. Industry funding was collected from the Centers for Medicare & Medicaid Services Open Payments Program (CMS) for all available years (2014-2020). A total of 2812 academic orthopaedic surgeons were included in the study. Among academic orthopaedic surgeons in the United States, T20 trained orthopedists had more publications and citations (p < 0.001), along with higher h-indices (p < 0.001), RCR (p < 0.001), and industry funding (p = 0.043). Additionally, T20 trained orthopedists were 1.375 times more likely to obtain professor status (95% confidence interval: 1.150-1.645, p < 0.001). Even after propensity-matched analysis, T20 trained orthopedists maintained these differences. Training at a T20 residency program is associated with promotion, productivity, and funding. These findings are especially of concern to medical students who must consider the importance of a residency program's reputation when deciding where to apply for residency.

Contemporary Trends in the Orthopaedic Surgery Residency Match and the Effects of COVID-19

Objective

We aimed to elucidate associations between geographic location, size, and ranking of medical schools that orthopaedic surgery residents graduate from and the residencies that they match both pre-COVID-19 and post-COVID-19 pandemic by examining the 2017 to 2022 orthopaedic surgery residency cohorts.

Methods

Demographics were extracted using Doximity Residency Navigator platform, the 2021 US News and World Report, and program websites. Medical schools were classified as large if they had >613 medical students. Postgraduate year 1 (PGY-1) (2021 match) and PGY-2 (2022 match) residents were classified as the COVID-19 cohort. Location was categorized as Northeast, Midwest, South, and West. Chi-square tests, Cohen's H value, and descriptive statistics were used for analysis with statistical significance set at p <0.05.

Results

Four thousand two hundred forty-three residents from 160 accredited US orthopaedic residency programs (78.4%) were included. Northeastern applicants were most likely to match in the same region (p <0.01), and southern applicants were most likely to match at their home program (p <0.001). Applicants affected by the COVID-19 pandemic did not differ from their predecessors with regards to matching to the same region (p = 0.637) or home program (p = 0.489). Applicants from public medical schools were more likely to match in the same region and at their home program (p <0.001), whereas those from private medical schools were more likely to match at top-ranked residencies (p <0.001). Students from both top 25- and top 50-ranked medical schools were more likely to match at their home program (p <0.01) and attend top 20-ranked residency programs (p <0.0001).

Conclusion

These results demonstrate significant associations between matched residencies and attended medical schools' geographic location, school type, and ranking. During the pandemic, geographic trends were overall unchanged, whereas residents from large or lower-ranked schools were more likely to match at home programs, and those from private or top-ranked schools were less likely to attend top residencies.

Does Medical School Rank Impact Matching into Otolaryngology?

OBJECTIVE(S)

To describe how medical school rank may be associated with matriculation into otolaryngology residency programs.

METHODS

A list of medical students who matched into otolaryngology residencies in 2020, 2021, and 2022 was obtained from Otomatch (Otomatch.com). For each student, their medical school, U.S. News & World Report Best Medical School (Research) ranking, and region based on the U.S. Census divisions were recorded. The medical schools were divided into four Tiers according to rank: 1-40 (Tier 1), 41-80 (Tier 2), 81-124 (Tier 3), and 125-191 (Tier 4). Residency programs were also grouped by region, whether they were large (>3 residents/year) or small (<3 residents/year), and their Doximity reputation ranking: 1-31 (Tier 1), 32-61 (Tier 2), 62-91 (Tier 3), and 92-125 (Tier 4).

RESULTS

Nine hundred and ninety-five medical students were included in this study. The majority of residency matriculants were MDs (N = 988, 99.3%) who came from Tier 1 (N = 410, 41.2%) or Tier 2 (N = 313, 31.5%) medical schools. Those who attended higher-tier medical schools were more likely to match into higher-tier residency programs (p < 0.001). 57.8% (N = 237) of the applicants who attended Tier 1 medical schools matriculated into a Tier 1 residency program, whereas only 24.7% (N = 42) of the applicants from Tier 4 medical schools matriculated into a Tier 1 residency program.

CONCLUSION

Applicants who attend top-tier medical schools represent significantly more residents at top-tier otolaryngology residency programs than those from lower tier medial schools.

LEVEL OF EVIDENCE

NA Laryngoscope, 133:3353-3357, 2023.

Maximizing Standardization While Ensuring Equity: Exploring the Role of Applicant Experiences, Attributes, and Metrics on Performance of a Surgery-Specific Situational Judgment Test

BACKGROUND

Situational judgment tests (SJT) are hypothetical but realistic scenario-based assessments that allow residency programs to measure judgment and decision-making among future trainees. A surgery-specific SJT was created to identify highly valued competencies among residency applicants. We aim to demonstrate a stepwise process for validation of this assessment for applicant screening through exploration of two often-overlooked sources of validity evidence - relations with other variables and consequences.

METHODS

This was a prospective multi-institutional study involving 7 general surgery residency programs. All applicants completed the SurgSJT, a 32-item test aimed to measure 10 core competencies: adaptability, attention to detail, communication, dependability, feedback receptivity, integrity, professionalism, resilience, self-directed learning, and team orientation. Performance on the SJT was compared to application data, including race, ethnicity, gender, medical school, and USMLE scores. Medical school rankings were determined based on the 2022 U.S. News & World Report rankings.

RESULTS

In total, 1491 applicants across seven residency programs were invited to complete the SJT. Of these, 1454 (97.5%) candidates completed the assessment. Applicants were predominantly White (57.5%), Asian (21.6%), Hispanic (9.7%), Black (7.3%), and 52% female. A total of 208 medical schools were represented, majority were allopathic (87.1%) and located in United States (98.7%). Less than a quarter of applicants (22.8%; N=337) were from a top 25 school based on U.S. News & World Report rankings for primary care, surgery, or research. Average USMLE Step 1 score was 235 (SD 37) and Step 2 score was 250 (SD 29). Sex, race, ethnicity, and medical school ranking did not significantly impact performance on the SJT. There was no relationship between SJT score and USMLE scores and medical school rankings.

CONCLUSIONS

We demonstrate the process of validity testing and importance of two specific sources of evidence-consequences and relations with other variables, in implementing future educational assessments.

The Mentee Perspective: Evaluating Mentorship of Medical Students in the Field of Orthopaedic Surgery

INTRODUCTION

Mentorship is an invaluable facet of medical education. The purpose of this study was to analyze medical student perspectives of mentorship they received and the influence this has on their participation in the field of orthopaedic surgery.

METHODS

We conducted a cross-sectional study of medical students interested in pursuing orthopaedic surgery through an 18-question survey distributed through social media and e-mail.

RESULTS

Two hundred fifteen students completed this survey, with over 50% of students reporting that they have a mentor in orthopaedic surgery while 34% were actively seeking one. Most students found mentors through research opportunities (25%) and cold e-mails (20%). Common hurdles to mentorship were access (38%) and finding common time (30%). Peer mentorship had a higher mean satisfaction score in all domains, except facilitating matching, and there was a significant difference between groups (e.g., peer mentor versus program director; P < 0.001). Sex, race, and degree type were not significantly related to students' access to or their evaluation of mentors (P > 0.05 for all).

CONCLUSION

Overall, this study demonstrates that medical students across the nation rely on mentorship to guide them on their path to becoming an orthopaedic surgeon.

Academic Faculty Demonstrate Weak Agreement in Evaluating Orthopaedic Surgery Residents

Background

As objective metrics fade, subjective elements of orthopaedic surgery applicants carry increasing importance during recruitment. Academic orthopaedic surgeons believe they can select for high-performing orthopaedic residents. However, can they agree? The purpose of this study was (1) to analyze an academic orthopaedic surgery department to determine whether they agree on which residents performed best during residency and; (2) to correlate preresidency and intraresidency factors with postresidency evaluations of resident performance.

Methods

With Institutional Review Board [IRB] approval, an orthopaedic surgery department completed surveys to evaluate residency performance for 10 years of graduated residents (2012-2022). Faculty determined (1) Post-Residency Class Rank (PRCR)-ranked from the highest (1) to lowest performing resident (5) relative to their class based on faculty perspective of performance-and (2) Rank List Score (RLS)-ranked based off the 5-point AOA SLOR used during recruitment interviews. RLS assessed how likely the department would have graduates in the program again based on residency performance. Free marginal Cohen's kappa statistics assessed faculty inter-rater agreement. Preresidency metrics (United States Medical Licensing Exam [USMLE] 1 and 2 scores, research publications, etc) were correlated with Orthopaedic In-Training Exam (OITE) scores, research productivity, American Board of Orthopaedic Surgery (ABOS) scores, and faculty-derived rankings. Linear regressions with forward variable entry (p < 0.05) were used to determine factors associated with excellent resident performance.

Results

Eighteen of 25 faculty members (72%) provided survey responses evaluating 46 residents. Faculty agreed 37% and 38% of the time for PRCR (kappa 0.26) and RLS (kappa 0.23), respectively. Step 2 score was the only preresidency factor significantly associated with PRCR (p = 0.03, r2 = 0.15) and RLS (p = 0.02, r2 = 0.3). The only intraresidency factor significantly correlated with PRCR (p = 0.002, r2 = 0.50) and RLS (p = 0.01, r2 = 0.39) was PGY-4 OITE score.

Conclusions

An academic orthopaedic surgery department is able to come to a consensus on evaluations of residency performance relative to peers in the same year of training (PRCR) and an objective standard (RLS). Step 2 and Post-Graduate Year (PGY)-4 OITE scores were the only preresidency and intraresidency factors with significant association to higher postresidency, faculty-derived performance scores.

Level of Evidence

III.

Predictive value of medical school ranking in the academic scholarship of ophthalmology residents

Objective

To determine whether a resident's medical school ranking predicts their scholarship during residency.

Design

The authors stratified ophthalmology residents in ACGME accredited programs into tiers based off their medical school background's US News & World Report ranking: T1 (schools 1-20), T2 (21-50), and T3 (51-90). Investigators queried PubMed and Scopus for number of total publications, first/second author publications, publications in the top 10 impact factor journals in ophthalmology, and publications with the senior author affiliated with the resident's residency program/medical school. Authors collected data from start of ophthalmology residency to December 5th, 2021, and performed Pearson chi squared, ANOVA, Eta squared, Tukey, and multivariable logistic regression tests.

Results

1054 residents were included for analysis, with 370 from T1 schools, 296 from T2 schools, and 388 from T3 schools. T3 residents had a significantly decreased likelihood of publishing at least one (OR = 0.659;95%CI = 0.481,0.905;p = .010), two (OR = 0.643;95%CI = 0.436,0.949;p = .026), or five (OR = 0.407;95%CI = 0.187,0.886;p = .024) total publications compared to T1 residents. T3 residents also were partially predicted to publish fewer first author works, high impact journal articles, and articles with senior authors affiliated with their medical school. T2 residents were more likely to publish at least one second author work than T1 residents (OR = 1.604;95%CI = 1.101,2.337;p = .014). There was no significant difference between tiers in publications with senior authors affiliated with the same residency program.

Conclusions

The authors observed little difference in scholarship between residents from T1 and T2 schools, but some differences may exist between T3 and T1/T2 residents. Merit of rankings should be further explored.

Diversity in Orthopaedic Surgery Residencies Based on Allopathic Medical School Affiliation

Orthopaedic surgery is one of the most competitive and least diverse specialties in medicine. Affiliation of an orthopaedics with an allopathic medical school impacts research opportunities and early exposure to clinical orthopaedics. The purpose of this study is to examine the potential effect allopathic medical school affiliation has on orthopaedic surgery resident demographics and academic characteristics.

Methods

All 202 Accreditation Council for Graduate Medical Education (ACGME)-accredited orthopaedics programs were divided into 2 groups: Group 1 consisted of residency programs without an affiliated allopathic medical school, and Group 2 consisted of programs with an affiliated allopathic medical school. Affiliations were determined by cross-referencing the ACGME residency program list with the medical school list published by Association of American Medical Colleges (AAMC). Program and resident characteristics were then compiled using AAMC's Residency Explorer including region, program setting, number of residents, and osteopathic recognition. Resident characteristics included race, gender, experiences (work, volunteer, and research), peer-reviewed publications, and US Medical Licensing Examination Step 1 scores.

Results

Of the 202 ACGME-accredited orthopaedics residencies, Group 1 had 61 (30.2%) programs, and Group 2 had 141 (69.8%) programs. Group 2 had larger programs (4.9 vs. 3.2 resident positions/year; p < 0.001) and 1.7 times the number of residency applicants (655.8 vs. 385.5; p < 0.001). Most Group 2 residents were allopathic medical school graduates, 95.5%, compared with 41.6% in Group 1. Group 1 had 57.0% osteopathic medical school graduates, compared with 2.9% in Group 2. There were 6.1% more White residents in Group 1 residencies (p = 0.025), and Group 2 residencies consisted of 3.5% more Black residents in relation to Group 1 (p = 0.03). Academic performance metrics were comparable between the 2 groups (p > 0.05).

Conclusion

This study demonstrated that candidates who successfully match into an orthopaedic surgery residency program achieve high academic performance, regardless of whether the program was affiliated with an allopathic medical school. Differences may be influenced by increased representation of minority faculty, greater demand for allopathic residents, or stronger emphasis on promotion of diversity in those residency programs.

Availability of Data and Material

Available on reasonable request.

Level of Evidence

Level III. See Instructions for Authors for a complete description of levels of evidence.

Analysis of Current Neurological Surgery Residents and Prior Medical Education: Do Medical School Attributes Matter?

BACKGROUND

With the recent changes to the U.S. Medical Licensing Examination grading system, an understanding of the factors that influence the neurological surgery residency match process is crucial for residency directors. The aim of the present retrospective study was to explore the associations of medical school location, ranking, private school status, size, and presence of an American Association of Neurological Surgeons (AANS) chapter or neurological surgery interest group (NSIG) with the neurosurgery match outcomes.

METHODS

An enrollment list of all accredited U.S. neurosurgery residency programs was compiled on June 28, 2021. For the included residents, the residency program, degree, and previously attended medical school were retrieved. The geographic location, ranking, private school status, and size were collected for the residency programs and medical schools attended by the residents at each program.

RESULTS

A total of 1437 residents from 101 neurosurgery residency programs (89%) were included. Graduates from the top 25 medical schools were more likely to match into their home residency programs (P < 0.001) and highly ranked residency programs (P < 0.001). Students from larger medical schools were also more likely to match into larger (P < 0.001) and highly ranked (P < 0.001) programs than were applicants from smaller schools. Students from medical schools with an AANS chapter or NSIG were also more likely to match into top ranked programs (P < 0.001 for both).

CONCLUSIONS

Medical students from the top 25 medical schools, private medical schools, medical schools with an AANS chapter, and medical schools with an NSIG were more likely to match into a prestigious residency program. These findings suggest that underlying biases might be present for program directors to consider in the resident selection process.

Trends in Medical Training and Leadership at Academic Orthopedic Programs

Introduction: When deciding on which programs to rank or fellowships to enter, medical students and residents may assess the program's prestige and specialty training opportunities. This report aimed to analyze the demographics of orthopedic department chairs and program directors (PDs), focusing on the prestige of their orthopedic training and medical school. Secondary data included fellowship, higher-level education, sex, professorship, years of practice, and total published research.

Methods: We used U.S. News and Doximity to rank 192 medical schools and 200 orthopedic residency programs based on prestige rankings, respectively. We searched for the department chair, vice-chair, and PD via program websites, Council of Orthopaedic Residency Directors (CORD), Orthopedic Residency Information Network (ORIN), personal websites, LinkedIn, and Doximity. Subsequently, we searched for each individual’s demographic information, education and research history, employment history, and medical school attended.

Results: We gathered data on 268 orthopedic surgeons with leadership positions at academic hospitals. Of the 268, 115 were department chairs, 15 were vice-chairs, 126 were PDs, 11 were both the chair and PD, and one was vice-chair and PD. Of the 268 physicians, 244 physicians were male (91.0%), while 22 were female (9.0%). The average residency reputation ranking overall was 59.7 ± 5.7. More specifically, for chairs, the average was 57.0 ± 8.3 (p < 0.005), and for PDs, the average was 63.6 ± 8.0 (p <0.005). There was no significant difference between chairs and PDs (p = 0.26).

Conclusion: Orthopedic leaders were found to have trained at more prestigious programs. This trend could be explained by increased research opportunities at more prestigious programs or programs attempting to increase their own reputation. 9.0% of the leaders identify as female, which is comparable to the 6.5% of practicing female orthopedic surgeons. However, this further demonstrates a need for gender equity in orthopedic surgery. Assessing trends in the training of orthopedic surgeons with leadership positions will allow a better understanding of what programs look for in the hiring process.