Factors Predictive of Orthopaedic In-training Examination Performance and Research Productivity Among Orthopaedic Residents

Factors associated with infectious diseases fellowship academic success

BACKGROUND

A multitude of factors are considered in an infectious disease (ID) training program's meticulous selection process of ID fellows but their correlation to pre and in-fellowship academic success as well as post-fellowship academic success and short-term outcomes is poorly understood. Our goal was to investigate factors associated with subsequent academic success in fellowship as well as post-fellowship short-term outcomes.

METHODS

In 2022, we retrospectively analyzed deidentified academic records from 39 graduates of the Mayo Clinic Rochester ID Fellowship Program (1 July 2013- 30 June 2022). Data abstracted included demographics, degrees, honor society membership, visa/citizenship status, medical school, residency training program, United States Medical Licensure Exam (USMLE) scores, letters of recommendation, in-training examination (ITE) scores, fellowship track, academic rank, career choice, number of honors, awards, and abstracts/publications prior to fellowship, during training, and within 2 years of graduation.

RESULTS

Younger fellows had higher USMLE step 1 scores, pre and in-fellowship scholarly productivity, and higher ITE performance. Female fellows had significantly higher USMLE step 3 scores. Prior research experience translated to greater in-fellowship scholarly productivity. Higher USMLE scores were associated with higher ID ITE performance during multiple years of fellowship, but USMLE step 2 clinical knowledge and 3 scores were associated with higher pre and in-fellowship scholarly productivity and receiving an award during fellowship. The USMLE step 1 score did not correlate with fellowship performance beyond year 1 and 2 ITE scores.

CONCLUSIONS

Multiple aspects of a prospective fellow's application must be considered as part of a holistic review process for fellowship selection. USMLE step 2 CK and 3 scores may predict fellowship performance across multiple domains.

Association between physicians' maldistribution and core clinical competency of resident physicians: a nationwide cross-sectional study

OBJECTIVES

With physician maldistribution recognised as a global issue, Japan implemented the physician uneven distribution (PUD) index as a strategic measure. Currently, there is a lack of objective assessment of core clinical competencies in regions influenced by varying levels of physician distribution. In this study, we objectively assess the core clinical competencies in regions affected by physician maldistribution and explore the relationship between the PUD index and the clinical competencies of resident physicians.

DESIGN, SETTING AND PARTICIPANTS

In this cross-sectional study, we gathered data from the January 2023 General Medicine In-Training Examination (GM-ITE) survey. Participants included postgraduate year 1 and 2 resident physicians in Japanese hospitals mandating the GM-ITE or those who voluntarily took it. The GM-ITE scores of the resident physicians were assessed. The PUD index, a Japanese policy indicator, reflects regional physician disparities. A low PUD index signals a medical supply shortage compared with local demand. The trial registration number is 23-7.

RESULTS

The high and low PUD index groups included 2143 and 1580 participants, respectively. After adjusting for relevant confounders, multivariate linear regression analyses revealed that the low PUD index group had significantly higher GM-ITE scores than the high PUD index group (adjusted coefficient: 1.14; 95% CI 0.62 to 1.65; p<0.001).

CONCLUSIONS

The study revealed no clinically differences in GM-ITE scores between residents in regions with disparate physician distributions, suggesting that factors beyond PUD may influence clinical competency. This finding prompts a re-evaluation of whether current assessment methodologies or educational frameworks fully support learning across varied community settings.

Acquisition of Medical Knowledge Varies During Allopathic and Osteopathic Orthopedic Surgery Residency Training

Currently, little is known about the differences in medical knowledge acquisition between osteopathic and allopathic orthopedic surgery residents. The purpose of this study was to determine the relationship between Orthopaedic In-Training Examination (OITE) performance and training pathway in orthopedic surgery. This was a retrospective cohort study of all orthopedic surgery residents taking the OITE during the 2019-2020 academic year. Comparisons in OITE performance were made with parametric tests. A total of 4407 orthopedic surgery residents were in allopathic (86%) and osteopathic (14%) training programs. There was significant improvement in OITE performance between subsequent postgraduate year (PGY) levels among allopathic residents (P<.001). Among osteopathic residents, OITE performance increased between PGY1 and PGY4 (P<.001) but plateaued between PGY4 and PGY5 (P>.05). At the PGY1 level, osteopathic residents had higher OITE performance than allopathic residents (P<.001), but scores were equivalent at the PGY2 to PGY4 levels (P>.05). At the PGY5 level, allopathic residents had higher OITE performance than osteopathic residents (P<.001). Allopathic medical students scored higher on the Step 1 (248±19 vs 242±17, P<.001) and Step 2 (255±16 vs 250±15, P<.001) board examinations. Medical knowledge increases during orthopedic surgery residency. Disparities exist by training pathway, with osteopathic residents outperforming allopathic residents at the PGY1 level but then underperforming at the PGY5 level. Ultimately, this study provides insights into how resident promotion and training pathway impacts the acquisition of medical knowledge during orthopedic surgery residency. [Orthopedics. 2023;46(6):379-383.].

Factors Associated with Infectious Diseases Fellowship Academic Success

Background

A multitude of factors are considered in an infectious diseases (ID) training program's meticulous selection process of ID fellows but their correlation to pre and in-fellowship academic success as well as post-fellowship academic success and short-term outcomes is poorly understood. Our goal was to investigate factors associated with subsequent academic success in fellowship as well as post-fellowship short-term outcomes.

Methods

In 2022, we retrospectively analyzed deidentified academic records from 39 graduates of the Mayo Clinic Rochester ID Fellowship Program (July 1, 2013- June 30, 2022). Data abstracted included demographics, degrees, honor society membership, visa/citizenship status, medical school, residency training program, United States Medical Licensure Exam (USMLE) scores, letters of recommendation, in-training examination (ITE) scores, fellowship track, academic rank, career choice, number of honors, awards, and abstracts/publications prior to fellowship, during training, and within 2 years of graduation.

Results

Younger fellows had higher USMLE step 1 scores, pre and in-fellowship scholarly productivity, and higher ITE performance. Female fellows had significantly higher USMLE step 3 scores. Prior research experience translated to greater in-fellowship scholarly productivity. Higher USMLE scores were associated with higher ID ITE performance during multiple years of fellowship, but USMLE step 2 clinical knowledge and 3 scores were associated with higher pre and in-fellowship scholarly productivity and receiving an award during fellowship. USMLE step 1 score did not correlate with fellowship performance beyond year 1 and 2 ITE scores.

Conclusions

Multiple aspects of a prospective fellow's application must be considered as part of a holistic reviewprocess for fellowship selection. USMLE step 2 CK and 3 scores may predict fellowship performance across multiple domains.

Trends in Residency Academic Productivity of Ophthalmic Plastic and Reconstructive Surgery Fellows Between 2012 and 2019

PURPOSE

To characterize research productivity of ophthalmic plastic and reconstructive surgery (OPRS) fellows during residency.

METHODS

A database was compiled of OPRS fellows listed on the American Society of Ophthalmic Plastic and Reconstructive Surgery (ASOPRS) Annual Fall Scientific Symposium program books who began their fellowship between 2012 and 2019. PubMed was searched for all publications published between July 1st of the year they began residency and September 30th of the year they began fellowship training. Bibliometric variables captured for each fellow included: the number of publications, first-author publications, and ophthalmology-related publications.

RESULTS

A total of 197 OPRS fellows who began their fellowship training between 2012 and 2019 published a mean (± SD) of 2.42 ± 2.80 publications, 1.43 ± 1.85 first-author publications, and 2.33 ± 2.74 ophthalmology-related publications during residency. Linear regression revealed that the number of publications ( P < 0.001), first-author publications ( P < 0.001), and ophthalmology-related publications ( P < 0.001) that OPRS fellows published during residency have all significantly increased over the time assessed.

CONCLUSIONS

The academic productivity of OPRS fellows during residency was quantified through bibliometric analysis to establish a national benchmark for the benefit of both prospective applicants and program directors. Residency research output of OPRS fellows has significantly increased between 2012 and 2019. Since ASOPRS program requirements necessitate academic productivity and thesis completion, publication records and involvement in research become valuable considerations when evaluating fellowship applicants. The knowledge of what accepted fellows have published provides the opportunity to make historical comparisons and may prove useful in the evaluation of the competitiveness of a given year's applicant pool.

The Impact of Post-graduate Year and Program Accreditation Status on In-Training Examination Performance in Orthopaedic Surgery

Introduction  The progression of medical knowledge competency during surgical residency training is poorly understood. This study measures the acquisition of medical knowledge as orthopedic surgery residents advance during training and the impact of accreditation status on orthopedic in-training examination (OITE) performance. Methods  Orthopedic surgery residents taking the OITE during 2020 and 2021 were included. Residents were grouped into cohorts by post-graduate year (PGY) and Accreditation Council for Graduate Medical Education (ACGME) accreditation status. Comparisons were made with parametric tests. Results  Eight thousand eight hundred and seventy-one ACGME-accredited residents (89%) and 1,057 non-ACGME-accredited residents (11%) were evenly distributed by the PGY level (range, 19-21%). Residents in both ACGME- and non-ACGME-accredited residency programs had a significant increase in OITE performance at each PGY level (P<0.001). At ACGME-accredited programs, OITE performance increased from PGY1 (51%), PGY2 (59%), PGY3 (65%), PGY4 (68%), and PGY5 (70%) (P<0.001). There were progressively smaller percentage increases in OITE performance during accredited residency training (range, 2-8%), but this increase was linear in non-accredited residency training (range, 4%). At each PGY level, residents at accredited programs outperformed their counterparts at non-accredited programs (P<0.001). Conclusion OITE performance increases during residency training. Among ACGME-accredited residents, performance on the OITE progresses rapidly during junior years and plateaus during senior years. Residents in ACGME-accredited residency programs outperform their counterparts in non-accredited residency programs. More research is needed to understand optimal training environments that promote medical knowledge acquisition during orthopedic surgery residency.

Trends in Integrated Plastic Surgery Applicant, Resident, and Junior Attending Research Productivity

INTRODUCTION

Research productivity is critical for matching into integrated plastic surgery residency. This study will identify how pre and intraresidency research productivity correlate with resident/junior attending productivity.

MATERIALS AND METHODS

Retrospective review from 2006 to 2015 issues of the American Board of Plastic Surgery's Annual Newsletter to Diplomates was performed to identify newly board certified plastic surgeons. Only surgeons from US medical schools matching directly into integrated programs were included. Residency type/length, graduation year from medical school, and publication counts were recorded for each surgeon. Publications were categorized as preresidency, intraresidency, and junior attending (6 y post residency/fellowship training).

RESULTS

Six hundred fifty-five integrated plastic surgery graduates were analyzed. The median number of total publications (preresidency, intraresidency, and junior attending) was 4 (interquartile range [IQR], 1 to 10). Linear regression revealed negligible correlation between preresidency and junior attending publications (r = 0.019, P = 0.002). Total publications and increasing graduation y had a significant correlation of 0.89 (P < 0.001). Graduates of fellowships had significantly increased median total publications compared to those without fellowships (7 IQR, 3 to 18 versus 3 IQR, 1 to 7, respectively, P < 0.001). Dedicated research years during residency were associated with significant (P < 0.001) increases in median total and junior attending publications. Total publications ranged from 3 (IQR, 1 to 6) to 8 (IQR, 7 to 18) for those who completed 5- and 8-y residencies, respectively.

CONCLUSIONS

Increased preresidency research productivity is not strongly associated with increased junior attending productivity in integrated plastic surgery. Better markers are completing dedicated research years in residency or fellowship after residency.

The Association Between USMLE Step 2 Clinical Knowledge Scores and Residency Performance: A Systematic Review and Meta-Analysis

PURPOSE

With the change in Step 1 score reporting, Step 2 Clinical Knowledge (CK) may become a pivotal factor in resident selection. This systematic review and meta-analysis seeks to synthesize existing observational studies that assess the relationship between Step 2 CK scores and measures of resident performance.

METHOD

The authors searched MEDLINE, Web of Science, and Scopus databases using terms related to Step 2 CK in 2021. Two researchers identified studies investigating the association between Step 2 CK and measures of resident performance and included studies if they contained a bivariate analysis examining Step 2 CK scores' association with an outcome of interest: in-training examination (ITE) scores, board certification examination scores, select Accreditation Council for Graduate Medical Education core competency assessments, overall resident performance evaluations, or other subjective measures of performance. For outcomes that were investigated by 3 or more studies, pooled effect sizes were estimated with random-effects models.

RESULTS

Among 1,355 potential studies, 68 met inclusion criteria and 43 were able to be pooled. There was a moderate positive correlation between Step 2 CK and ITE scores (0.52, 95% CI 0.45-0.59, P < .01). There was a moderate positive correlation between Step 2 CK and ITE scores for both nonsurgical (0.59, 95% CI 0.51-0.66, P < .01) and surgical specialties (0.41, 95% CI 0.33-0.48, P < .01). There was a very weak positive correlation between Step 2 CK scores and subjective measures of resident performance (0.19, 95% CI 0.13-0.25, P < .01).

CONCLUSIONS

This study found Step 2 CK scores have a statistically significant moderate positive association with future examination scores and a statistically significant weak positive correlation with subjective measures of resident performance. These findings are increasingly relevant as Step 2 CK scores will likely become more important in resident selection.

Association of an Early Interest in Orthopedic Surgery with Match Rate into Orthopedic Surgery

OBJECTIVES

Orthopedic surgery residency is considered one of the most competitive specialties in which to match. Studies examining the factors associated with a successful match have neglected whether participation in an orthopedic interest group (OIG) improves the chances of orthopedic residency match. The goal of this study was to test the hypothesis that participation in the OIG would improve matching into an orthopedic surgery residency.

METHODS

We performed a retrospective cohort study between May 2017 and 2019 at one state-funded medical school. All of the applicants in orthopedic surgery from 2004 to 2019 were identified and contacted for OIG membership status. The Office of Student Affairs provided academic performance data (US Medical Licensing Examination scores and third-year clinical clerkship grades), Alpha Omega Alpha and Gold Humanism Honor Society status, and demographics (race and sex) of applicants.

RESULTS

Between 2004 and 2019, 67 students (56 OIG and 11 non-OIG) applied for orthopedic surgery residency match. The match rate for the OIG was 86% compared with 64% for the non-OIG group, resulting in an adjusted odds ratio (adjusted for academic performance) of 10.23 (95% confidence interval 1.14-92.3, P = 0.038).

CONCLUSIONS

OIG membership was associated with a significantly higher rate of orthopedic surgery residency matches. The higher rate of match associated with OIG membership may be the result of opportunities to diversify a residency application. Future studies are needed to further evaluate the potential association between OIG involvement and orthopedic surgery match.

Association Between USMLE Step 1 Scores and In-Training Examination Performance: A Meta-Analysis

PURPOSE

On February 12, 2020, the sponsors of the United States Medical Licensing Examination announced that Step 1 will transition to pass/fail scoring in 2022. Step 1 performance has historically carried substantial weight in the evaluation of residency applicants and as a predictor of subsequent subject-specific medical knowledge. Using a systematic review and meta-analysis, the authors sought to determine the association between Step 1 scores and in-training examination (ITE) performance, which is often used to assess knowledge acquisition during residency.

METHOD

The authors systematically searched Medline, EMBASE, and Web of Science for observational studies published from 1992 through May 10, 2020. Observational studies reporting associations between Step 1 and ITE scores, regardless of medical or surgical specialty, were eligible for inclusion. Pairs of researchers screened all studies, evaluated quality assessment using a modified Newcastle-Ottawa Scale, and extracted data in a standardized fashion. The primary endpoint was the correlation of Step 1 and ITE scores.

RESULTS

Of 1,432 observational studies identified, 49 were systematically reviewed and 37 were included in the meta-analysis. Overall study quality was low to moderate. The pooled estimate of the correlation coefficient was 0.42 (95% confidence interval [CI]: 0.36, 0.48; P < .001), suggesting a weak-to-moderate positive correlation between Step 1 and ITE scores. The random-effects meta-regression found the association between Step 1 and ITE scores was weaker for surgical (versus medical) specialties (beta -0.25 [95% CI: -0.41, -0.09; P = .003]) and fellowship (versus residency) training programs (beta -0.25 [95% CI: -0.47, -0.03; P = .030]).

CONCLUSIONS

The authors identified a weak-to-moderate positive correlation between Step 1 and ITE scores based on a meta-analysis of low-to-moderate quality observational data. With Step 1 scoring transitioning to pass/fail, the undergraduate and graduate medical education communities should continue to develop better tools for evaluating medical students.

Predictors of Success in an Orthopaedic Residency

»

Orthopaedic surgery is one of the most competitive residencies to match into. Meanwhile, the average applicant's United States Medical Licensing Examination (USMLE) test scores, research involvement, and number of clinical honors increase every year.

»

Measures such as USMLE scores, productivity in research, Alpha Omega Alpha (AΩA) honor society status, number of clinical honors, and performance on away rotations have all been cited as factors contributing to program directors choosing applicants for interviews and ranking them for their program. However, questions remain as to whether these measures translate to success on board examinations, high faculty evaluations, and designation as chief resident during orthopaedic residency.

»

USMLE scores have been shown to correlate with Orthopaedic In-Training Examination (OITE) and American Board of Orthopaedic Surgery (ABOS) scores, while clinical grades and AΩA status correlate with faculty evaluations. Participating in research as a medical student was predictive of research productivity in residency but did not correlate with standardized testing scores or faculty evaluations.

»

The literature has suggested ways in which measures such as personality and grit may be used in the application process and how these factors may contribute to predictors of success. However, additional research is needed to measure and define personality and grit during the application evaluation process.

Do Medical Students Who Participate in a Research Gap Year Produce More Research During Residency?

The research gap year has become increasingly popular among medical students. It is also a well-known factor in consideration for orthopaedic surgery residency programs. Although medical students who participated in a research gap year typically enter residency with more research experience than their counterparts, it is unknown whether this translates to increased research productivity during residency compared with their peers. The purpose of this study was to investigate (1) whether residents who participated in a research gap year during medical school publish more peer-reviewed publications than their peers during residency, (2) whether residents who participated in a research gap year during medical school publish more first-author publications than their peers during residency, and (3) which applicant characteristics are associated with a greater number of peer-reviewed publications produced during residency.

The Current State of the Residency Match

This article explores the current state of the residency match in 2020 with a focus on orthopedic surgery, analyzing the utility of current applicant screening methods in producing future generations of competent surgeons. Discussed are anticipated changes to the residency application process considering the COVID-19 pandemic and Step 1 becoming pass/fail in January 2022. Also explored are potential changes to improve the process for applicants and residency programs, such as identifying and using predictive factors of resident success in the applicant screening process, finding better ways to match applicants with programs, and increasing female and underrepresented minorities within orthopedics.

Three Decades Later: A Scoping Review of the Literature Related to the United States Medical Licensing Examination

PURPOSE

To conduct a scoping review of the timing, scope, and purpose of literature related to the United States Medical Licensing Examination (USMLE) given the recent impetus to revise USMLE scoring.

METHOD

The authors searched PubMed, PsycInfo, and ERIC for relevant articles published from 1990 to 2019. Articles selected for review were labeled as research or commentaries and coded by USMLE Step level, sample characteristics (e.g., year(s), single/multiple institutions), how scores were used (e.g., predictor/outcome/descriptor), and purpose (e.g., clarification/justification/description).

RESULTS

Of the 741 articles meeting inclusion criteria, 636 were research and 105 were commentaries. Publication totals in the past 5 years exceeded those of the first 20 years.Step 1 was the sole focus of 38%, and included in 84%, of all publications. Approximately half of all research articles used scores as a predictor or outcome measure related to other curricular/assessment efforts, with a marked increase in the use of scores as predictors in the past 10 years. The overwhelming majority of studies were classified as descriptive in purpose.

CONCLUSIONS

Nearly 30 years after the inception of the USMLE, aspirations for its predictive utility are rising faster than evidence supporting the manner in which the scores are used. A closer look is warranted to systematically review and analyze the contexts and purposes for which USMLE scores can productively be used. Future research should explore cognitive and noncognitive factors that can be used in conjunction with constrained use of USMLE results to inform evaluation of medical students and schools and to support the residency selection process.

Factors Considered When Ranking Military Orthopedic Surgery Residency Candidates

INTRODUCTION

The intangible personal characteristic grit has become a popular topic of concentration within contemporary psychology studies. Grit is defined as persistent work on a focused topic for an extended period of time until a desired goal is achieved. Its application to physician selection and development is not well known. We sought to determine which factors were considered most important among leadership within the U.S. Army's orthopedic surgical training programs when selecting applications and ascertain if grit was one of the primary factors.

MATERIALS AND METHODS

A standardized, 18-question survey was sent to all program chairs, program directors, and assistant program directors at the six U.S. Army orthopedic surgery residency programs. Questions included demographic factors pertaining to both the individual and their respective training program. Participants were asked to rank, in order, the 10 most important variables when considering applicants. Each variable was ranked using a point system (1-10 with 10 as the best score), referred to as the factor score (FS). Further statistical analyses using descriptive statistics, paired t-test, and ANOVA were performed and reported.

RESULTS

The response rate was 83% (15/18). The most important variable considered was the applicant's performance on their audition rotation (FS = 9.8), which was significantly more important than any other variable (P = 0.001). The second most important variable was the applicant's USMLE scores (FS = 7.13), followed by involvement in research (FS = 5.60) and conscientiousness (FS = 4.73), respectively. Grit was considered the fifth most important variable (FS = 4.27). There was no significant difference in the ranking of grit among the different programs (P = 0.282). In applicants with low United States Medical Licensing Examination scores according to leadership idiosyncrasy, their audition rotation was considered a redeeming portion of their application (P = 0.02) followed by their level of grit.

CONCLUSIONS

The most important factor when considering an orthopedic surgery applicant was how well the applicant performed on an audition rotation, followed by their United States Medical Licensing Examination scores. Grit has become an important consideration in resident selection.

Lack of Additional Advanced Graduate Training by Orthopaedic Surgeons in Academic Practice: Current Employment and Recruitment Trends

Background:

Orthopaedic surgery is ever changing and depends on diverse technical and intellectual skill sets. The purpose of the current study was to evaluate the percentage of academic orthopaedic surgeons with additional graduate degrees in the United States.

Methods:

Data including advanced degree(s) (eg, PhD, MS, MBA, MPH, JD, and DVM), academic rank, leadership position, subspecialty, years since training completion, and sex were collected from websites for all academic orthopaedic surgery departments in the United States. Univariate analyses were performed to evaluate for differences in demographic data based on the advanced degree status. Data from the National Resident Matching Program (NRMP) were used to characterize graduate degree-holding US senior medical students who ranked orthopaedic surgery first relative to peers without additional advanced degrees and to applicants who ranked other specialties first.

Results:

Of 4,519 faculty at 175 academic orthopaedic surgery departments in the United States, 7.1% held a graduate degree in addition to a medical doctorate. There was no difference in the percentage of faculty who held departmental leadership positions (P = 0.62) or who were full professors (P = 0.66) based on holding an additional graduate degree. Of 678 US senior applicants who ranked orthopaedic surgery first and successfully matched into the specialty in 2018, 12.5% held an additional graduate degree and 1.3% were MD-PhDs. Orthopaedic surgery had the second lowest percentage of matched medical students with additional advanced degrees, which was significantly lower than the top 10 specialties (range 16.1% to 21.6%; P < 0.05). Orthopaedic surgery recruited 1.6% of all MD-PhD applicants in 2018.

Discussion:

Few academic orthopaedic surgery faculty and admitted orthopaedic residency candidates have additional graduate school training. The low percentage of orthopaedic faculty and trainees with additional advanced degrees relative to other specialties may represent a missed opportunity to recruit individuals with diverse skills to advance the field of orthopaedic surgery.

Integrating the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education: A Narrative Review and Proposed Model for Implementation

The osteopathic undergraduate medical education standards have evolved over the past 2 decades to require undergraduate medical student participation in research and scholarly activity. The authors' objective was to review those evolving standards and develop a model for introducing the principles and practice of research that combines core content with experiential learning. They identified fundamental topics pertinent to the research process and herein provide their recommendations for incorporating these topics into the curriculum as self-study, online modules, and team-based and active learning exercises. The authors' proposed educational model would provide an avenue for osteopathic medical schools to meet the requirements for scholarly activity. Increasing students' and residents' knowledge of the research process will lay the foundation for their engagement is research and scholarly activity and their practice of evidence-based medicine.