Orthopaedic resident-selection criteria

Analysis of variability and trends in medical school clerkship grades

Background

Medical school clerkship grades are used to evaluate orthopedic surgery residency applicants, however, high interinstitutional variability in grade distribution calls into question the utility of clerkship grades when evaluating applicants from different medical schools. This study aims to evaluate the variability in grade distribution among medical schools and look for trends in grade distribution over recent years.

Methods

Applications submitted to Rush University's orthopedic surgery residency program from 2015, 2019, and 2022 were collected from the Electronic Residency Application Service. Applications from the top 100 schools according to the 2023-2024 U.S. News and World Report Research Rankings were reviewed. The percentage of "honors" grades awarded by medical schools for the surgery and internal medicine clerkships were extracted from applicants' Medical Student Performance Evaluation letters.

Results

The median percentage of honors given in 2022 was 36.0 % (range 10.0-82.0) for the surgery clerkship and 33.0 % (range 6.7-80.0) for the internal medicine clerkship. Honors were given 6.6 % more in the surgery clerkship in 2022 compared to 2015. There was a negative correlation between a higher (worse) U.S. News and World Report research ranking and the percentage of honors awarded in 2022 for the surgery and internal medicine clerkships.

Conclusion

There is substantial interinstitutional variability in the rate that medical schools award an "honors" grade with evidence of grade inflation in the surgery clerkship. Residency programs using clerkship grades to compare applicants should do so cautiously provided the variability demonstrated in this study.

Characterization of Unpublished Manuscripts by Applicants to an Orthopedic Hand Surgery Fellowship

Purpose

Obtaining a hand surgery fellowship is becoming increasingly competitive, and research is an important factor when assessing applications. Given the competitive nature of the fellowship application process, applicants may feel the need to bolster their application by misrepresenting their research experience. One form of misrepresentation rarely discussed in prior studies is the listing of submitted works under a "Publications" heading in curricula vitae. This study examines the prevalence of misclassification of manuscripts by applicants to a hand surgery fellowship and identifies factors that might be associated with incorrect classification.

Methods

A retrospective review of 122 applicants to the 2020-2021 cycle for hand surgery fellowship was performed. Names and identifiable information were redacted prior to review. Demographic data collected included sex, United States Medical Licensing Examination Step 1 score, medical school rank, residency specialty, total publications, presence of submitted manuscripts in the "Publications" section, total number of submitted manuscripts, and total published abstracts and poster presentations.

Results

A total of 1,098 listed publications across the 122 applicants were reviewed with a median of five publications per applicant. Submitted manuscripts were listed as publications by 33 applicants (27%). No observable differences by age, United States Medical Licensing Examination Step 1 score, or total number of publications were seen. Misclassification rates were not associated with publication totals.

Conclusions

More than one-quarter of applicants incorrectly listed submitted or unaccepted manuscripts as publications. It is our hope that making fellowship applicants aware of this issue will decrease the rates of misrepresentation in future application cycles.

Clinical relevance

The competition for hand surgery fellowships has become more intense, and this may explain our finding that 27% of applicants misrepresent the status of research on hand surgery fellowship applications.

Effectiveness of an Orthopaedic Trauma Research Fellowship to Produce Successfully Matching Residents

OBJECTIVE

Orthopaedics is becoming increasingly competitive. Approximately 25% of applicants to orthopaedic surgery go unmatched each year. The mean US Medical Licensing Examination step scores and average publication numbers have increased markedly in recent years. Reapplicants have a match rate of <60%. This study describes the results of an orthopaedic trauma research fellowship and its effectiveness in obtaining a successful orthopaedic match.

METHODS

A 1 to 2-year research fellowship was established at a level 1 academic trauma center. Prefellowship and fellowship metrics of 11 fellows were recorded, including undergraduate and medical schools; step-1 + step-2 scores; Alpha Omega Alpha appointment; and publication, podium, poster, and chapter accomplishments.

RESULTS

The average step-1 score of the fellows was 218 (range, 192 to 252) and 232 (range, 212 to 254) for step-2. Seven of 11 fellows were reapplicants. Prefellowship, the average number of journal publications was 1, one podium, two posters, and zero textbook chapters. During fellowship, the average publications was 5, five podiums, six posters, and 1.5 textbook chapters. Ten of 11 fellows successfully matched into an orthopaedic residency, with six of seven being reapplicants.

CONCLUSIONS

Six of 7 reapplying fellows (86%) successfully matched highlighting the effectiveness of this fellowship. Research fellowships should be considered as an excellent choice for applicants who may be less than ideal candidates or reapplicants.

The use of virtual interviews and new trends in orthopedic surgery residency match

This retrospective cross-sectional study sought to determine if there was a change in geographical trends in the orthopedic surgery residency match with the use of virtual interviews. Due to the COVID-19 pandemic, visiting rotations at outside institutions were restricted and all residency interviews were conducted virtually for the 2021 match. Given these restrictions, it was hypothesized that applicants would match at a higher rate to their medical school-affiliated residency programs, or geographically nearby their medical school. Data was collected from residency program website and social media accounts to determine if the use of virtual interviews correlated with a decreased rate of matching at outside institutions. During the 2021 match, applicants who applied to orthopedic surgery in 2021 were more likely to match at their medical school-affiliated institution (OR, 1.46; 95% CI, 1.18-1.80; p < 0.01) compared to applicants during previous years. However, match rates were not different in terms of geographical regions. Virtual interviews are more cost-effective for both applicants and programs, however, this study demonstrates there are associated changes with this new interview structure. The utility of virtual interviews as a standard method in the future should take this change in trend into consideration.

Preference-Signaling During the Orthopaedic Surgery Residency Application Process

The orthopaedic surgery residency selection process has grown more competitive over recent years, with programs receiving an unprecedented number of applications. As an effort to target applications to programs of interest, the American Orthopaedic Association has announced the introduction of a formal preference-signaling program into the 2022 to 2023 orthopaedic surgery residency selection cycle. This system will allow applicants to assign "signals" to a total of 30 programs. The purpose of this article was to (1) discuss implications of the new preference-signaling program, (2) introduce the framework of the "strategic signaling spear" for applicants to conceptualize the power of all methods of preference-signaling to improve their odds of matching, and (3) describe the role of strong mentorship at all stages of the residency application process.

Perspectives of Orthopedic Surgery Program Directors on the USMLE Step 1 Scoring Change

The orthopedic surgery residency application process is highly competitive, and residency programs rely on objective measures, such as the United States Medical Licensing Examination (USMLE) Step 1 score, to assist in applicant selection. To deemphasize standardized test scores and improve student well-being, governing bodies have elected to change the Step 1 examination to a pass/fail grading system beginning in 2022. Given the utility of Step 1 in the orthopedic surgery residency application process, this change will significantly affect how applicants are assessed. To determine how this change will affect the process, we developed and validated a 19-item anonymous electronic survey and invited program directors (PDs) from orthopedic surgery residency programs accredited by the Accreditation Council for Graduate Medical Education to respond. The 86 surveys that were completed represented a 43.2% response rate (86 of 199). Only 4.7% of orthopedic surgery PDs believe that this change is a good idea, and 91.7% of PDs believe that the change will make it more difficult to compare applicants objectively. In addition, 90.7% of PDs report that they will increase emphasis on the Step 2 CK (clinical knowledge) examination in comparing residency applicants for their program, with 90.4% of PDs reporting that they will require students to submit their Step 2 scores via the Electronic Residency Application Service. These results indicate that changing the Step 1 scoring to pass/fail is unpopular among orthopedic surgery PDs. Further, losing the numeric Step 1 score will increase reliance on Step 2 scores and emphasis on less objective measures, such as where an applicant attended medical school. [Orthopedics. 2022;45(5):e257-e262.].

Race, But Not Gender, Is Associated With Admissions Into Orthopaedic Residency Programs

BACKGROUND

Orthopaedic surgery is one of the most competitive but least diverse surgical specialties, with ever-increasing academic achievements (such as test scores) shown by applicants. Prior research shows that white applicants had higher United States Medical Licensing Exam (USMLE) Step 1 and Step 2 Clinical Knowledge scores as well as higher odds of Alpha Omega Alpha status compared with Black, Hispanic, and other applicant groups. Yet, it still remains unknown whether differences in application metrics by race/ethnicity sufficiently explain the underrepresentation of certain racial or ethnic minority groups in orthopaedic residency programs.

QUESTIONS/PURPOSES

In this study, we sought to determine (1) the relative weight of academic variables for admission into orthopaedic residency, and (2) whether race and gender are independently associated with admission into an orthopedic residency.

METHODS

The Electronic Residency Application System (ERAS) data from the Association of American Medical Colleges (AAMC) and the National Board of Medical Examiners (NBME) of first-time MD applicants (n = 8966) for orthopaedic surgery residency positions in the United States and of admitted orthopaedic residents (n = 6218) from 2005 to 2014 were reviewed. This dataset is the first and most comprehensive of its kind to date in orthopaedic surgery. Academic metrics, such as USMLE Step 1 and Step 2 Clinical Knowledge scores, number of publications, Alpha Omega Alpha status, volunteer experiences, work experience, as well as race and gender, were analyzed using hierarchical logistic regression models. The first model analyzed the association of academic metrics with admission into orthopaedic residency. In the second model, we added race and gender and controlled for metrics of academic performance. To determine how well the models simulated the actual admissions data, we computed the receiver operating characteristics (ROC) including the area under curve (AUC), which measures the model's ability to simulate which applicants were admitted or not admitted, with an AUC = 1.0 representing a perfect simulation. The odds ratio and confidence interval of each variable were computed.

RESULTS

When only academic variables were analyzed in the first model, Alpha Omega Alpha status (odds ratio 2.12 [95% CI 1.80 to 2.50]; p < 0.001), the USMLE Step 1 score (OR 1.04 [95% CI 1.03 to 1.04]; p < 0.001), the USMLE Step 2 Clinical Knowledge score (OR 1.01 [95% CI 1.01 to 1.02]; p < 0.001), publication count (OR 1.04 [95% CI 1.03 to 1.05]; p < 0.001), and volunteer experience (OR 1.03 [95% CI 1.01 to 1.04]; p < 0.001) were associated with admissions into orthopaedics while work and research experience were not. This model yielded a good prediction of the results with an AUC of 0.755. The second model, in which the variables of race and gender were added to the academic variables, also had a good prediction of the results with an AUC of 0.759. This model indicates that applicant race, but not gender, is associated with admissions into orthopaedic residency. Applicants from Asian (OR 0.78 [95% CI 0.67 to 0.92]), Black (OR 0.63 [95% CI 0.51 to 0.77], Hispanic (OR 0.48 [95% CI 0.36 to 0.65]), or other race groups (OR 0.65 [95% CI 0.55 to 0.77]) had lower odds of admission into residency compared with white applicants.

CONCLUSION

Minority applicants, but not women, have lower odds of admission into orthopaedic surgery residency, even when accounting for academic performance metrics. Changes in the residency selection processes are needed to eliminate the lower admission probability of qualified minority applicants in orthopaedic residency and to improve the diversity and inclusion of orthopaedic surgery. Changes including increasing the diversity of the selection committee, bias training, blinding applications before review, removal of metrics with history of racial disparities from an interviewer's candidate profile before an interview, and use of holistic application review (where an applicants' experiences, attributes, and academic metrics are all considered) can improve the diversity landscape in training. In addition, cultivating an environment of inclusion will be necessary to address these long-standing trends in orthopaedic surgery.

CLINICAL RELEVANCE

Race, but not gender, is associated with the odds of acceptance into orthopaedic surgery residency despite equivalent academic metrics. Changes in residency selection processes are suggested to eliminate the lower admission probability of qualified minority applicants into orthopaedic residency and to improve the diversity and inclusion of orthopaedic surgery.

Factors involved in applicant interview selection and ranking for chronic pain medicine fellowship

INTRODUCTION

Applicants to chronic pain medicine fellowship programs often express confusion regarding the importance of various selection criteria. This study sought to elucidate program directors' considerations in applicant selection for fellowship interviews and ranking and to correlate these criteria with match statistics to provide a guide for prospective candidates.

METHODS

An electronic survey was sent to all Accreditation Council for Graduate Medical Education-accredited chronic pain fellowship directors. The importance of various applicant characteristics were evaluated and compared with recent match data.

RESULTS

Fifty-seven program directors completed the survey. The most important factors involved in applicant interview selection were perceived commitment to the specialty, letters of recommendation from pain faculty, scholarly activities, and leadership experiences. Although completion of a pain rotation was valued highly, experience with procedures was of relatively low importance. There was no preference if rotations were completed within the responders' department. Variability was noted when considering internal applicants or the applicant's geographic location. When citing main factors in ranking applicants, interpersonal skills, interview impression and applicant's fit within the institution were highly ranked by most responders.

DISCUSSION

Assessment of an applicant's commitment to chronic pain is challenging. Most responders prioritize the applicant's commitment to chronic pain as a specialty, scholarly activity, participation in chronic pain rotations, pain-related conferences and letters of recommendation from pain faculty. Chronic pain medicine fellowship candidates should establish a progressive pattern of genuine interest and involvement within the specialty during residency training to optimize their fellowship match potential.

Social Media and the Orthopaedic Surgery Residency Application Process

Introduction: Social media use in the residency application process has been on the rise, yet applicant and program director (PD) perspectives in orthopaedic residency have not been explored in depth. Our objective is to evaluate perspectives of orthopaedic residency applicants and PDs on social media usage and its impact on the residency application process.

Methods: Cross-sectional survey study conducted in 2021 with two related, anonymous surveys sent to residency applicants and PDs. One hundred thirteen of 562 (20.1%) applicants to our institution and 29 of 148 (19.6%) PDs listed on the Accreditation Council for Graduate Medical Education (ACGME) website responded to our survey and were included in the analysis.

Results: Applicants reported learning about residency programs through Instagram (69.2%), the program website (58.9%), and Doximity (29.9%). Thirty-four percent of applicants reported social media influencing their rank list, with non-training-related posts being the most impactful. PDs reported that 97% of their programs have an official web page, 41% have an active Instagram site, 27% have a Twitter account but none regularly update Doximity. Just over 48% of PDs reported institutional support for online content creation. Financial investment varied, with 35% reporting no spending, and 24% spending over $2501. In response to coronavirus disease 2019 (COVID-19) restrictions, 79% of PDs reported generating increased social media content.

Discussion: Social media provides a low-cost but far-reaching opportunity for PDs to recruit residents and highlight their respective programs. Social media content should display the culture and lifestyle of the program, with consistency in content creation. PDs should also ensure accuracy on external sites such as Doximity.

The US Medical Licensing Examination Step 1 Scoring Change: A Survey of Orthopaedic Surgery Residency Applicants From the 2019 to 2020 Match Cycle

INTRODUCTION

The USMLE Step 1 examination has been used as an objective measure for comparing residency applicants. Recently, the National Board of Medical Examiners and the Federation of State Medical Boards decided that the USMLE Step 1 examination will transition to a pass/fail result starting no earlier than 2022. The purpose of this study was to investigate the perspective of medical students who applied for orthopaedic surgery residency positions during the 2019 to 2020 interview cycle on the USMLE scoring change, and the potential effect this change may result in for future applicants and the residency selection process.

METHODS

A 15-item anonymous web-based survey was sent to 1,090 orthopaedic surgery residency applicants from four regionally diverse residency programs. The survey elicited attitudes toward the transition of the Step 1 examination to pass/fail and perspectives this change may or may not have on the residency selection process.

RESULTS

Responses were received from 356 applicants (32.7%). The majority (61.6%) disagreed with the change to pass/fail scoring, and 68.5% do not believe that the change will decrease stress levels in medical students. For interview invitations, respondents chose Step 2 clinical knowledge, letters of recommendation, and performance on away rotations as the most influential factors in the absence of a Step 1 score.

CONCLUSION

Most of the students surveyed who applied for an orthopaedic surgery residency position during the most recent application cycle disagreed with the National Board of Medical Examiner/Federation of State Medical Board decision to change Step 1 to pass/fail and feel that this change may have disadvantage in certain student groups while either increasing or having no effect on medical student stress.

LEVEL OF EVIDENCE

IV.

Examining the Factors Influencing Applicants' Placement on One General Surgery Program's Rank Order List

PURPOSE

Surgery residency applications include variables that determine an individual's rank on a program's match list. We performed this study to determine which residency application variables are the most impactful in creating our program's rank order list.

METHODS

We completed a retrospective examination of all interviewed applicants for the 2019 match. We recorded United States Medical Licensing Examinations (USMLE) step I and II scores, class quartile rank from the Medical Student Performance Evaluation (MSPE), Alpha Omega Alpha (AOA) membership, geographic region, surgery clerkship grade, and grades on other clerkships. The MSPE and letters of recommendation were reviewed by two of the authors and assigned a score of 1 to 3, where 1 was weak and 3 was strong. The same two authors reviewed the assessments from each applicant's interview and assigned a score from 1-5, where 1 was poor and 5 was excellent. Univariate analysis was performed, and the significant variables were used to construct an adjusted multivariate model with significance measured at P < .05.

RESULTS

Univariate analysis for all 92 interviewed applicants demonstrated that USMLE step 2 scores (P = .002), class quartile rank (P = .004), AOA status (P = .014), geographic location (P < .001), letters of recommendation (P < .001), and interview rating (P < .001) were significant in predicting an applicant's position on the rank list. On multivariate analysis only USMLE step 2 (P = .018) and interview (P < .001) remained significant.

CONCLUSION

USMLE step 2 and an excellent interview were the most important factors in constructing our rank order list. Applicants with a demonstrated strong clinical fund of knowledge that develop a rapport with our faculty and residents receive the highest level of consideration for our program.

The Geographic Movement Patterns and Career and Personal Interests of Orthopaedic Surgery Residents in the United States

Orthopaedic surgery continues to be a highly desired residency specialty for graduating medical students in the United States. The geographic preferences and trajectories of orthopaedic surgery applicants are not well understood.

Are Narrative Letters of Recommendation for Medical Students Interpreted as Intended by Orthopaedic Surgery Residency Programs?

BACKGROUND

Narrative letters of recommendation are an important component of the residency application process. However, because narrative letters of recommendation are almost always positive, it is unclear whether those reviewing the letters understand the writer's intended strength of support for a given applicant.

QUESTIONS/PURPOSES

(1) Is the perception of letter readers for narrative letters of recommendation consistent with the intention of the letter's author? (2) Is there inter-reviewer consistency in selection committee members' perceptions of the narrative letters of recommendation?

METHODS

Letter writers who wrote two or more narrative letters of recommendation for applicants to one university-based orthopaedic residency program for the 2014 to 2015 application cycle were sent a survey linked to a specific letter of recommendation they authored to assess the intended meaning regarding the strength of an applicant. A total of 247 unstructured letters of recommendation and accompanying surveys were sent to their authors, and 157 surveys were returned and form the basis of this study (response percentage 64%). The seven core members of the admissions committee (of 22 total reviewers) at a university-based residency program were sent a similar survey regarding their perception of the letter. To answer our research question about whether letter readers' perceptions about a candidate were consistent with the letter writer's intention, we used kappa values to determine agreement for survey questions involving discrete variables and Spearman correlation coefficients (SCCs) to determine agreement for survey questions involving continuous variables. To answer our research question regarding inter-reviewer consistency among the seven faculty members, we compared the letter readers' responses to each survey question using intraclass correlation coefficients (ICCs).

RESULTS

There was a negligible to moderate correlation between the intended and perceived strength of the letters (SCC 0.26 to 0.57), with only one of seven letter readers scoring in the moderate correlation category. When stratifying the applicants into thirds, there was only slight agreement (kappa 0.07 to 0.19) between the writers and reviewers. There were similarly low kappa values for agreement about how the writers and readers felt regarding the candidate matching into their program (kappa 0.14 to 0.30). The ICC for each question among the seven faculty reviewers ranged from poor to moderate (ICC 0.42 to 0.52).

CONCLUSION

Our results demonstrate that the reader's perception of narrative letters of recommendation did not correlate well with the letter writer's intended meaning and was not consistent between letter readers at a single university-based urban orthopaedic surgery residency program.

CLINICAL RELEVANCE

Given the low correlation between the intended strength of the letter writers and the perceived strength of those letters, we believe that other options such as a slider bar or agreed-upon wording as is used in many dean's letters may be helpful.

Rate of USMLE Step 2 CK Scores Included on Orthopedic Surgery Applications and Associations With Step 1 Score

Background At present, orthopedic surgery applicants do not universally include Step 2 Clinical Knowledge (Step 2 CK) scores on their applications and current inclusion rates are not yet reported. As Step 1 transitions to pass/fail scoring, we suspect more applicants will include Step 2 CK scores. We sought to identify what percentage of applications currently include Step 2 CK, if applicants who include Step 2 CK (Step 1+CK) score lower on Step 1 than those not including Step 2 CK (Step 1-CK), and what correlations exist between Step 1 and Step 2 CK scores among those who include the scores on their applications. Methodology Applicants to one residency program over two application cycles (2019-2020) were analyzed. The percentage including Step 1 and Step 2 scores was recorded. Step 1 scores were compared between Step 1+CK and Step 1-CK applicants. Differences between Step 2 CK and Step 1 scores were stratified by Step 1 score. Results A total of 1,688 applicants applied to our institution from 2019 to 2020. Of those reporting United States Medical Licensing Examination scores, 1,316/1,660 (79%) reported a Step 2 CK score. Step 1-CK applicants scored higher on Step 1 (250.7 ± 10.9) versus Step 1+CK applicants (244.3 ± 13.1) (p < 0.0001). More applicants who scored lower on Step 1 improved upon their percentile rank between Step 1 and Step 2 CK than those who scored higher on Step 1 (χ² _(8,1316) = 79.1, p < 0.0001). Conclusions From 2019 to 2020, 79% of applicants included Step 2 CK. Lower Step 1 scores were more likely to include Step 2 CK and improve upon their percentile score. It is unclear how Step 1 scoring change will affect current practice.

Characterizing the use of class rank in evaluating applicants for veterinary internship and residency positions

OBJECTIVE

To characterize how class rank and other criteria are used to evaluate applicants for veterinary internship and residency positions.

SAMPLE

Program directors for 572 internship and residency programs.

PROCEDURES

A survey was sent to program directors asking them to score the importance of 7 items (cover letter, letters of reference, curriculum vitae, veterinary class rank, grade point average, grades for classes specifically related to the internship or residency specialty area, and interview) they could use in evaluating applicants for an internship or residency and to rank those 7 items, along with an open item asking participants to list other criteria they used, from most to least important.

RESULTS

Responses were obtained for 195 internship and 222 residency programs. For both internship programs and residency programs, mean importance scores assigned to the 7 items resulted in the same ordering from most to least important, with letters of reference, interview, curriculum vitae, and cover letter most important. Rankings of the importance of the 7 items, along with an "other" item, were similar for internship and residency programs; the most important item was a candidate's letters of reference, followed by the interview, cover letter, and curriculum vitae.

CONCLUSIONS AND CLINICAL RELEVANCE

Results suggested that although most veterinary internship and residency programs consider class rank and overall grade point average when evaluating applicants, these 2 items were not the most important. For both internship and residency programs, the most important items were an applicant's letters of reference, followed by the interview, cover letter, and curriculum vitae. (J Am Vet Med Assoc 2021;258:776-785).

The Change of USMLE Step 1 to Pass/Fail: Perspectives of the Surgery Program Director

OBJECTIVE

This study sought to evaluate the perspectives of surgical program directors regarding the change of USMLE Step 1 to pass/fail grading.

DESIGN

Validated electronic survey.

SETTING

Vanderbilt University Medical Center Department of Plastic Surgery.

PARTICIPANTS

Program directors of all ACMGE-accredited General Surgery, Integrated Vascular Surgery, Integrated Thoracic Surgery, and Integrated Plastic Surgery residency programs.

RESULTS

The overall response rate was 55.5%. Most PDs (78.1%) disagreed with the scoring change. Only 19.6% believe this change will improve medical student well-being. For 63.5% of PDs, medical school pedigree will become more important, and 52.7% believe it will place international medical graduates at a disadvantage. Only 6.2% believe Step 2 CK should also be pass/fail, while 88.7% will increase the weight of Step 2 CK and 88.4% will now require Step 2 CK score submission with the electronic residency application service.

CONCLUSIONS

While well-intentioned, changing USMLE Step 1 to pass/fail may have unintended consequences and may disadvantage certain groups of applicants. The emphasis on Step 1, and resulting test-taking apprehension, will likely shift to Step 2 CK. Proponents of equitable evaluation should direct their efforts toward increasing, not decreasing, the number of objective measures available for student assessment.

Resident Selection in the Wake of United States Medical Licensing Examination Step 1 Transition to Pass/Fail Scoring

INTRODUCTION

The numeric score for the United States Medical Licensing Examination Step 1 is one of the only universal, objective, scaled criteria for comparing the many students who apply to orthopaedic surgery residency. However, on February 12, 2020, it was announced that Step 1 would be transitioning to pass/fail scoring. The purpose of this study was to (1) determine the most important factors used for interview and resident selection after this change and (2) to assess how these factors have changed compared with a previous report on resident selection.

METHODS

A survey was distributed to the program directors (PDs) of all 179 orthopaedic surgery programs accredited by the Accreditation Council for Graduate Medical Education. Questions focused on current resident selection practices and the impact of the Step 1 score transition on expected future practices.

RESULTS

A total of 78 PDs (44%) responded to the survey. Over half of PDs (59%) responded that United States Medical Licensing Examination Step 2 clinical knowledge (CK) score is the factor that will increase most in importance after Step 1 transitions to pass/fail, and 90% will encourage applicants to include their Step 2 CK score on their applications. The factors rated most important in resident selection from zero to 10 were subinternship performance (9.05), various aspects of interview performance (7.49 to 9.01), rank in medical school (7.95), letters of recommendation (7.90), and Step 2 CK score (7.27). Compared with a 2002 report, performance on manual skills testing, subinternship performance, published research, letters of recommendations, and telephone call on applicants' behalf showed notable increases in importance.

DISCUSSION

As Step 2 CK is expected to become more important in the residency application process, current applicant stress on Step 1 scores may simply move to Step 2 CK scores. Performance on subinternships will remain a critical aspect of residency application, as it was viewed as the most important resident selection factor and has grown in importance compared with a previous report.

Factors Considered When Ranking Military Orthopedic Surgery Residency Candidates

INTRODUCTION

The intangible personal characteristic grit has become a popular topic of concentration within contemporary psychology studies. Grit is defined as persistent work on a focused topic for an extended period of time until a desired goal is achieved. Its application to physician selection and development is not well known. We sought to determine which factors were considered most important among leadership within the U.S. Army's orthopedic surgical training programs when selecting applications and ascertain if grit was one of the primary factors.

MATERIALS AND METHODS

A standardized, 18-question survey was sent to all program chairs, program directors, and assistant program directors at the six U.S. Army orthopedic surgery residency programs. Questions included demographic factors pertaining to both the individual and their respective training program. Participants were asked to rank, in order, the 10 most important variables when considering applicants. Each variable was ranked using a point system (1-10 with 10 as the best score), referred to as the factor score (FS). Further statistical analyses using descriptive statistics, paired t-test, and ANOVA were performed and reported.

RESULTS

The response rate was 83% (15/18). The most important variable considered was the applicant's performance on their audition rotation (FS = 9.8), which was significantly more important than any other variable (P = 0.001). The second most important variable was the applicant's USMLE scores (FS = 7.13), followed by involvement in research (FS = 5.60) and conscientiousness (FS = 4.73), respectively. Grit was considered the fifth most important variable (FS = 4.27). There was no significant difference in the ranking of grit among the different programs (P = 0.282). In applicants with low United States Medical Licensing Examination scores according to leadership idiosyncrasy, their audition rotation was considered a redeeming portion of their application (P = 0.02) followed by their level of grit.

CONCLUSIONS

The most important factor when considering an orthopedic surgery applicant was how well the applicant performed on an audition rotation, followed by their United States Medical Licensing Examination scores. Grit has become an important consideration in resident selection.

When (Almost) Everyone is Above Average: A Critical Analysis of American Orthopaedic Association Committee of Residency Directors Standardized Letters of Recommendation

The American Orthopaedic Association introduced standardized letters of recommendations (SLORs) to improve on traditional letters of recommendations by “providing a global prospective on an applicant.” However, no study has defined the utilization of SLORs, the distribution of applicant ratings in SLORs, or the impact of sex, race, or degree of involvement between the letter writer and applicant on SLOR domain ratings.

Personality Assessment and Emotional Intelligence in Orthopaedic Surgery Residency Selection and Training

Orthopaedic surgery has increasingly emerged as one of the most popular and competitive of all medical specialties. The selection of the best applicants has become a more difficult process because the number of qualified medical students has concomitantly grown. Although there are standardized guidelines in residency selection, there are several intangible factors that determine an applicant's fit into a program. Personality assessments, such as the Neuroticism-Extraversion-Openness Personality Inventory and the Myers-Briggs Type Indicator, are proficiencies that have garnered significant interest to help fill this void. Understanding and measuring empirically supported measures of personality traits and styles of medical students and residents may be valuable to medical educators and program directors in a variety of applications such as residency selection, mentoring, and education. Similar to personality identification, emotional intelligence assessments, such as the Mayer-Salovey-Caruso Emotional Intelligence Test, can prove to be pivotal tools in residency education and training. Emotional intelligence has shown to align with current Accreditation Council for Graduate Medical Education residency standardized core competencies that emphasize aptitude in noncognitive characteristics.

Lack of Additional Advanced Graduate Training by Orthopaedic Surgeons in Academic Practice: Current Employment and Recruitment Trends

Background:

Orthopaedic surgery is ever changing and depends on diverse technical and intellectual skill sets. The purpose of the current study was to evaluate the percentage of academic orthopaedic surgeons with additional graduate degrees in the United States.

Methods:

Data including advanced degree(s) (eg, PhD, MS, MBA, MPH, JD, and DVM), academic rank, leadership position, subspecialty, years since training completion, and sex were collected from websites for all academic orthopaedic surgery departments in the United States. Univariate analyses were performed to evaluate for differences in demographic data based on the advanced degree status. Data from the National Resident Matching Program (NRMP) were used to characterize graduate degree-holding US senior medical students who ranked orthopaedic surgery first relative to peers without additional advanced degrees and to applicants who ranked other specialties first.

Results:

Of 4,519 faculty at 175 academic orthopaedic surgery departments in the United States, 7.1% held a graduate degree in addition to a medical doctorate. There was no difference in the percentage of faculty who held departmental leadership positions (P = 0.62) or who were full professors (P = 0.66) based on holding an additional graduate degree. Of 678 US senior applicants who ranked orthopaedic surgery first and successfully matched into the specialty in 2018, 12.5% held an additional graduate degree and 1.3% were MD-PhDs. Orthopaedic surgery had the second lowest percentage of matched medical students with additional advanced degrees, which was significantly lower than the top 10 specialties (range 16.1% to 21.6%; P < 0.05). Orthopaedic surgery recruited 1.6% of all MD-PhD applicants in 2018.

Discussion:

Few academic orthopaedic surgery faculty and admitted orthopaedic residency candidates have additional graduate school training. The low percentage of orthopaedic faculty and trainees with additional advanced degrees relative to other specialties may represent a missed opportunity to recruit individuals with diverse skills to advance the field of orthopaedic surgery.

The Current Status of Medical School Clerkship Grades in Residency Applicants

BACKGROUND

Two criteria that have been investigated for evaluating orthopedic surgery residency candidates are achieving an "honors" grade during a surgery clerkship and the total number of honors grades received in all clerkships. Unfortunately, the rate of honors grades given and the criteria for earning an honors grade differ between medical schools, making comparison of applicants from different medical schools difficult.

OBJECTIVE

We measured the rate of honors grades in clerkships at different medical schools in the United States to examine the utility of clerkship grades in evaluating orthopedic surgery residency applicants.

METHODS

Adequate data via the Electronic Residency Application Service were available for 86 of 142 Association of American Medical Colleges medical schools from the 2017 Match cycle. Descriptive statistics and Wilcoxon rank sum tests were performed to identify differences in grade distributions within each clerkship and in school ranking for research by U.S. News & World Report.

RESULTS

For the surgery clerkship, the median rate of honors grades given was 32.5% (range 5%-67%). There was a high rate of interinstitutional variability in all clerkships. We were unable to demonstrate a statistically significant relationship between research ranking and percentage honors grades given for individual clerkships.

CONCLUSIONS

A standardized method for grading medical students during clinical clerkships does not exist, resulting in a high degree of interinstitutional variability. Surgery clerkship grades are an unreliable measure for comparing orthopedic surgery residency applicants from different medical schools. Standardized measures of applicant evaluation might be helpful in the future.

Selecting Residents for Predetermined Factors Identified and Thought to be Important for Work Performance and Satisfaction: A Methodology

OBJECTIVE

The medical profession seeks to hire and train individuals who consistently meet and/or exceed both job and cultural expectations. Resident selection is often not structured to meet this goal. The objective of this quality improvement project was to evaluate a classic unscripted interview process (OI) in conjunction with a structured, scripted interview process (SI) developed using an established hiring methodology from industry not yet utilized in health care. Qualitative questions we sought to answer: (1) Can SI be practically applied to the selection of residents? (2) Is there a significant difference in the relative position of applicants between the OI and SI rank lists? (3) Qualitatively, does SI help the evaluation/discussion of the affective domain?

METHODS

Design: Prospective qualitative comparison of OI versus SI.

SETTING

Dartmouth Hitchcock Medical Center, Lebanon, NH.

PARTICIPANTS

Applicants were assessed by OI and SI. SI factors were selected based on a job profile. Interview scripts were created from validated behavioral and attitudinal questions. Online assessments assessed 2 important attributes - adaptability and values. Rank lists were compared for relative rank position of applicants. Feedback from faculty was obtained.

RESULTS

Fifty-two applicants. Critical attributes were self-management, integrator-synthesizer, versatility, communication, and achievement. Absolute mean difference in rank/applicant was 9.8 (standard deviation 8.9, Range 0-36) positions. Comparing the top 20 candidates of each rank list, 40% of those applicants were only on one list. Faculty felt that applicants were given a greater opportunity to show "who they are."

CONCLUSIONS

In conjunction with OI, an industry proven methodology was practically applied to define and select for high performance for the authors' specific institution. Comparing OI and SI resulted in substantial differences in rank lists. This initiative seemed to provide a structure to evaluate values and motivations that are inherently difficult to assess. Faculty felt SI in conjunction with OI gave a greater chance for applicants to show "who they are."

Factors Predictive of Orthopaedic In-training Examination Performance and Research Productivity Among Orthopaedic Residents

INTRODUCTION

Selection of qualified candidates for orthopaedic residency is necessary for growth and innovation. The purpose of this study was to determine predictors of Orthopaedic In-training Exam (OITE) performance and research productivity.

METHODS

A survey was distributed to 13 residency programs collecting demographics, United States Medical Licensing Examination (USMLE) and OITE scores, and authored publications. Associations between preresidency qualifications and OITE scores and publications were determined.

RESULTS

A total of 274 of 294 surveys were returned (93.2%). We found a positive correlation between USMLE step 1 and 2 scores with recent OITE percentile (P < 0.001). Preresidency authorship (P < 0.001) and postgraduate training year (P < 0.001) were independent predictors of authorship during residency, whereas USMLE step 1 score was not (P = 0.094).

CONCLUSION

Candidates who perform well on the USMLE are likely to perform well on the OITE, whereas those with greater authored publications are likely to continue research during residency.

Matching into an Orthopedic Residency: Which Application Components Correlate with Final Rank List Order?

OBJECTIVE

To determine if any of the unique elements of the applications of medical students who were granted interviews to an orthopedic residency program correlate to the program's final rank order list of candidate residents.

DESIGN

Eight domains of the standard residency application were considered as independent variables for 36 applicants. Personal, identifying information was removed from the application material within each domain, thus blinding the application domains for 5 core faculty members from the program to review and rank independently. These 8 domain rank lists were then compared to the program's final rank list order to determine the correlation of each domain with the final rank list order.

SETTING

Academic medical center.

PARTICIPANTS

Applicants to a university-based orthopedic surgery residency program who were granted interviews in the 2016 academic year.

RESULTS

Two domains of the application correlated with the final rank list order: interview and personal statement. None of the other domains had a significant correlation with the final rank list order. Interobserver variability among the faculty members was high for the rankings of the different domains.

CONCLUSIONS

Interview and personal statement were the only domains within the application that had correlation with the final rank list for an orthopedic residency program. A better understanding of how these 2 components affect the rank list may provide opportunity for process improvements.

The Match: To Thine Own Self Be True

The residency match process, culminating with the Match Day celebration, plays out in medical schools across the United States and Canada every year. The process may seem strange and mysterious for observers outside of medicine. The notion that each graduating student's employer for the next several years is first revealed to thousands of people, all at the same moment, through the opening of an envelope is surreal. The emotional reactions accompanying the process range from jubilance to deep disappointment. Much attention and care have been given to developing the algorithm underpinning the Match, and the process seems just: Optimization favors applicants over training programs. Witnessing students as they progress to their next stage of medical training is special for those involved in medical education. Faculty are filled with pride. But the process is far from perfect. The author of this Invited Commentary notes several concerns about the Match: the arduous process that students undergo to maximize their chances of success; the costs attendant to the travel and related expenses of multiple, geographically dispersed interviews; and the metrics that students and their medical schools use to judge the outcomes. The author worries that for some students, the "ideal" match may not be the one driven by their dreams and aspirations but, rather, by an amalgamation of those of many well-meaning friends, family members, and faculty. Medical students should seek advice and guidance, but the author hopes that, ultimately, students follow their own drumbeat and are true first to themselves.

A Computer Simulation Model to Analyze the Application Process for Competitive Residency Programs

BACKGROUND

The residency match process for competitive specialties hinders programs' ability to holistically review applications.

OBJECTIVE

A computer simulation model of the residency application process was created to test the hypotheses that (1) it is advantageous to medical students to apply to the maximum number of programs under the current system, and (2) including a medical student's residency program preferences at the beginning of the application process improves the efficiency of the system for applicants and programs as quantified by the number of interview invitations received.

METHODS

The study was conducted in 2016 using 2014 Otolaryngology Match data. A computer model was created to perform simulations for multiple scenarios to test the hypotheses. Students were assigned scores representing easy and hard metrics and program preferences, simulating a mixture of individual student preference and general program popularity.

RESULTS

We modeled a system of 99 otolaryngology residency programs with 292 residency spots and 460 student applicants. While it was individually advantageous for an applicant to apply to the maximum number of programs, this led to a poor result for the majority of students when all applicants undertook the strategy. The number of interview invitations improved for most applicants when preference was revealed.

CONCLUSIONS

Offering applicants an option to provide program preference improves the practical number of interview invitations. This enables programs to review applicants holistically-instead of using single parameters such as United States Medical Licensing Examination scores-which facilitates a selection of applicants who will be successful in residency.

Psychomotor Testing for Orthopedic Residency Applicants: A Pilot Study

OBJECTIVE

The purpose of this study was to develop an objective motor skills testing system to aid in the evaluation of potential orthopedic residents.

DESIGN

Participants attempted a battery of 5 motor skills tests (4 novel tests and the Grooved Pegboard [GPT] Test) in one 10-minute session. A percentile-based scoring system was created for each test based on raw scores. One-way analysis of variance was used to compare testing scores among 3 cohorts. Each novel test and overall scores were compared with GPT scores as a relative measure of validity.

SETTING

The 2015 orthopedic surgery residency interview season at an academic institution.

PARTICIPANTS

Thirty orthopedic residents and 72 nonresidents (15 community volunteers and 57 orthopedic surgery residency applicants).

RESULTS

Overall, residents performed better than nonresidents (p < 0.0001) and applicants performed worse than residents or volunteers (p < 0.0001). There were positive correlations between the GPT score and overall battery score (r = 0.63), screw and nut test (r = 0.40), and mimic a structure test (r = 0.26). The fracture reduction test and drilling test scores did not correlate to performance on the GPT.

CONCLUSIONS

Psychomotor testing for surgical applicants is an area in need of study. This investigation successfully piloted a novel battery of tests, which is easily reproducible and thus may be feasible for use in the orthopedic surgery residency interview setting. Longitudinal evaluation is required to explore correlation with future operative skill.

The Role of Letters of Recommendation in the Selection Process of Surgical Residents in Canada: A National Survey of Program Directors

OBJECTIVE

Letters of recommendation (LOR) provide valuable information that help in selecting new residents. In this study, we aim to investigate the perceptions of surgical residency program directors (PDs) in Canada on the elements that can affect the strength and value of LOR.

DESIGN

Cross-sectional; survey.

SETTING

A national survey was conducted using an online questionnaire consisting of 2 main sections to collect data from PDs from all surgical subspecialties. The first section included basic background questions about the participant, such as the specialty and experience in selecting resident candidates, whereas the second section was about the elements and characteristics of LOR. Participants were asked to rate the importance of 34 different variables using a Likert scale.

PARTICIPANTS

Surgical PDs in Canada.

RESULTS

Of 122 PDs, 65 (53.3%) participated in the survey. Work ethic (57; 87.7%), interpersonal skills (52; 80.0%), and teamwork (49; 75.4%) were considered very important parts of the LOR by more than three-quarters of the PDs. Thirty-three (50.8%) PDs reported that a familiar author of LOR would always affect their impression regarding the letter. Additionally, 57 (87.7%) and 35 (53.8%) directors thought that LOR are important in evaluating the candidates and can help in predicting the residents' performance during their residency training.

CONCLUSIONS

LOR are important for the selection of new surgical residents in Canada. Information about the candidate's work ethic, interpersonal skills, and teamwork is essential for a good LOR. Familiarity of PDs with authors of LOR could increase the value of the letter.

Factors Motivating Medical Students in Selecting a Career Specialty: Relevance for a Robust Orthopaedic Pipeline

INTRODUCTION

Selection of a career specialty by medical students is a complex and individualized decision. Our goals were to understand the factors that influenced medical students in selecting their career specialty, identify the stage at which this decision was made, and understand the role of demographics, mentors, and curricula in this process.

METHODS

Medical students from 10 institutions participated in a web-based survey. Results were stratified by sex, race/ethnicity, and level of interest in orthopaedic surgery.

RESULTS

A total of 657 students responded to the survey. Specialty content (mean rating, 8.4/10) and quality of life/lifestyle/stress level (7.5/10) were the primary motivating factors in selecting a specialty. Interest in orthopaedic surgery was lower in women than in men (2.7 versus 3.9; P < 0.01) and was equivalent among race/ethnicity groups. Although 27% of students reported moderate or extensive medical school curriculum exposure to orthopaedics, this education did not sway them toward the specialty.

CONCLUSIONS

Levels of interest in orthopaedics among medical students may be lower than generally assumed. Increasing the attractiveness of the specialty will require a multifaceted approach, including recognition of lifestyle factors, adjustments in the orthopaedic clerkship to make the specialty more appealing, mentorship by orthopaedic faculty, and conversion of high levels of interest in the specialty among minority medical students into successful residency applications.

LEVEL OF EVIDENCE

IV.

Factors affecting orthopedic residency selection: a cross-sectional survey

BACKGROUND

Annually, orthopedic residency programs rank and recruit the best possible candidates. Little evidence exists identifying factors that potential candidates use to select their career paths. Recent literature from nonsurgical programs suggests hospital, social and program-based factors influence program selection. We sought to determine what factors influence the choice of an orthopedic career and a candidate's choice of orthopedic residency program.

METHODS

We surveyed medical student applicants to orthopedic programs and current Canadian orthopedic surgery residents (postgraduate year [PGY] 1-5). The confidential online survey focused on 3 broad categories of program selection: educational, program cohesion and noneducation factors. Questions were graded on a Likert Scale and tailed for mean scores.

RESULTS

In total, 139 residents from 11 of 17 Canadian orthopedic programs (49% response rate) and 23 medical student applicants (88% response rate) completed our survey. Orthopedic electives and mandatory rotations were reported by 71% of participants as somewhat or very important to their career choice. Collegiality among residents (4.70 ± 0.6), program being the "right fit" (4.65 ± 0.53) and current resident satisfaction with their chosen program (4.63 ±0.66) were ranked with the highest mean scores on a 5-point Likert scale.

CONCLUSION

There are several modifiable factors that residency programs may use to attract applicants, including early availability of clerkship rotations and a strong mentorship environment emphasizing both resident-resident and resident-staff cohesion. Desirable residency programs should develop early access to surgical and operative skills. These must be balanced with a continued emphasis on top-level orthopedic training.

Orthopaedic Surgery Residency: Perspectives of Applicants and Program Directors on Medical Student Away Rotations

INTRODUCTION

Senior medical students frequently rotate at orthopaedic residency programs away from their home medical schools. However, to our knowledge, the perspective of program directors (PDs) and applicants on the value, objectives, and costs of these rotations has not been studied.

METHODS

Surveys evaluating the frequency, costs, benefits, and objectives of away rotations were distributed to all orthopaedic residency PDs in the United States and applicants in the 2014-2015 Match cycle. Data analysis was conducted to perform inferential and descriptive statistics; comparisons were made between and among PD and applicant groups using two-tailed means Student t-test and analysis of variance.

RESULTS

A total of 74 PD (46.0%) and 524 applicant (49.3%) responses were obtained from a national distribution. Applicants completed an average of 2.4 away rotations, with an average cost of $2,799. When stratified on self-reported likelihood of Matching, there were no substantial differences in the total number of rotations performed. The only marked differences between these groups were the United States Medical Licensing Examination Step 1 score and the number of applications to residency programs. PDs reported that significantly fewer rotations should be allowed, whereas applicants suggested higher limits (2.42 rotations versus 6.24, P < 0.001). PDs and applicants had similar perspectives on the value of away rotations; both groups reported more value in finding a "good fit" and making a good impression at the program and placed less value on the educational impact.

DISCUSSION

The value of orthopaedic away rotations appears more utilitarian than educational for both PDs and applicants. Rotations are performed regardless of perceived likelihood of Matching and are used by students and programs to identify a "good fit." Therefore, given the portion of an academic year that is spent on orthopaedic rotations, findings showing perceived low educational value and marked discrepancies between the expected number of rotations by PDs and applicants indicate that the current structure of away rotations may not be well aligned with the mission of undergraduate medical education.

LEVEL OF EVIDENCE

V.

The prevalence and cost of medical student visiting rotations

BACKGROUND

Performance on visiting rotations during the senior year of medical school is consistently cited by residency program directors as a critical factor in selecting residents. Nevertheless, the frequency with which visiting rotations are undertaken and the associated financial costs they impose have not been systematically examined.

METHOD

Under the auspices of the Electronic Residency Application Service, a survey was sent in March 2015 to all U.S. applicants for residency programs in the 2014-15 academic year. Students were asked how many visiting rotations they performed; the estimated cost of performing each rotation; their perception of their educational value and primary motivation for performing them; and the Match outcome of their residency application.

RESULTS

The survey was completed by 2817 applicants, yielding a response rate of 11.3 %. 1898 applicants (67.4 %) performed visiting rotations: 647 applicants (30.0 %) performed one; 640 (22.7 %) performed two; 322 (11.4 %) performed three; and 289 (10.3 %) reported four or more. When accounting for potential response bias, the true prevalence of away rotators was estimated to be 58.7 % of all fourth-year medical students (95 % CI 54.0-63.4 %). The mean number of rotations for participating students was 2.1. Most students performed rotations equally as an audition for residency placement and for education, with some of the more competitive subspecialties reporting more of an audition experience. The mean estimated cost for performing a single rotation was $958. Thirty-six percent of applicants reported matching at an institution where they had rotated, either their home institution or one at which a visiting rotation was performed.

CONCLUSIONS

Visiting rotations are prevalent, expensive, and only partly educational. As such, these rotations may impede optimal use of the senior year of medical school and limited student financial resources.

Selecting the Best and Brightest: A Structured Approach to Orthopedic Resident Selection

BACKGROUND

Resident selection is integral to the graduate medical educational process and the future of our profession. There is no consensus among residency directors as to how to systematically and consistently screen and select applicants who would perform well as residents. The purpose of this study was to introduce and assess a high volume application screening tool and semistructured interview process.

METHODS

This study took place in an academic orthopedic surgery department over 2 years (2013-2014). Overall, 1382 applications were screened in 7 categories, with a maximum score of 100. A total of 14 faculty reviewed applications; 218 interviews were offered; 165 applicants accepted the interview. Overall, 4 interview domains (cognitive, affective, activities, and theme), and an impression score were ranked from 1 (Exceptional) to 6 (Concern). Each room had an assigned "theme" (ethics, affective, cognitive, research, and "fit") with standardized questions. A summary score was generated of all scores to determine the preliminary rank list; the final rank list was determined after group discussion. Correlation between preliminary rank, final rank, and screening scores were assessed.

RESULTS

The average screening score was 62.5 (range: 0-100, median = 64). The average interview score was 69.5 (range: 32.24-95.0). Final rank lists correlated most highly with initial rank (0.912, p < 0.001), impression (0.847, p < 0.001), and affective domain (0.834, p < 0.001). Cognitive domain (0.628, p < 0.001) and screening scores (0.264, p < 0.001) less highly correlated with final rank position.

CONCLUSIONS

A systematic approach was used to screen and evaluate a large number of orthopedic surgery applicants. Our system demonstrated excellent feasibility, reliability, and predictability for the final rank list.

Selection for Surgical Training: An Evidence-Based Review

PURPOSE

The predictive relationship between candidate selection criteria for surgical training programs and future performance during and at the completion of training has been investigated for several surgical specialties, however there is no interspecialty agreement regarding which selection criteria should be used. Better understanding the predictive reliability between factors at selection and future performance may help to optimize the process and lead to greater standardization of the surgical selection process.

METHODS

PubMed and Ovid MEDLINE databases were searched. Over 560 potentially relevant publications were identified using the search strategy and screened using the Cochrane Collaboration Data Extraction and Assessment Template.

RESULTS

57 studies met the inclusion criteria. Several selection criteria used in the traditional selection demonstrated inconsistent correlation with subsequent performance during and at the end of surgical training. The following selection criteria, however, demonstrated good predictive relationships with subsequent resident performance: USMLE examination scores, Letters of Recommendation (LOR) including the Medical Student Performance Evaluation (MSPE), academic performance during clinical clerkships, the interview process, displaying excellence in extracurricular activities, and the use of unadjusted rank lists.

CONCLUSIONS

This systematic review supports that the current selection process needs to be further evaluated and improved. Multicenter studies using standardized outcome measures of success are now required to improve the reliability of the selection process to select the best trainees.

Academic Characteristics of Orthopaedic Surgery Residency Applicants from 2007 to 2014

BACKGROUND

Based on a relatively stable match rate, several authors have concluded that the competition for orthopaedic residency positions has not changed over the past 3 decades. However, the objective measures of applicant competitiveness have not been quantified in detail.

METHODS

National Resident Matching Program (NRMP) data from 2007 to 2014 for U.S. orthopaedic surgery applicants were compared with data for applicants to all specialties. Trends in the United Stated Medical Licensing Examination (USMLE) Step-1 and Step-2 scores, publications and research experiences, Alpha Omega Alpha (AOA) status, and the presence of an advanced degree are reported.

RESULTS

From 2007 to 2014, the match rate for orthopaedic surgery applicants remained stable near 80% (p = 0.14). For orthopaedic applicants who matched, the mean USMLE Step-1 scores increased from 234 points in 2007 to 245 points in 2014 (p = 0.005), and the mean scores increased from 220 points in 2007 to 229 points in 2014 for all applicants (p = 0.019). The mean USMLE Step-2 scores of orthopaedic applicants who matched increased from 235 points in 2007 to 251 points in 2014 (p = 0.005), and the mean scores of all applicants increased from 225 points in 2007 to 242 points in 2014 (p = 0.002). The mean number of research publications, presentations, and abstracts reported by orthopaedic applicants who matched more than doubled from 3.0 in 2007 to 6.7 in 2014 (p = 0.02) and increased less dramatically for all applicants from 2.2 in 2007 to 4.2 in 2014 (p = 0.004). The percentage of orthopaedic applicants elected to AOA or with advanced degrees did not significantly change (p > 0.2). Although orthopaedic applicants with AOA status experienced a very high match rate (97.1% in 2014), those with advanced degrees experienced match rates similar to or slightly lower than the applicant pool (73.7% in 2014).

CONCLUSIONS

The USMLE Step-1 and 2 scores of U.S. orthopaedic surgery residency applicants have increased significantly from 2007 to 2014. Additionally, the number of publications and presentations reported by orthopaedic applicants has more than doubled. These factors signal an increasing level of academic accomplishment in orthopaedic surgery applicants despite a consistent match rate.

Does Residency Selection Criteria Predict Performance in Orthopaedic Surgery Residency?

BACKGROUND

More than 1000 candidates applied for orthopaedic residency positions in 2014, and the competition is intense; approximately one-third of the candidates failed to secure a position in the match. However, the criteria used in the selection process often are subjective and studies have differed in terms of which criteria predict either objective measures or subjective ratings of resident performance by faculty.

QUESTIONS/PURPOSES

Do preresidency selection factors serve as predictors of success in residency? Specifically, we asked which preresidency selection factors are associated or correlated with (1) objective measures of resident knowledge and performance; and (2) subjective ratings by faculty.

METHODS

Charts of 60 orthopaedic residents from our institution were reviewed. Preresidency selection criteria examined included United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores, Medical College Admission Test (MCAT) scores, number of clinical clerkship honors, number of letters of recommendation, number of away rotations, Alpha Omega Alpha (AOA) honor medical society membership, fourth-year subinternship at our institution, and number of publications. Resident performance was assessed using objective measures including American Board of Orthopaedic Surgery (ABOS) Part I scores and Orthopaedics In-Training Exam (OITE) scores and subjective ratings by faculty including global evaluation scores and faculty rankings of residents. We tested associations between preresidency criteria and the subsequent objective and subjective metrics using linear correlation analysis and Mann-Whitney tests when appropriate.

RESULTS

Objective measures of resident performance namely, ABOS Part I scores, had a moderate linear correlation with the USMLE Step 2 scores (r = 0.55, p < 0.001) and number of clinical honors received in medical school (r = 0.45, p < 0.001). OITE scores had a weak linear correlation with the number of clinical honors (r = 0.35, p = 0.009) and USMLE Step 2 scores (r = 0.29, p = 0.02). With regards to subjective outcomes, AOA membership was associated with higher scores on the global evaluation (p = 0.005). AOA membership also correlated with higher global evaluation scores (r = 0.60, p = 0.005) with the strongest correlation existing between AOA membership and the "interpersonal and communication skills" subsection of the global evaluations.

CONCLUSIONS

We found that USMLE Step 2, number of honors in medical school clerkships, and AOA membership demonstrated the strongest correlations with resident performance. Our goal in analyzing these data was to provide residency programs at large a sense of which criteria may be "high yield" in ranking applicants by analyzing data from within our own pool of residents. Similar studies across a broader scope of programs are warranted to confirm applicability of our findings. The continually emerging complexities of the field of orthopaedic surgery lend increasing importance to future work on the appropriate selection and training of orthopaedic residents.

Characteristics That Define a Successful Pharmacy Resident as Perceived by Residency Programs

PURPOSE

To identify the subjective and objective characteristics that pharmacy residency programs use to define a successful resident and to determine what percentage of their 2009-2010 residency class they felt were successful.

METHODS

An electronic survey was sent via e-mail to all residency program directors (RPDs) of postgraduate year 1 (PGY1) and postgraduate year 2 (PGY2) pharmacy residency programs in the United States. A 3-part survey instrument was developed following validation of questions for clarity and reliability using a pilot survey. Respondents were asked to rank the importance of 20 subjective characteristics for a resident to possess in order to be considered successful and the importance of different objective measurements of accomplishment in the definition of a "successful" resident using a Likert scale where 1 = not at all important, 2 = some importance, 3 = very important, and 4 = critical.

RESULTS

Of the 1,081 surveys sent to RPDs, 473 respondents answered at least one question, yielding a response rate of 43.8%. The most critically important subjective characteristics in defining a successful resident as ranked among PGY1 residency programs are dependability, professionalism, self-motivation/initiative, and work ethic. PGY2 programs ranked clinical knowledge and skills, critical thinking, and dependability as the most important. The most critically important objective characteristic in defining a successful resident as ranked among both PGY1 and PGY2 programs is obtaining a clinical position. The majority of PGY1 and PGY2 respondents felt that 76% to 100% of their 2009-2010 residency class was successful based on the characteristics they rated most important.

CONCLUSION

Identification of the characteristics that pharmacy residency programs use to define success will allow them to identify predictors of success and optimal methods of selecting residents who possess these characteristics.

Selecting Residents in Podiatric Medicine and Surgery

Limited information exists to guide students of podiatric medicine and residency directors through the resident selection process. The present study aimed to evaluate the podiatric medicine and surgery resident selection process using an online survey. Residency directors of podiatric medicine and surgery programs across the United States and fourth-year students across all 9 colleges of podiatric medicine were contacted for participation. Two separate surveys were created, one for the directors and one for the students. The directors and students were asked the relative importance of 21 items considered in resident selection on a 7-point importance scale. Subsequent questions covered an array of related topics. The directors, compared with the students, identified the following items as more important (p < .05): previous disciplinary actions against the student, number of classes failed during school, undergraduate experiences and activities, number of Part I board attempts, class rank, involvement in research, and grade point average during podiatric medical school. The manual dexterity portion of the residency interview was considered significantly more important by the students than the directors. The directors more satisfied with their residents placed greater importance on the following items (p < .05): opinions of current residents, opinions of other attending physicians, and letters of recommendation. Additional trends and differences were also discovered. The results of the present study provide baseline data on the selection of podiatric medicine and surgery residents.

Assessment of moral reasoning skills in the orthopaedic surgery resident applicant

BACKGROUND

Assessing orthopaedic surgery residency applicants is a complex process. One important applicant characteristic not commonly measured during the interview process is moral behavior, a key component of the guidelines of the Accreditation Council for Graduate Medical Education for professionalism competency. We sought to determine whether a relationship exists between the results of residency selection interviews, as measured by match rank order, and moral reasoning skills, as measured by the revised version of the Defining Issues Test (DIT-2).

METHODS

The DIT-2, a psychological tool that has been validated in assessing moral reasoning in >30,000 subjects, was administered to orthopaedic surgery residency candidates on a voluntary basis during their interview day. Data were collected annually from four applicant classes (2009 to 2012) at one institution. Each candidate was interviewed by an orthopaedic faculty team and was ranked comparatively with the other interviewing applicants. The five-part aggregate interview method for ranking applicants for the match list included a focused interview station presenting ethical dilemmas as one part of determining rank order.

RESULTS

One hundred and twenty-four fourth-year medical students interviewing for orthopaedic surgery residency were assessed with the DIT-2. The results from the DIT-2 produced a postconventional score, which correlates with various levels of moral development. Each applicant's postconventional score was then compared with the patient's overall rank order. Applicants had highly variable moral reasoning skills, with no relationship to the final rank list.

CONCLUSIONS

Despite a wide range of DIT-2 scores, the interview process did not differentiate applicants based on moral reasoning skills. It remains unclear whether the DIT-2 could (or should) be used as an additional data point to help stratify orthopaedic residency applicants and to predict their performance.

What factors influence applicants' rankings of orthopaedic surgery residency programs in the National Resident Matching Program?

BACKGROUND

In accordance with the American Academy of Orthopaedic Surgeons' strategic goal of enriching our field by building a more diverse orthopaedic workforce, the specialty needs further information delineating the factors important to the applicant pool as a whole and more specifically to women and other underrepresented minority groups.

QUESTIONS/PURPOSES

This study aims to identify (1) factors important to residency applicants selecting an orthopaedic residency program; (2) differences in factor importance for men, women, and minorities, and (3) the importance of different information sources used when making his or her rank list.

METHODS

All 742 applicants who applied to the authors' orthopaedic surgery residency program in the 2013 National Resident Matching Program were queried. The response rate was 28% (207 of 742). Respondents were asked to rank, on a 5-point Likert scale, 37 factors that may have affected their rank lists to differing degrees. Respondents also identified the importance of sources of information used to make their rank lists, factors that residency programs considered important when ranking applicants, and their level of agreement with various sex- and racial-specific statements regarding orthopaedic training.

RESULTS

The most important factors affecting rank lists were perceived happiness/quality of life of current residents, resident camaraderie, and impression after an away rotation. Women weighed their personal interactions and a program's proximity to family and friends more heavily when determining a rank list. Sixty-eight percent of women eliminated residency programs from their options based on perceived sex biases versus less than 1% of men. Applicants valued information obtained from away rotations at an institution and in talking with current residents most when determining his or her rank list.

CONCLUSIONS

Programs should consider interpersonal factors, like quality of life and resident camaraderie as factors in attracting applicants. They also should minimize perceived biases and emphasize interactions with current residents during the application process to meet their goals of attracting an exceptional and more diverse orthopaedic workforce.

Is orthopedics more competitive today than when my attending matched? An analysis of National Resident Matching Program data for orthopedic PGY1 applicants from 1984 to 2011

OBJECTIVE

This study evaluated supply and demand trends for orthopedic postgraduate year 1 (PGY1) positions from 1984 to 2011 for the purpose of estimating national intercandidate competition over time.

DESIGN

National Resident Matching Program (NRMP) data for orthopedic surgery from 1984 to 2011 were collected. Proxy variables including (total number of orthopedic applicants/number of orthopedic PGY1 positions), (number of US senior applicants to orthopedics/number of orthopedic PGY1 positions), (number of US seniors matching into orthopedics/number of US senior orthopedic applicants), (total number of matched orthopedic applicants/total number of orthopedic applicants), and (total number of US applicants who fail to match into orthopedics/total number of US senior applicants into orthopedics) as well as average United States Medical Licensing Examination Step 1 scores were used to gauge the level of competition between candidates and were compared over time.

SETTING

Academic medical center in the Midwestern United States.

PARTICIPANTS

Medical professors and medical students.

RESULTS

The NRMP data suggested that the number of positions per applicant decreased or remained stable since 1984 and that the percentage of applicants who did not match was no higher now than in the past. This finding was primarily because of the relative decrease in the ratio of applicants to available PGY1 positions, which stems from the number of positions increasing more rapidly than the number of applicants.

CONCLUSIONS

The NRMP data from 1984 to 2011 supported our hypothesis that intercandidate competition intensity for orthopedic PGY1 positions has not increased over time. The misconception that orthopedics is becoming more competitive likely arises from the increased number of applications submitted per candidate and the resulting relative importance placed on objective criteria such as United States Medical Licensing Examination Step 1 scores when programs select interview cohorts.

Council of Emergency Medicine Residency Directors' standardized letter of recommendation: the program director's perspective

OBJECTIVES

The standardized letter of evaluation (SLOE) was created in 1997 to provide residency program directors (PDs) with a summative evaluation that incorporates normative grading (i.e., comparisons to peers applying to emergency medicine [EM] training). Although the standard letter of recommendation (SLOR) has become increasingly popular and important in decision-making, it has not been studied in the past 12 years. To assess the SLOR's effectiveness and limitations, the perspective of EM PDs was surveyed in this study.

METHODS

After validation of the questionnaire by 10 retired PDs, the survey was sent to the PD of each of the 159 EM residencies that existed at that time. The survey was circulated via the Council of Emergency Medicine Residency Directors' (CORD) listserv from January 24, 2013, to February 13, 2013. Weekly e-mail reminders to all PDs served to increase participation.

RESULTS

A total of 150 of 159 PDs (94.3%) completed the questionnaire. Nearly all respondents (149 of 150; 99.3%) agreed that the SLOR is an important evaluative tool and should continue to be used. In the application process, 91 of 150 (60.7%) programs require one or more SLORs, and an additional 55 (36.7%) recommend but do not require a SLOR to be considered for interview. When asked to identify the top three factors in deciding who should be interviewed, the SLOR was ranked first (139 of 150; 92.7%), with EM rotation grades ranked second (73 of 150; 48.7%). The factors that were most often identified as the top three that diminish the value of the SLOR in order were 1) "inflated evaluations" (121 of 146; 82.9%), 2) "inconsistency between comments and grades" (106 of 146; 72.6%), and 3) "inadequate perspective on candidate attributes in the written comments" and "inexperienced authors" (60 of 146; 41.1% each).

CONCLUSIONS

The SLOR appears to be the most important tool in the EM PD's armamentarium for determining which candidates should be interviewed for residency training. Although valuable, the SLOR's potential utility is hampered by a number of factors, the most important of which is inflated evaluations. Focused changes in the SLOR template should be mindful that it appears, in general, to be successful in its intended purpose.

Selection criteria of residents for residency programs in Kuwait

BACKGROUND

In Kuwait, 21 residency training programs were offered in the year 2011; however, no data is available regarding the criteria of selecting residents for these programs. This study aims to provide information about the importance of these criteria.

METHODS

A self-administered questionnaire was used to collect data from members (e.g. chairmen, directors, assistants …etc.) of residency programs in Kuwait. A total of 108 members were invited to participate. They were asked to rate the importance level (scale from 1 to 5) of criteria that may affect the acceptance of an applicant to their residency programs. Average scores were calculated for each criterion.

RESULTS

Of the 108 members invited to participate, only 12 (11.1%) declined to participate. Interview performance was ranked as the most important criteria for selecting residents (average score: 4.63/5.00), followed by grade point average (average score: 3.78/5.00) and honors during medical school (average score: 3.67/5.00). On the other hand, receiving disciplinary action during medical school and failure in a required clerkship were considered as the most concerning among other criteria used to reject applicants (average scores: 3.83/5.00 and 3.54/5.00 respectively). Minor differences regarding the importance level of each criterion were noted across different programs.

CONCLUSIONS

This study provided general information about the criteria that are used to accept/reject applicants to residency programs in Kuwait. Future studies should be conducted to investigate each criterion individually, and to assess if these criteria are related to residents' success during their training.

In-training factors predictive of choosing and sustaining a productive academic career path in neurological surgery

BACKGROUND

Factors during neurosurgical residency that are predictive of an academic career path and promotion have not been defined.

OBJECTIVE

To determine factors associated with selecting and sustaining an academic career in neurosurgery by analyzing in-training factors for all graduates of American College of Graduate Medical Education (ACGME)-accredited programs between 1985 and 1990.

METHODS

Neurological surgery residency graduates (between 1985 and 1990) from ACGME-approved training programs were analyzed to determine factors associated with choosing an academic career path and having academic success.

RESULTS

Information was available for 717 of the 720 (99%) neurological surgery resident training graduates (678 male, 39 female). One hundred thirty-eight graduates (19.3%) held full-time academic positions. One hundred seven (14.9%) were professors and 35 (4.9%) were department chairs/chiefs. An academic career path/success was associated with more total (5.1 vs 1.9; P < .001) and first-author publications (3.0 vs 1.0; P < .001) during residency. Promotion to professor or chair/chief was associated with more publications during residency (P < .001). Total publications and first-author publications were independent predictors of holding a current academic position and becoming professor or chair/chief. Although male trainees published more than female trainees (2.6 vs 0.9 publications; P < .004) during training, no significant sex difference was observed regarding current academic position. Program size (≥ 2 graduates a year; P = .02) was predictive of an academic career but not predictive of becoming professor or chair/chief (P > .05).

CONCLUSION

Defined in-training factors including number of total publications, number of first-author publications, and program size are predictive of residents choosing and succeeding in an academic career path.