Novel Strategies to Improve Resident Selection by Improving Cultural Fit: AOA Critical Issues

Diversity in Orthopaedic Surgery Residencies Based on Allopathic Medical School Affiliation

Orthopaedic surgery is one of the most competitive and least diverse specialties in medicine. Affiliation of an orthopaedics with an allopathic medical school impacts research opportunities and early exposure to clinical orthopaedics. The purpose of this study is to examine the potential effect allopathic medical school affiliation has on orthopaedic surgery resident demographics and academic characteristics.

Methods

All 202 Accreditation Council for Graduate Medical Education (ACGME)-accredited orthopaedics programs were divided into 2 groups: Group 1 consisted of residency programs without an affiliated allopathic medical school, and Group 2 consisted of programs with an affiliated allopathic medical school. Affiliations were determined by cross-referencing the ACGME residency program list with the medical school list published by Association of American Medical Colleges (AAMC). Program and resident characteristics were then compiled using AAMC's Residency Explorer including region, program setting, number of residents, and osteopathic recognition. Resident characteristics included race, gender, experiences (work, volunteer, and research), peer-reviewed publications, and US Medical Licensing Examination Step 1 scores.

Results

Of the 202 ACGME-accredited orthopaedics residencies, Group 1 had 61 (30.2%) programs, and Group 2 had 141 (69.8%) programs. Group 2 had larger programs (4.9 vs. 3.2 resident positions/year; p < 0.001) and 1.7 times the number of residency applicants (655.8 vs. 385.5; p < 0.001). Most Group 2 residents were allopathic medical school graduates, 95.5%, compared with 41.6% in Group 1. Group 1 had 57.0% osteopathic medical school graduates, compared with 2.9% in Group 2. There were 6.1% more White residents in Group 1 residencies (p = 0.025), and Group 2 residencies consisted of 3.5% more Black residents in relation to Group 1 (p = 0.03). Academic performance metrics were comparable between the 2 groups (p > 0.05).

Conclusion

This study demonstrated that candidates who successfully match into an orthopaedic surgery residency program achieve high academic performance, regardless of whether the program was affiliated with an allopathic medical school. Differences may be influenced by increased representation of minority faculty, greater demand for allopathic residents, or stronger emphasis on promotion of diversity in those residency programs.

Availability of Data and Material

Available on reasonable request.

Level of Evidence

Level III. See Instructions for Authors for a complete description of levels of evidence.

The Orthopaedic Resident Selection Process: Proposed Reforms and Lessons From Other Specialties

INTRODUCTION

Proposals for substantive reforms to the orthopaedic resident selection process are growing, given increasing applicant competitiveness, burgeoning inefficiencies and inequities of the current system, and impending transition of Step 1 to pass/fail. The COVID-19 pandemic has further catalyzed the need for reforms, offering unprecedented opportunities to pilot novel changes. However, a comprehensive collation of all proposed and implemented orthopaedic reforms is currently lacking. Thus, we aimed to characterize proposed orthopaedic-specific resident selection reforms in the context of reforms implemented by other specialties.

METHODS

EMBASE, MEDLINE, Scopus, and Web of Science databases were searched for references proposing reforms to the orthopaedic resident selection process published from 2005 to 2020. An inductive approach to qualitative content analysis was used to categorize reforms.

RESULTS

Twenty-six articles proposing 13 unique reforms to the orthopaedic resident selection process were identified. The most commonly proposed reforms included noncognitive assessments (n = 8), application caps (n = 7), standardized letters of recommendation (n = 5), program-specific supplemental applications (n = 5), creation of a centralized database of standardized program information (n = 4), use of a standardized applicant composite score (n = 4), and a moratorium on postinterview communication (n = 4). Importantly, nearly all of these reforms have also been proposed or implemented by other specialties.

DISCUSSION

Numerous reforms to the orthopaedic resident selection process have been suggested over the past 15 years, several of which have been implemented on a program-specific basis, including noncognitive assessments, supplemental applications, and standardized letters of recommendation. Careful examination of applicant and program experiences and Match outcomes after these reforms is imperative to inform future directions.

Otolaryngology residency selection: are we doing it right?

PURPOSE OF REVIEW

This article will provide an overview of recent disruptions to the otolaryngology residency match process and conclude with questions and resources that can guide future selection system design.

RECENT FINDINGS

During the implementation of the single accreditation system, the loss of osteopathic programs, reduction of osteopathic leadership positions, and lack of interest in Osteopathic Recognition represent serious threats to the profession; this has implications for the distribution of the otolaryngology workforce, plausibly decreasing healthcare access in less-populated communities. Next, the impacts of COVID-19 reverberated throughout the application process, including the reduction/elimination of away rotations, modification of application requirements, conversion to virtual interviews, and initiation of preference signaling. Soon, the transition to pass/fail scoring for the United States Medical Licensing Exam Step 1 could stimulate a paradigm shift, with a heightened emphasis on holistic review.

SUMMARY

The last two match cycles have been the most dynamic and unpredictable in decades. Out of the commotion, the otolaryngology community has an opportunity for a fresh start, combining insights from past literature with recent articles compiled for this review. Moving forward, it will be advantageous to approach residency selection as a well-executed quality improvement project, requiring continuous assessment and adjustment.

Systems-Level Reforms to the US Resident Selection Process: A Scoping Review

BACKGROUND

Calls to reform the US resident selection process are growing, given increasing competition and inefficiencies of the current system. Though numerous reforms have been proposed, they have not been comprehensively cataloged.

OBJECTIVE

This scoping review was conducted to characterize and categorize literature proposing systems-level reforms to the resident selection process.

METHODS

Following Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines, searches of Embase, MEDLINE, Scopus, and Web of Science databases were performed for references published from January 2005 to February 2020. Articles were included if they proposed reforms that were applicable or generalizable to all applicants, medical schools, or residency programs. An inductive approach to qualitative content analysis was used to generate codes and higher-order categories.

RESULTS

Of 10 407 unique references screened, 116 met our inclusion criteria. Qualitative analysis generated 34 codes that were grouped into 14 categories according to the broad stages of resident selection: application submission, application review, interviews, and the Match. The most commonly proposed reforms were implementation of an application cap (n = 28), creation of a standardized program database (n = 21), utilization of standardized letters of evaluation (n = 20), and pre-interview screening (n = 13).

CONCLUSIONS

This scoping review collated and categorized proposed reforms to the resident selection process, developing a common language and framework to facilitate national conversations and change.

Interviewing in the Wake of COVID-19: How Orthopaedic Residencies, Fellowships, and Applicants Should Prepare for Virtual Interviews

On May 7, 2020, the Coalition for Physician Accountability's released "Medical Students in the Class of 2021: Moving Across Institutions for Post Graduate Training," which comprises official recommendations on keeping programs and medical students safe during the upcoming match cycle with the challenges posed by COVID-19. In these recommendations, away rotations are discouraged, and all programs are compelled to commit to virtual interviews. Unlike employers and applicants in other industries, orthopaedic residency/fellowship programs and candidates seeking those positions have not routinely conducted virtual interviews. Without in-person interviews, applicants may perceive a limited ability to demonstrate their qualifications, judge program culture, and gauge ultimate program compatibility. Likewise, programs may perceive the inability to evaluate a candidate in real time, physically show program strengths, and ultimately judge applicant compatibility. Careful preparation and execution of a virtual interview can overcome these perceived limitations, whereas benefits, such as decreased cost for both programs and applicants, can make virtual interviews appealing. The purpose of this review was to help define a virtual interview, illustrate the benefits, and offer tips to both programs and applicants on how to prepare and perform optimally on an interview day.

When (Almost) Everyone is Above Average: A Critical Analysis of American Orthopaedic Association Committee of Residency Directors Standardized Letters of Recommendation

The American Orthopaedic Association introduced standardized letters of recommendations (SLORs) to improve on traditional letters of recommendations by “providing a global prospective on an applicant.” However, no study has defined the utilization of SLORs, the distribution of applicant ratings in SLORs, or the impact of sex, race, or degree of involvement between the letter writer and applicant on SLOR domain ratings.

Use of Standardized Letters of Recommendation for Orthopaedic Surgery Residency Applications: A Single-Institution Retrospective Review

BACKGROUND

Standardized letters of recommendation (SLORs) were introduced to provide a more objective method of evaluating applicants for orthopaedic surgery residency positions. We sought to establish whether there exists an association between the SLOR summative rank statement (SRS), which is a question that asks the letter-writing authors where they would rank a student relative to other applicants, and success in matching into orthopaedic surgery residency.

METHODS

We reviewed 858 applications to an orthopaedic surgery residency program from 2017 to 2018. Data on 9 assessment categories, SRSs, and written comments in the SLORs were extracted. The match success of applicants was determined by an internet search algorithm. Logistic regression was used to evaluate the association between the SRSs and match success. The Spearman correlation was performed between the SRSs and other variables.

RESULTS

Only 60% of all LORs were SLORs. With 24% of the SLORs, a supplemental letter was used. Median percentile rank ranged from 90% to 100% for the 9 categories in the SLORs. Recommendations of "high rank" or higher were found in 88% of the SRSs. The mean of the SLOR SRSs was associated with match success.

CONCLUSIONS

The mean of the SLOR SRSs was associated with match success. However, the SLOR is not uniformly used. Future efforts should be devoted to improving question design and validity in order to better discriminate among applicants, increase adherence to the rating scale, and quantify the strength of the written comments in the SLOR.

Selecting Residents for Predetermined Factors Identified and Thought to be Important for Work Performance and Satisfaction: A Methodology

OBJECTIVE

The medical profession seeks to hire and train individuals who consistently meet and/or exceed both job and cultural expectations. Resident selection is often not structured to meet this goal. The objective of this quality improvement project was to evaluate a classic unscripted interview process (OI) in conjunction with a structured, scripted interview process (SI) developed using an established hiring methodology from industry not yet utilized in health care. Qualitative questions we sought to answer: (1) Can SI be practically applied to the selection of residents? (2) Is there a significant difference in the relative position of applicants between the OI and SI rank lists? (3) Qualitatively, does SI help the evaluation/discussion of the affective domain?

METHODS

Design: Prospective qualitative comparison of OI versus SI.

SETTING

Dartmouth Hitchcock Medical Center, Lebanon, NH.

PARTICIPANTS

Applicants were assessed by OI and SI. SI factors were selected based on a job profile. Interview scripts were created from validated behavioral and attitudinal questions. Online assessments assessed 2 important attributes - adaptability and values. Rank lists were compared for relative rank position of applicants. Feedback from faculty was obtained.

RESULTS

Fifty-two applicants. Critical attributes were self-management, integrator-synthesizer, versatility, communication, and achievement. Absolute mean difference in rank/applicant was 9.8 (standard deviation 8.9, Range 0-36) positions. Comparing the top 20 candidates of each rank list, 40% of those applicants were only on one list. Faculty felt that applicants were given a greater opportunity to show "who they are."

CONCLUSIONS

In conjunction with OI, an industry proven methodology was practically applied to define and select for high performance for the authors' specific institution. Comparing OI and SI resulted in substantial differences in rank lists. This initiative seemed to provide a structure to evaluate values and motivations that are inherently difficult to assess. Faculty felt SI in conjunction with OI gave a greater chance for applicants to show "who they are."

The Match: To Thine Own Self Be True

The residency match process, culminating with the Match Day celebration, plays out in medical schools across the United States and Canada every year. The process may seem strange and mysterious for observers outside of medicine. The notion that each graduating student's employer for the next several years is first revealed to thousands of people, all at the same moment, through the opening of an envelope is surreal. The emotional reactions accompanying the process range from jubilance to deep disappointment. Much attention and care have been given to developing the algorithm underpinning the Match, and the process seems just: Optimization favors applicants over training programs. Witnessing students as they progress to their next stage of medical training is special for those involved in medical education. Faculty are filled with pride. But the process is far from perfect. The author of this Invited Commentary notes several concerns about the Match: the arduous process that students undergo to maximize their chances of success; the costs attendant to the travel and related expenses of multiple, geographically dispersed interviews; and the metrics that students and their medical schools use to judge the outcomes. The author worries that for some students, the "ideal" match may not be the one driven by their dreams and aspirations but, rather, by an amalgamation of those of many well-meaning friends, family members, and faculty. Medical students should seek advice and guidance, but the author hopes that, ultimately, students follow their own drumbeat and are true first to themselves.