Preference Signaling Pilot in the Urology Match: Outcomes and Perceptions

Preference Signaling in the Urology Match

PURPOSE

This manuscript summarizes the introduction, evolution and current outcomes for preference signaling in Urology as well as its use in other medical specialties. Overapplication plagues the residency recruitment process and PS has emerged as a process to improve the bottleneck of the interview selection process.

RECENT FINDINGS

PS has been shown to be associated with a higher likelihood of interview among many subspecialties. Applicants and programs report satisfaction with the process. Further solutions are needed to increase transparency of program information and selection criteria to applicants so that critical decisions on which programs an applicant should apply to can be data driven.

Preference signaling in orthopaedic surgery: applicant perspectives and opinions

CONTEXT

Orthopaedic surgery has become increasingly competitive over the years, with the COVID-19 pandemic creating additional challenges for applicants and programs. To promote an equitable match experience, the American Orthopaedic Association (AOA) introduced a formal preference signaling (PS) system into the 2022-2023 application cycle. PS allows applicants to indicate their heightened interest in specific programs, which improves the likelihood of receiving an interview and ultimately matching at their desired residency program.

OBJECTIVES

The objective of this anonymous survey is to assess applicants' opinions and perspectives toward PS in orthopaedic surgery prior to the 2022-2023 match results. Additionally, we sought to evaluate the signaling strategies being utilized by applicants.

METHODS

An anonymous 22-question survey was distributed to applicants of an orthopaedic surgery residency program (34.2 % response rate). Responses were collected after the application submission deadline but before the match lists and results were available. This survey included questions germane to demographics, signal utilization, signaling reasons and strategies, and opinions toward PS. Descriptive statistics were calculated utilizing R (version 4.2.1) and RStudio.

RESULTS

Most respondents (96.1 %) participated in PS, and 96.7 % utilized all 30 signals. Signaling encouraged 24.2 % of applicants to apply to fewer programs. In accordance with guidelines, 83.2 % of respondents signaled each away rotation program; however, only 53 % signaled their home program. Applicants commonly signaled 1-10 "reach" and "safety" programs each. Proximity to Family and Perceived Operative Experience were the most important reasons for signaling, whereas Program Prestige was the least. A program's social presence and virtual interview option did not influence many applicants' decisions for signaling. Most applicants believe that the COVID-19 pandemic and pass/fail licensure examinations influenced PS adoption. Sixty-seven of 149 respondents (45 %) claimed that applicants and programs benefit equally from PS, while 41 % believe programs benefit more. Nearly half (40.94 %) knew very little or nothing about PS.

CONCLUSIONS

During the inaugural introduction of PS in orthopaedic surgery, nearly every applicant utilized all 30 signals, prioritizing factors like family proximity and perceived operative experience over program prestige. This shift reflects the importance of geographic location and presumed training quality. Despite unfamiliarity toward PS, personalized signaling strategies were implemented, accompanied by a slight decrease in application volumes. The 30 allotted signals in orthopaedic surgery may serve as an informal application cap due to the necessity of signaling a program for an interview invite. However, improved educational efforts are needed to enhance the understanding and maximize the benefits of PS for both applicants and programs.

Preference Signaling and the Integrated Plastic Surgery Match: A National Survey Study

OBJECTIVE

Rooted in economics market strategies, preference signaling was introduced to the Plastic Surgery Common Application (PSCA) in 2022 for integrated plastic surgery residency applicants. This study surveyed program and applicant experience with preference signaling and assessed how preference signals influenced likelihood of interview invitations.

DESIGN

Two online surveys were designed and distributed to all program directors and 2022-2023 applicants to integrated plastic surgery. Opinions regarding the utility of preference signaling were solicited, and the influence of preference signals on likelihood of interview offers was assessed.

SETTING

All integrated plastic surgery programs.

PARTICIPANTS

All 88 program directors and 2022-2023 applicants to integrated plastic surgery.

RESULTS

A total of 45 programs and 99 applicants completed the survey (response rates, 54.2% and 34.2%, respectively). Overall, 79.6% of applicants and 68.9% of programs reported that preference signals were a useful addition to the application cycle. Programs reported that 41.4% of students who sent preference signals received interview offers, compared to 84.6% of home students, 64.8% of away rotators, and 7.1% of other applicants; overall, students who signaled were 5.8 times more likely to receive an interview offer compared to students who were not home students and did not rotate or signal. After multivariable adjustment, programs with higher Doximity rankings, numbers of away rotators, and numbers of integrated residents per year received more preference signals (all p < 0.05).

CONCLUSIONS

Applicants and programs report that preference signaling was a useful addition to the integrated plastic surgery application cycle. Sending preference signals resulted in a higher likelihood of interview offers among nonrotators. Preference signaling may be a useful tool to reduce congestion in the integrated plastic surgery application cycle.

Effect of United States Medical Licensing Examination Score Cutoffs on Recruitment of Underrepresented Applicants in the Urology Match

OBJECTIVE

To determine how the use of United States Medical Licensing Examination (USMLE) score cutoffs during the screening process of the Urology Residency Match Program may affect recruitment of applicants who are underrepresented in medicine (URM).

MATERIALS AND METHODS

Deidentified data from the Association of American Medical Colleges' (AAMC) Electronic Residency Application Service (ERAS) system was reviewed, representing all applicants to our institution's urology residency program from 2018 to 2022. We analyzed self-reported demographic variables including race/ethnicity, age, sex/gender, as well as USMLE Step 1 and Step 2 scores. Chi-square tests and ANOVA were used to determine the association between race/ethnicity and other sociodemographic factors and academic metrics. Applicants were stratified according to USMLE Step 1 cutoff scores and the distribution of applicants by race/ethnicity was assessed using a Gaussian nonlinear regression fit.

RESULTS

A total of 1258 applicants submitted applications to our program during the 5-year period, including 872 males (69.3%) and 386 females (30.7%). Most applicants were White (43.5%), followed by Asian (28.3%), Hispanic/Latino (11.7%), and Black (7.0%). There was an association between race/ethnicity and USMLE scores. Median USMLE Step 1 scores for White, Asian, Hispanic/Latino, and Black applicants were 242, 242, 237, and 232, respectively (P < .001). As cutoff score increases, percentage of URM applicants decreases.

CONCLUSION

The use of cutoffs based on USMLE scores disproportionately affects URM applicants. Transitioning from numeric scores to pass/fail may enhance holistic review processes and increase the representation of URM applicants offered interviews at urology residency programs.

Impact of Applicants' Characteristics and Geographic Connections to Residency Programs on Preference Signaling Outcomes in the Match

PURPOSE

To assess the impact of applicant and residency program characteristics on preference signaling outcomes in the Match during the first 2 years of implementation across 6 specialties.

METHOD

Data were obtained from the Texas Seeking Transparency in Application to Residency survey for applicants applying into otolaryngology during the 2020-2021 and 2021-2022 application cycles and into dermatology, internal medicine (categorical and preliminary year), general surgery, and urology during the 2021-2022 application cycle. The primary outcome was signal yield, defined as the number of interviews at signaled programs divided by the total number of signals sent. Associations with applicant-reported characteristics and geographic connections to residency programs were assessed using Wilcoxon rank sum testing, Spearman's rank correlation testing, and ordinary least squares regression.

RESULTS

1,749 applicants with preference signaling data were included from internal medicine (n = 884), general surgery (n = 291), otolaryngology (n = 217), dermatology (n = 147), urology (n = 124), and internal medicine preliminary year (n = 86). On average 60.9% (standard deviation 32.3%) of signals resulted in an interview (signal yield). There was a stepwise increase in signal yield with the percentage of signals sent to programs with a geographic connection (57.3% for no signals vs. 68.9% for 5 signals, P < .01). Signal yield was positively associated with applicant characteristics, such as United States Medical Licensing Exam Step 1 and 2 scores, honors society membership, and number of publications ( P < .01). Applicants reporting a lower class rank quartile were significantly more likely to have a higher percentage of their interviews come from signaled programs ( P < .01).

CONCLUSIONS

Signal yield is significantly associated with geographic connections to residency programs and applicant competitiveness based on traditional metrics. These findings can inform applicants, programs, and specialties as preference signaling grows.

Child Neurology and Neurodevelopmental Disabilities Program Directors' Opinions on Preference Signaling in the 2023-2024 National Resident Matching Program Match: A Survey

BACKGROUND

Changes in residency recruitment have significantly altered how programs and applicants evaluate each other including virtual interviews, discontinuation of the United States Medical Licensing Exam Step 2 Clinical Skills exam, and transition of United States Medical Licensing Exam Step 1 to pass-fail scoring. To improve program-applicant fit, the Electronic Residency Application Service introduced supplemental application features including geographic preference, program signaling, and the opportunity to highlight impactful and meaningful experiences. We sought to evaluate child neurology (CN) and neurodevelopmental disabilities (NDD) program director's (PD) opinions regarding these changes.

METHODS

A 10-question anonymous survey was sent to CN (n=75) and NDD (n=8) PDs. The questions centered on PDs' opinions regarding components of the supplemental application, having a standard application review period and in-person recruitment activities. Answer choices to the questions were all close-ended. Respondents could select questions to complete.

RESULTS

Thirty-eight CN residency PDs (49%) and 4 NDD residency PDs (50%) responded to the survey. Among CN PDs, there was strong support for use of the supplemental application questions and for the use of 3 program signals per applicant. Most PDs supported a standardized application review period prior to programs sending interview offers; however, there was no consensus on the appropriate length of time. Nearly half agreed with virtual-only interviews, and 62% agreed with the option of in-person second-look visits.

CONCLUSIONS

CN PDs generally support many of the recent or proposed changes to residency recruitment. The impact of these changes on recruitment will be a topic of future investigation.

The New Urology Match: How Recent Innovations Including Virtual Interviews and Preference Signaling Have Changed Match Outcomes

OBJECTIVES

To determine how recent changes in the urology match occurring from 2021 to 2023, including virtual interviews (VIs) and preference signals (PS), affected match outcomes.

METHODS

The American Urological Association (AUA) match data from 2021 to 2023 was compared to the 15 years prior. This was obtained from the AUA website and a previous study of public AUA match data. Self-reported applicant characteristics and outcomes from the Urology Residency Applicant Spreadsheet 2021-2023 were compared to the four years prior.

RESULTS

Between 2021 and 2023, residency programs offered 43 interviews each, compared to an average of 35 in the 15 years prior. Programs have been receiving more applications each year, from a low of 225 in 2019 to a peak of 347 in 2022. This resulted in an interview offer rate of 13% between 2021 and 2023, compared to 16% in the five years prior. Applicants applied to a mean of 88 programs in 2023, increasing each year since 40 in 2006. Applicants attended 12 interviews on average between 2021 and 2023, compared to 13 in the two years prior. Self-reported applicant data similarly demonstrated that, compared to the four years prior, applicants between 2021 and 2023 applied to more programs (81 vs. 70), had a lower interview offer rate (22% vs. 32%), and a higher interview acceptance rate (90% vs. 75%).

CONCLUSIONS

During the years with VIs, programs offered more interviews and applicants attended fewer on average, indicating a larger applicant pool was interviewed. Despite the introduction of PS, applicants applied to more programs in 2022 and 2023 than ever before.

The impact of virtual interviews on the Fellowship Council application and main match process

BACKGROUND

The COVID-19 pandemic led the Fellowship Council (FC) to transition rapidly from in-person to virtual interviews. We investigated the impact of this transition on the FC application and main match process.

METHODS

Five years (2018-2022) of deidentified FC applicant, program, and match rank data were used to assess differences between in-person (2018-2019) and virtual interview (2021-2022) cycles. Data are expressed as mean ± SD and one-way and two-way MANOVA tests were applied.

RESULTS

Trainees applied to an average of 30.4 ± 24.3 programs and ranked an average of 10.7 ± 9.7 programs with a 57% match rate and average rank position of 3.6 ± 3.3. Fellowship programs received an average of 64.9 ± 28.6 applications and ranked an average of 15.4 ± 8.8 applicants with a 95% match rate and average applicant rank position of 3.0 ± 3.4. Applicants who interviewed virtually applied to a greater number of programs (32.7 vs. 27.0; p < 0.001) and ranked a greater number of programs (11.5 vs. 10.0; p = 0.004) with no difference in match rates (58% vs. 55%, p = 0.291). Among matched applicants, there was a significant difference in average rank position (3.20 vs. 4.30, p < 0.001), favoring the in-person cohort. Fellowship programs had more applicants per program (69.2 vs. 57.8; p < 0.001) and ranked more applicants (17.4 vs. 13.3; p < 0.001) during the virtual interview cycles. No difference in either match rates (93% vs. 96%, p = 0.178) or applicant rank position (3.09 vs. 2.93, p = 0.561) was seen between in-person and virtual application cycles.

CONCLUSION

Virtual interviews were associated with an increased number of applications for fellowship and applicants ranked by programs but did not impact match rates of either group. Rank match position declined somewhat for applicants but not for fellowship programs. Virtual interviews offer more opportunities for applicants and a greater number of candidates for fellowship programs with only a slight decrement in fellow match rank position.

Diversity Attracts Diversity: 2023 AUA Match Results

OBJECTIVE

To assess the rationale behind the choice of programs for preference signaling (PS) and subinternships by urology applicants in the 2023 cycle.

METHODS

We emailed an anonymous, multiple-choice survey to the 403 prospective candidates who applied to our institution for the 2023 Urology Residency Match.

RESULTS

121 applicants (30.0%) responded to the survey. 81.8% were in favor of the continuation of PS, with 4.1% against it. When choosing where to PS or subinternship, geographic location and perceived quality of clinical training were the highest-ranked criteria. Racial/ethnic diversity of the residents influenced PS or subinternship choice for 77.8% of Black, 72.7% of Asian, 57.1% of Latinos, and 46.5% of White respondents (P < .05 for Black and Asian vs White). Institutional statements on diversity influenced PS or subinternship choice for 88.9% of Black, 55% of Asian, 64.3% of Latino, and 25.4% of White respondents (P < .05 for Black, Asian and Latino vs White). Females had an increased likelihood of PS or choosing subinternship programs with gender diversity of residents (81.6% vs 33.8, P = .002). A program with PS was associated with a 2.74 increase in likelihood of obtaining an interview compared to programs that were not PS. Of 107 matched applicants, 47.5% matched at a program where they completed a subinternship, and 25.7% matched at a signaled program.

CONCLUSION

PS plays a major role in the match process. When choosing where to PS, applicants prioritize geographic location and perceived quality of training. Diversity of residents plays a major role in recruiting diverse applicants.

Attending Perceptions on the Use of Preference Signaling in the Ophthalmology Residency Application Process

Objective  This article characterizes perspectives of ophthalmologists involved in the residency selection process regarding the potential impact of preference signaling on the ophthalmology residency match. Methods  An anonymous online questionnaire generated from SurveyMonkey was approved by the Association of University Professors of Ophthalmology (AUPO) Data Resource Committee for distribution to 391 individuals from the AUPO Departmental Chairs, Program Directors, and Directors of Medical Student Education email listservs in August 2022. Results  A total of 96 (24.6%) ophthalmology faculty completed the questionnaire. The majority ( n  = 76, 79.2%) agreed or strongly agreed that preference signaling should be implemented in the ophthalmology residency application system. Most respondents agreed or strongly agreed that preference signaling will allow for more holistic reviews of applications ( n  = 55, 57.3%), agreed or strongly agreed that it will benefit applicants who do not have connections to home programs or faculty that can reach out to desired programs ( n  = 81, 84.4%), and agreed or strongly agreed that it will improve the distribution of interviews to applicants ( n  = 76, 79.2%). Participants agreed or strongly agreed that applicants who have signaled interest in their program will receive preference when offering interviews ( n  = 59, 61.5%), and those signals will be used as a tiebreaker for similar applications ( n  = 75, 78.1%). The majority of participants believed that the ideal number of preference signals' applicants should be given three to four signals ( n  = 35, 36.0%) or five to six signals ( n  = 29, 30.2%). Conclusion  A majority of ophthalmology faculty surveyed support the integration of preference signaling into the ophthalmology residency match.