Optimizing the residency application process: insights from neurological surgery during the pandemic virtual application cycle

Virtual Interviews in Neurosurgery Resident Selection: A Follow-up Report

BACKGROUND

The 2021 U.S. neurosurgery residency match interviews were conducted virtually; we surveyed applicants and interviewers to determine satisfaction with that virtual interview process. Subsequently, we conducted a follow-up survey to determine satisfaction with the virtual interview process after the residency match for faculty interviewers and 2022 interns.

METHODS

A 22-question online faculty survey was sent to 116 U.S. neurosurgery training programs. A 26-question survey was sent to these programs for distribution to their intern classes. Data were analyzed quantitatively, including mean Likert score. Open-ended questionnaire responses were reviewed to identify themes.

RESULTS

Overall, 32 interns representing 20 programs and 73 faculty representing 62 programs responded. Most respondents agreed that virtual interviews were more convenient (86% faculty, 90% interns) and cost-effective (100% interns) than in-person interviews. Faculty respondents agreed or strongly agreed that virtual interviews were effective to evaluate applicants' competence as residents (44%); fewer faculty agreed or strongly agreed that virtual interviews were an effective way to evaluate candidates' fit in the program (27%). For interns, 44% agreed or strongly agreed that virtual interviews gave them a good sense of the program faculty; 75% agreed or strongly agreed they were satisfied with the process related to where they matched.

CONCLUSIONS

Virtual interviews offer an advantage in terms of time and cost but potentially at the expense of adequate faculty assessment of candidates' "fit" within a program's culture. Despite this, interns undergoing an all-virtual interview process report high satisfaction with the results of the residency match.

Medical School Interview Preference in the Post-COVID-19 Era: A Single Institution Experience

BACKGROUND

The coronavirus disease 2019 (COVID-19) pandemic presented unforeseen obstacles to prospective medical students such as Medical College of Admission Test (MCAT) scheduling postponements and technical challenges during virtual interviews. Students were also faced with difficult decisions post-submission such as having to choose a program without ever visiting a school in person. The primary objective of the present study is to assess the changes in medical school interview preferences and experiences in the post-COVID-19 era.

METHODS

A retrospective survey of the class of 2024 (in-person interview group) and class of 2025 (virtual interview group) at an allopathic medical school was conducted in the Fall of 2021 via the Qualtrics XM online survey software (Qualtrics, Provo, UT, USA).

RESULTS

There were 195 survey respondents: 77 students from the in-person interview group and 89 students from the virtual group. More students in the virtual cohort had to reschedule their MCAT compared to the in-person cohort (56.1% versus 14.3%; p<0.001). The in-person group had higher travel-related expenses (>$500) compared to the group who interviewed virtually (65.1% versus 2.4%; p<0.001). More students from the in-person cohort preferred the in-person interview format compared to the virtual cohort (85.7% versus 22.5%; p<0.001). Lastly, 87% of the in-person group and 24.7% of the virtual group felt they were able to gather a clear impression of the atmosphere and culture of a school from the interview trail alone (p<0.001).

CONCLUSION

Matriculated medical students at an allopathic medical school who applied during the COVID-19 pandemic had more pre-application hurdles when compared to the cohort who applied just prior to the pandemic. Students who primarily had virtual interviews during the pandemic had less travel-related costs but felt more limited in their experience of a school's culture and ability to establish rapport with interviewers. Despite this, however, the virtual group still expressed a preference for virtual interviews.

A cost analysis of medical students applying to neurological surgery residency: An analysis of the Texas STAR database

INTRODUCTION

Medical Students applying to neurosurgery residency programs incur substantial costs associated with interviews, away rotations, and application fees. However, few studies have compared expenses prior to and during the COVID-19 pandemic. This study evaluates the financial impact of COVID-19 on the neurosurgery residency application and identifies strategies that may alleviate the financial burden of prospective neurosurgery residents.

METHODS

The TEXAS STAR database was surveyed for applicants of neurosurgical residency programs during the COVID-19 pandemic (2021) and post-pandemic (2022). 66 applicants for the 2021 application cycle and 50 applicants for the 2022 application cycle completed the survey. We compared application fees, away rotations cost, interview cost, and total expenses as reported by the neurosurgery applicants of the 2021 and 2022 application cycle. A Shapiro-Wilk test was used to test for data normality, and a Mann-Whitney U-Test was used to compare costs during the 2021 and 2022 neurosurgery application cycle.

RESULTS

There was a statistically significant reduction in total expenses in 2021 vs 2022 ($3,934 vs $9,860). Interview and away rotation expenses decreased in 2021 vs 2022 (interview expenses $786 vs $4511, away rotation $1,083 vs $3,000, p < 0.001). Application fee expenses were not different between 2021 and 2022. The greatest reduction in application cost ($11,908) was seen in the South for 2021.

CONCLUSIONS

The COVID-19 pandemic significantly reduced total fees associated with the neurosurgical residency application. Virtual platforms in place of in-person interviews could lessen the financial burden on applicants and alleviate socioeconomic barriers in the neurosurgical application process after COVID-19.

A Simulation of the Effect of Interview Caps on the National Resident Matching Program Match in Otolaryngology

Concerns regarding interview hoarding in the Match process have led to calls for a cap on the number of interviews an applicant can accept. However, no study has examined the effects of interview caps on applicants or a specialty. The authors created a simulation otolaryngology Match to analyze the effects of interview caps. The simulation included 120 residency programs, 360 positions, and 570 applicants (each assigned a competitiveness weighted value). Programs created interview lists (subject to 4 different interview cap conditions: no, 16, 12, and 8 cap) using weighted random selection and the competitiveness weighted values. Programs then selected 3 interviewees to fill their positions via weighted random selection using the competitiveness weighted values. Data analysis explored the effects of interview caps on individual applicants' probability of matching, the probability of matching with a certain number of interviews, and the average matched applicant competitiveness rank using one-way analysis of variance. The introduction of interview caps caused interviews to be distributed more equally among applicants. The 12-interview cap caused the applicants in the 100th competitiveness percentile to see their probability of matching fall from 99.3% to 67.2% ( P < .001). Applicants with ≥ 12 interviews had a probability of matching of 94.4% when there was no cap, compared to 76.9% with a 12-interview cap ( P < .001). The average competitiveness rank of matched applicants fell from 191 with no cap to 245 with a 12-interview cap ( P < .001). Interview caps in the otolaryngology Match may create major distortions in the probability of matching, leading to much lower chances of matching for more competitive applicants, lower probabilities of matching with certain numbers of interviews, and a decreased average competitiveness of matched applicants.

Health worker education during the COVID-19 pandemic: global disruption, responses and lessons for the future-a systematic review and meta-analysis

BACKGROUND

This systematic review and meta-analysis identified early evidence quantifying the disruption to the education of health workers by the COVID-19 pandemic, ensuing policy responses and their outcomes.

METHODS

Following a pre-registered protocol and PRISMA/AMSTAR-2 guidelines, we systematically screened MEDLINE, EMBASE, Web of Science, CENTRAL, clinicaltrials.gov and Google Scholar from January 2020 to July 2022. We pooled proportion estimates via random-effects meta-analyses and explored subgroup differences by gender, occupational group, training stage, WHO regions/continents, and study end-year. We assessed risk of bias (Newcastle-Ottawa scale for observational studies, RοB2 for randomized controlled trials [RCT]) and rated evidence certainty using GRADE.

RESULTS

Of the 171 489 publications screened, 2 249 were eligible, incorporating 2 212 observational studies and 37 RCTs, representing feedback from 1 109 818 learners and 22 204 faculty. The sample mostly consisted of undergraduates, medical doctors, and studies from institutions in Asia. Perceived training disruption was estimated at 71.1% (95% confidence interval 67.9-74.2) and learner redeployment at 29.2% (25.3-33.2). About one in three learners screened positive for anxiety (32.3%, 28.5-36.2), depression (32.0%, 27.9-36.2), burnout (38.8%, 33.4-44.3) or insomnia (30.9%, 20.8-41.9). Policy responses included shifting to online learning, innovations in assessment, COVID-19-specific courses, volunteerism, and measures for learner safety. For outcomes of policy responses, most of the literature related to perceptions and preferences. More than two-thirds of learners (75.9%, 74.2-77.7) were satisfied with online learning (postgraduates more than undergraduates), while faculty satisfaction rate was slightly lower (71.8%, 66.7-76.7). Learners preferred an in-person component: blended learning 56.0% (51.2-60.7), face-to-face 48.8% (45.4-52.1), and online-only 32.0% (29.3-34.8). They supported continuation of the virtual format as part of a blended system (68.1%, 64.6-71.5). Subgroup differences provided valuable insights despite not resolving the considerable heterogeneity. All outcomes were assessed as very-low-certainty evidence.

CONCLUSION

The COVID-19 pandemic has severely disrupted health worker education, inflicting a substantial mental health burden on learners. Its impacts on career choices, volunteerism, pedagogical approaches and mental health of learners have implications for educational design, measures to protect and support learners, faculty and health workers, and workforce planning. Online learning may achieve learner satisfaction as part of a short-term solution or integrated into a blended model in the post-pandemic future.

The US Residency Selection Process After the United States Medical Licensing Examination Step 1 Pass/Fail Change: Overview for Applicants and Educators

The United States Medical Licensing Examination (USMLE) Step 1, arguably the most significant assessment in the USMLE examination series, changed from a 3-digit score to a pass/fail outcome in January 2022. Given the rapidly evolving body of literature on this subject, this paper aims to provide a comprehensive review of the historical context and impact of this change on various stakeholders involved in residency selection. For this, relevant keyword-based searches were performed in PubMed, Google Scholar, and Scopus to identify relevant literature. Given the unique history of USMLE Step 1 in the US residency selection process and the score's correlation with future performance in board-certifying examinations in different specialties, this scoring change is predicted to significantly impact US Doctor of Medicine students, US Doctor of Osteopathic Medicine students, international medical graduates, and residency program directors, among others. The significance and the rationale of the pass/fail change along with the implications for both residency applicants and educators are also summarized in this paper. Although medical programs, academic institutions, and residency organizing bodies across the United States have swiftly stepped up to ensure a seamless transition and have attempted to ensure equity for all, the conversion process carries considerable uncertainty for residency applicants. For educators, the increasing number of applications conflicts with holistic application screening, leading to the expected greater use of objective measures, with USMLE Step 2 Clinical Knowledge likely becoming the preferred screening tool in lieu of Step 1.

Virtual residency recruitment: future directions in the new era

INTRODUCTION

The COVID-19 pandemic led to many changes in healthcare including graduate medical education (GME). Residency and fellowship programs halted in-person recruitment and pivoted to virtual models. Residency selection and recruitment were practices ripe for redesign, as they relied on in-person interviewing as the major point of contact prior to match list creation. In this commentary, we review the state of virtual interviewing and propose a future state where virtual interactions are commonplace and integrated into a comprehensive recruitment process.

DISCUSSION

Virtual recruitment has led to a reduction of expenses, improved time efficiency for all parties and a reduced carbon footprint. Residency match outcomes have not changed substantially with the advent of virtual interviewing. Hybrid approaches, including virtual and in-person options have significant drawbacks and pitfalls which may limit adoption. Given the upheaval in GME recruitment caused by the pandemic, and the limitations of current methods for candidate assessment and interactions with programs, further innovation is needed to achieve an optimal state for all stakeholders. Multiple technology innovations are on the horizon which may improve the ability to interact virtually. Adoption of new technology along with expanding the timeline for residency recruitment may further optimize the process for both applicants and programs.

CONCLUSIONS

The GME community was able to adopt technology for the recruitment interview rapidly due to the pandemic. As more opportunities for technology-based interactions grow, the opportunity exists to reimagine recruitment beyond the interview. While resources are constrained, some of the efficiencies gained by adopting virtual interviewing can be leveraged to expand the interactions between programs and applicants. Incorporation of in-person interaction may still be needed. Models will need to be developed to build upon the best characteristics of the virtual and in-person environments to optimize GME recruitment.KEY MESSAGES:Virtual communication methods have substantially changed residency recruitment during the COVID -19 pandemic.COVID -19 related changes in residency recruitment, including wide adoption of virtual methods, should be maintained and strengthened.Efforts should be made to advance the gains in residency recruitment strategy during the pandemic by use of technologies that expand virtual interactions beyond the interview.

Virtual interviewing for graduate medical education recruitment and selection: A BEME systematic review: BEME Guide No. 80

BACKGROUND

The COVID-19 pandemic caused graduate medical education (GME) programs to pivot to virtual interviews (VIs) for recruitment and selection. This systematic review synthesizes the rapidly expanding evidence base on VIs, providing insights into preferred formats, strengths, and weaknesses.

METHODS

PubMed/MEDLINE, Scopus, ERIC, PsycINFO, MedEdPublish, and Google Scholar were searched from 1 January 2012 to 21 February 2022. Two authors independently screened titles, abstracts, full texts, performed data extraction, and assessed risk of bias using the Medical Education Research Quality Instrument. Findings were reported according to Best Evidence in Medical Education guidance.

RESULTS

One hundred ten studies were included. The majority (97%) were from North America. Fourteen were conducted before COVID-19 and 96 during the pandemic. Studies involved both medical students applying to residencies (61%) and residents applying to fellowships (39%). Surgical specialties were more represented than other specialties. Applicants preferred VI days that lasted 4-6 h, with three to five individual interviews (15-20 min each), with virtual tours and opportunities to connect with current faculty and trainees. Satisfaction with VIs was high, though both applicants and programs found VIs inferior to in-person interviews for assessing 'fit.' Confidence in ranking applicants and programs was decreased. Stakeholders universally noted significant cost and time savings with VIs, as well as equity gains and reduced carbon footprint due to eliminating travel.

CONCLUSIONS

The use of VIs for GME recruitment and selection has accelerated rapidly. The findings of this review offer early insights that can guide future practice, policy, and research.