Simulation as Soothsayer: Simulated Surgical Skills MMIs During Residency Interviews are Associated With First Year Residency Performance

Trends in Urology Residency Applications: Results from The Society of Academic Urologists Program Director Survey from 2022 to 2024

OBJECTIVES

To provide a cross-sectional view of the current opinions surrounding the urology match by analyzing data from the annual Society of Academic Urologists Program Director Surveys conducted between 2022 and 2024.

METHODS

Data collected through surveys distributed to all urology program directors consisting of questions covering program demographics, applicant selection criteria, preference signals, virtual interviews, and other relevant topics.

RESULTS

89, 90, and 89 program directors participated in the surveys for the years 2022, 2023, and 2024, respectively. The analysis revealed shifts in application review criteria, with increased emphasis on subinternship performance, letters of recommendation, and USMLE Step 2 Clinical Knowledge scores. Preference signals were widely adopted and viewed positively by program directors, with a majority supporting the continuation in a large-volume format. Virtual interviews were met with mixed sentiments, with concerns raised about their effectiveness in assessing candidates and replicating in-person experiences. However, they were recognized for their potential to level the playing field.

CONCLUSIONS

This study underscores the evolving landscape of urology resident match. The adoption of virtual interviews, preference signals, and other changes co-occurring in medical education have altered the means through which applicants have been historically assessed. The findings highlight the need for ongoing feedback and transparency to ensure equitable practices for both applicants and residency programs.

Emotional intelligence and LEGO-based communication assessments as indicators of peer evaluations

INTRODUCTION

Much attention and effort are devoted to general surgery residency applicant interview strategies and ranking. However, few interview strategies are positively associated with applicant communication performance. The purpose of this study was to assess the potential of a LEGO-based communication assessment and a standardized patient-based emotional intelligence assessment to serve as an indicator for communication and interpersonal skills among peers in residency.

METHODS

We examined general surgery residents who interviewed virtually at our institution in 2021-2022, then matriculated in 2022-2023. Residents' LEGO-based communication and emotional intelligence interview assessments were evaluated for associations with peer evaluations in intern year. A regression analysis was conducted testing the association between each assessment and peer-evaluation metrics, controlling for resident track (preliminary, categorical).

RESULTS

Performance on LEGO-based communication assessments was significantly associated with peer evaluation of cross-cover performance (B = 0.09, standard error 0.044, 95% confidence interval 0.001-0.187, P = .048), whereas emotional intelligence scores had no significant association (P = .155). In contrast, performance on the LEGO-based communication assessments were associated with peer evaluations for desirability as a coworker (B = 0.098, standard error 0.038, 95% confidence interval 0.017-0.178, P = .021), whereas emotional intelligence scores were negatively associated with desirability as a coworker (B = -0.255, standard error 0.107, 95% confidence interval -0.482 to -0.029], P = .029).

CONCLUSION

Although LEGO-based communication assessments before residency are associated with better resident peer evaluations 1 year into residency, our emotional intelligence applicant assessment during recruitment did not provide clear insight into resident performance. Future research should consider the ability of standardized assessments to predict performance.

Performance in Irish Selection and Future Performance in Surgical Training

Importance

Selection processes for surgical training should aim to identify candidates who will become competent independent practitioners and should aspire to high standards of reliability and validity.

Objective

To determine the association between measured candidate factors at the time of an Irish selection and assessment outcomes in surgical training, examined via rate of progression to Higher Specialist Training (HST), attrition rates, and performance as assessed through a multimodal framework of workplace-based and simulation-based assessments.

Design, Setting, and Participants

This retrospective observational cohort study included data from all successful applicants to the Royal College of Surgeons in Ireland (RCSI) national Core Surgical Training (CST) program. Participants included all trainees recruited to dedicated postgraduate surgical training from 2016 to 2020. These data were analyzed from July 11, 2016, through July 10, 2022.

Exposures

Selection decisions were based on a composite score that was derived from technical aptitude assessments, undergraduate academic performance, and a 4-station multiple mini-interview.

Main outcomes and measures

Assessment data, attrition rates, and rates of progression to HST were recorded for each trainee. CST performance was assessed using workplace-based and simulation-based technical and nontechnical skill assessments. Potential associations between selection and assessment measures were explored using Pearson correlation, logistic regression, and multiple linear-regression analyses.

Results

Data were available for 303 trainees. Composite scores were positively associated with progression to HST (odds ratio [OR], 1.09; 95% CI, 1.05-1.13). There was a weak positive correlation, ranging from 0.23 to 0.34, between scores and performance across all CST assessments. Multivariable linear regression analysis showed technical aptitude scores at application were associated with future operative performance assessment scores, both in the workplace (β = 0.31; 95% CI, 0.14-0.48) and simulated environments (β = 0.57; 95% CI, 0.33-0.81). There was evidence that the interpersonal skills interview station was associated with future performance in simulated communication skill assessments (β = 0.55; 95% CI, 0.22-0.87).

Conclusions and Relevance

In this study, performance at the time of Irish national selection, measured across technical and nontechnical domains in a multimodal fashion, was associated with future performance in the workplace and in simulated environments. Future studies will be required to explore the consequential validity of selection, including potential unintended effects of selection and ranking on candidate performance.