Interview Techniques Utilized in Radiology Resident Selection-A Survey of the APDR

Need for a More Objective, Inclusive, and Equitable Selection Process for Gastroenterology Fellowships

Diseases related to the digestive system account for a significant proportion of the diseases burden in the United States and result in 36.8 million ambulatory visits, 3.8 million hospital admissions, and 22.2 million gastrointestinal endoscopies. To meet the challenge that this quantum of gastroenterological disorders poses, we are obligated to select and train competent gastroenterologists. Admission into a Gastroenterology (GI) fellowship program is highly selective. In 2023, only 62.7% of candidates who applied were successful in matching into a fellowship program, making it even more competitive than a cardiology fellowship (match rate of 68.4%). Therefore, it is imperative that we ensure that the selection process is fair and transparent. Additionally, we need to be socially more responsible by emphasizing diversity and inclusivity to produce gastroenterologists who reflect the changing society we live in. An analysis of current practices indicates that the process of selection is not standardized and is more subjective than objective. This review is an attempt to identify deficiencies that can be rectified by the introduction of a standardized system that includes structured interviews, Standard Letters of Recommendation (SLOR), and objective scoring protocols-all of which would make the process of selection more equitable, diverse, and inclusive. Newer methods like Casper exam, Psychometric testing, and Preference Signaling can also be explored to this end.

Blinding of Residency Applications in Medical Physics: Promises and Pitfalls

Purpose

The feasibility of blinding applications for a medical physics residency program has yet to be demonstrated in the literature. We explore the application of an automated approach with human review and intervention to blind applications during the annual medical physics residency review cycle.

Methods and Materials

Applications were blinded using an automated process and used for the first phase of residency review in the program. We retrospectively compared self-reported demographic and gender data with blinded and nonblinded cohorts from 2 sequential years of review from a medical physics residency program. Demographic data were analyzed comparing applicants with candidates selected to move to the next phase of the review process. Interrater agreement was also evaluated from the applicant reviewers.

Results

We show the feasibility of blinding applications for a medical physics residency program. We observed no more than a 3% difference between the gender selection within the first phase of application review but greater differences when examining race and ethnicity between the 2 methods. The greatest difference was shown to be between Asian and White candidates, where there are statistical differences in the scores in the rubric categories of essay and overall impression.

Conclusions

We suggest that each training program critically evaluate its selection criteria for potential sources of bias within the review process. We recommend further critical investigation of processes to promote equity and inclusion to ensure the methods and outcomes are aligned with the mission of the program. Finally, we recommend that the common application provide an option for blinding applications at the source so this can be an option to facilitate efforts for evaluating unconscious bias in the review process.

Views of Diagnostic Radiology Residency Program Directors Regarding Methods to Increase Female and Under-Represented in Medicine Residents: A Cross-sectional Study

RATIONALE AND OBJECTIVES

Diagnostic radiology remains one of the least diverse medical specialties. Recent reports have found that the number of female and under-represented in medicine (URiM) residents have not increased despite efforts to increase representation over the last decade. Given the critical role of residency program directors in selecting diverse applicants, this study was performed to identify which strategies were most preferred to increase the number of female and/or URiM residents by directors of diagnostic radiology residency training programs.

MATERIALS AND METHODS

This was an anonymous, cross-sectional study of diagnostic radiology residency program directors that included a survey about program characteristics, demographics, and strategies to increase the number of female and/or URiM residents.

RESULTS

The questionnaire was submitted to 181 potential participants with a 19.9% response rate. The most preferred strategies to increase diversity involved directly recruiting medical students, promoting mentorship, increasing the number of diverse teaching faculty, and unconscious bias training. The least supported strategies included deemphasizing exam scores, accepting more international graduates, accepting a minimum number of female and/or URiM applicants, and de-identifying applications. Female and/or URiM program directors indicated a statistically significant preference for medical student recruitment and providing an opportunity to discuss workplace issues for female and/or URiM trainees (p < 0.05).

CONCLUSION

Diagnostic radiology residency program directors endorsed a wide variety of strategies to increase diversity. Recruitment of female and/or URiM medical students and promoting the number of diverse faculty members and mentorship of trainees by these faculty appear to be the most preferred strategies to increase female and/or URiM residents. Female and/or URiM program directors placed a greater importance on recruiting diverse applicants and supporting safe discussion of workplace issues faced by female and/or URiM radiology residents.

RESIDENTS AS INTERVIEWERS: A COMPARISON OF RESIDENT AND FACULTY EVALUATIONS OF EMERGENCY MEDICINE RESIDENCY APPLICANTS

BACKGROUND

Interviews are an integral component of the residency selection process. Many programs use current residents as interviewers in addition to faculty. Although the reliability of interview scores between faculty members has been examined, little is known about the reliability between resident and faculty interviewers.

OBJECTIVE

This study evaluates the reliability of residents as interviewers compared with faculty.

METHODS

A retrospective analysis of interview scores from the 2020-2021 application cycle was performed at an emergency medicine (EM) residency program. Each applicant participated in five separate one-on-one interviews led by four faculty members and one senior resident. Interviewers assigned applicants a score from 0 to 10. Consistency between interviewers was measured using the intraclass correlation coefficient (ICC). Generalizability theory was used to measure variance components including applicant, interviewer, and rater type (resident vs. faculty) and their impact on scoring.

RESULTS

There were 250 applicants interviewed by 16 faculty members and 7 senior residents during the application cycle. The mean (SD) interview score given by resident interviewers was 7.10 (1.53) and the mean (SD) score given by faculty was 7.07 (1.69). There was no statistically significant difference between the pooled scores (p = 0.97). Reliability between interviewers was good to excellent (ICC = 0.90; 95% CI 0.88-0.92). The generalizability study showed most score variance was attributable to applicant characteristics and only 0.6% was attributable to interviewer or rater type (resident vs. faculty).

CONCLUSIONS

There was strong concordance between faculty and resident interview scores indicating reliability of EM resident scoring compared to faculty.

Best Practices for Reducing Bias in the Interview Process

PURPOSE OF REVIEW

Objective measures of residency applicants do not correlate to success within residency. While industry and business utilize standardized interviews with blinding and structured questions, residency programs have yet to uniformly incorporate these techniques. This review focuses on an in-depth evaluation of these practices and how they impact interview formatting and resident selection.

RECENT FINDINGS

Structured interviews use standardized questions that are behaviorally or situationally anchored. This requires careful creation of a scoring rubric and interviewer training, ultimately leading to improved interrater agreements and biases as compared to traditional interviews. Blinded interviews eliminate even further biases, such as halo, horn, and affinity bias. This has also been seen in using multiple interviewers, such as in the multiple mini-interview format, which also contributes to increased diversity in programs. These structured formats can be adopted to the virtual interviews as well. There is growing literature that using structured interviews reduces bias, increases diversity, and recruits successful residents. Further research to measure the extent of incorporating this method into residency interviews will be needed in the future.

Impact of COVID-19 on Radiology Residency Selection Process: A Survey of Radiology Residency Programs in the US

Rationale and Objectives

Virtual residency interviews have become crucial in maintaining CDC-recommended precautionary measures throughout the ongoing COVID-era. However, scant literature exists regarding the direct impact the pandemic has had on the radiology program selection process, including preferred modalities of residency interviews and social media utilization. This survey aimed to understand how radiology programs adapted to changes in the 2020–2021 virtual application cycle.

Materials and Methods

A 32-question survey was distributed to 194 residency programs between July and August of 2021. Data were analyzed primarily by using descriptive statistics and Paired Student's T-testing.

Results

A total of 31 programs completed the survey, yielding a response rate of 16%. When queried about the perceived success of virtual interviews during the 2020–2021 application cycle, 21 programs (68%) said the new interview format was very successful. Twenty-seven of the programs (87%) believed they were able to adequately gauge applicants through virtual interviews. However, when surveyed about personal preferences for interviews, the responses were more evenly distributed between virtual (11/31, 35%) and in-person (14/31, 45%). Pre-COVID, the top three criteria programs used to rank candidates were USMLE Step 1 score (20/31, 65%), letters of recommendation (17/31, 55%), and MSPE (12/31, 39%). Within the virtual, COVID-19 era, these criteria remained statistically unchanged (p = 0.22): USMLE Step 1 score (21/31, 68%), letters of recommendation (17/31, 55%), and MSPE (14/31, 45%). About half of programs who had not previously utilized social media (12 of 23, 52%) created accounts, mostly via Twitter or Instagram.

Conclusion

The primary findings indicate that programs perceived success with virtually interviewing and ranking applicants, the criteria to rank applicants remain unchanged despite the virtual environment, and programs’ social media utilization increased.

Impact of blinding interviewers to written applications on ranking of Gynecologic Oncology fellowship applicants from groups underrepresented in medicine

•

Despite attempts to improve diversity in healthcare, many populations continue to be underrepresented in medicine (URM)

•

Blinded interviews, which de-emphasize the written application, may reduce bias in application review.

•

We found that blinded interviewers were more likely to rank URM applicants more highly.

•

Other techniques to limit bias, such as standardized questions and implicit bias training, should be considered.

Biases in application review may limit access of applicants who are underrepresented in medicine (URM) to graduate medical training opportunities. We aimed to evaluate the association between blinding interviewers to written applications and final ranking of all applicants and URM applicants for Gynecologic Oncology fellowship. During 2020 virtual Gynecologic Oncology fellowship interviews, we blinded one group of interviewers to written applications, including self-reported URM status. Interviewers visually interacted with the applicants but did not review their application. Interviewers submitted independent rank lists. We compared pooled rankings of blinded and non-blinded interviewers for all applicants and for URM applicants using appropriate bivariate statistics.

We received 94 applications for two positions through the National Resident Matching Program, of which 18 (19%) self-identified as URM. We invited 40 applicants to interview and interviewed 30 applicants over six sessions. Ten interviewees (33%) self-identified as URM. Of 12 or 13 faculty interviewers during each interview session, 3 or 4 were blinded to the written application. There was no statistically significant difference in rank order when comparing blinded to non-blinded interviewers overall. However, blinded interviewers ranked URM applicants higher than non-blinded interviewers (p = 0.04). Blinding of written application metrics may allow for higher ranking of URM individuals.

The Evolving Status of Fellowships and Mini-Fellowships in Diagnostic Radiology: A Survey of Program Directors and Chief Residents

RATIONALE AND OBJECTIVES

Recent changes in radiology fellowships include musculoskeletal radiology adopting a match system, interventional radiology transitioning away from diagnostic radiology to offer direct-entry programs, and a common fellowship application timeline created by the Society for Chairs of Academic Radiology Departments (SCARD). The concept of mini-fellowships has also emerged with the elimination of the oral American Board of Radiology examinations that had been administered in the final year of residency training prior to 2014. This paper seeks to assess the opinions of fellowship program directors, residency program directors, and chief residents regarding these recent changes.

MATERIALS AND METHODS

This is a cross-sectional study using a web-based survey posed to fellowship program directors, residency program directors, and chief residents in 2020. Questions sought to explore current attitudes toward the following topics: (1) a common fellowship application timeline; (2) a common fellowship match; and (3) the status of mini-fellowships in diagnostic radiology. In addition, the number of fellowship positions for each subspecialty was estimated using subspecialty society directories, Accreditation Council for Graduate Medical Education (ACGME) data, and individual program websites.

RESULTS

Deidentified responses were collected electronically and aggregated. The three respondent groups preferred a common fellowship application timeline at rates of 67% among fellowship program directors, 80% residency program directors, and 74% residents. A common match system across all subspecialties was preferred at rates of 50% fellowship program directors, 74% residency program directors, and 26% chief residents. There was widespread reported compliance with the SCARD fellowship timeline policy. Subspecialty programs using the match system reported interviewing greater numbers of applicants per position. Fellowship directors and chief residents reported that the most common duration of mini-fellowship experiences was 2 to 3 months.

CONCLUSION

There is a division between chief residents and program directors regarding the preference for a common radiology match. Adopting a radiology-wide fellowship match would increase the number of interviews required. The SCARD fellowship timeline policy has been successful, and there is support across stakeholders regarding the common timeline. Mini-fellowships are highly variable in length and structure.

Utilization of a New Customizable Scoring Tool to Recruit and Select Pulmonary/Critical Care Fellows

Background: Finding the ideal candidate for a residency/fellowship program has always been difficult. Finding the “perfect” match has always been the ultimate goal. However, many factors affect obtaining that “perfect” match. In the past, we would have each attending physician review around 20 to 50 Electronic Residency Application Service (ERAS) applications and rank them into three categories: high, middle, or low. Depending on their ranking, the applicant would be invited for an interview. After the interview, the applicants’ files (ERAS and interview) would be reviewed and ranked by the faculty as a group. This was time-consuming and fraught with too much subjectivity and minimal objectivity. We, therefore, sought to find a way to assess and rank applicants in a more objective and less time-consuming manner. By creating a customizable scoring tool, we were able to screen applicants to our pulmonary/critical care fellowship program in an efficient and a more objective manner.

Objectives: A customizable scoring tool was developed weighting components in the ERAS and interview process, allowing residency/fellowship programs to create a final rank list consistent with the programs’ desired applicants.

Methods: Two hundred and sixty pulmonary/critical care fellowship applications were reviewed from 2013 to 2018. In 2018, we used our new scoring rubric to create a rank list and rescore previous applicants. The traditional and new lists were compared to the final rank list submitted to the National Residency Matching Program (NRMP) for 2018. We wanted to ascertain which scoring method correlated best with the final rank list submitted to the NRMP. We obtained feedback from eight faculty members who had reviewed applicants with both scoring tools.

Results: The novel customizable scoring tool positively correlated with the final rank list submitted to the NRMP (r= 0.86). The novel tool showed a better correlation to the final rank list than the traditional method. Faculties (6/6, 100%) responded positively to the new tool.

Conclusions: Our new customizable tool has allowed us to create a final rank list that is efficient and more focused on our faculty’s desired applicants. We hope to assess and compare the quality of applicants matched through this scoring system and the traditional method by using faculty evaluations, milestones, and test scores.

Video Interviewing: A Review and Recommendations for Implementation in the Era of COVID-19 and Beyond

Due to the COVID-19 pandemic, it is very likely that many radiology residency and fellowship programs will adopt interactive videoconference interviewing for the 2020-2021 residency match cycle. Although video interviewing has become a common part of the hiring process for business, experience with video interviewing for resident and fellow selection has been limited. Advantages of video interviews over traditional on-site interviews include cost-savings to both applicants and residency programs, less disruption to an applicant's educational activities, and potential for training programs to access a wider pool of candidates. The loss of the casual interactions that occur during an on-site interview and the inability of candidates to evaluate training facilities and their surrounding environments in-person are among the obstacles posed by video interviews, but training programs can mitigate these challenges with enhanced website content and creative media solutions. Through a review of the existing literature and internet resources, this article recommends specific measures medical schools, applicants, and radiology residency and fellowship programs can take to optimize the virtual interview experience for all involved parties.