Ophthalmology resident selection: current trends in selection criteria and improving the process

Analysis of Research Activity Among Ophthalmology Fellows

PURPOSE

The academic success of ophthalmology fellows may be significantly impacted by their research engagement. Evaluating the research activity of fellows by subspecialty may provide insight into trends useful for prospective applicants. The objective of this study was to assess the research activity of ophthalmology fellows in the year 2023.

METHODS

Ophthalmology fellowship programs were compiled using the Association of University Professors of Ophthalmology and the American Society of Ophthalmic Plastic and Reconstructive Surgery websites. Program subspecialties were categorized as: Cornea and External Disease [CED]; Cornea, External Disease, and Refractive Surgery [CEDRS]; Glaucoma; Medical Retina [MR]; Neuro-Ophthalmology [NO]; Oculoplastics; Pediatric Ophthalmology [PO]; Surgical Retina [SR]; Uveitis; and Other. "Other" consisted of Oncology, Pathology, combined Oncology and Pathology fellowships. We identified current fellows using publicly available online information. Programs and fellows that did not have information available were excluded. Total publications, Hirsch index (h-index), and weighted-relative citation ration (w-RCR) were utilized as measures of research output, while total citations and mean-RCR (m-RCR) served as proxies for research impact. Duration of publishing was calculated using the years of the oldest and most recent publications. Statistical analysis was performed using Kruskal-Wallis H tests with an alpha value of .05.

RESULTS

A total of 373 fellows from 229 programs met our inclusion criteria. More than half of fellows were men (54.4%), and the most common degree type was MD (93.0%). Across all subspecialties, the median h-index was 3.0 (IQR = 4), m-RCR was .9 (IQR = 1.2) and w-RCR was 3.2 (IQR = 9.8). The median number of publications was six (IQR = 10, with 34 citations (IQR = 110) and 4 years of publishing (IQR = 6). We observed significant differences in h-index (p = .038), total publications (p < .001), and w-RCR (p = .028) by subspecialty.

CONCLUSION

We observed significant differences in research output, but no differences in research impact by subspecialty. Overall, Uveitis and Oculoplastics fellows had higher research activity, while Medical Retina and Other fellows had the lowest. This data is pertinent to better understand the landscape of ophthalmology fellowship applications.

Evaluation of commitment to equity, diversity, and inclusion on Canadian ophthalmology residency program websites

OBJECTIVE

Program websites play a crucial role in providing information to prospective residency applicants. This research assesses the extent to which Canadian ophthalmology residency programs demonstrate their commitment to equity, diversity, and inclusion (EDI) through their websites and identifies areas for potential improvement.

METHODS

A cross-sectional analysis of websites affiliated with 15 accredited Canadian ophthalmology residency training programs was performed. A score was assigned based on 16 EDI elements derived from recent literature. The relationship between program website EDI elements with language, geography, and regional population diversity was explored.

RESULTS

Of the 15 Canadian ophthalmology residency programs, 12 (80%) met at least 1 of 16 EDI elements, with a mean program score of 2.4 (SD = 1.92). The program that attained the highest score met 7 of 16 EDI elements. Anglophone programs had a higher mean score (2.92; SD = 1.78) when compared with their Francophone counterparts (0.33; SD = 0.58; p = 0.023). No significant difference was found between programs in western, central, and eastern Canada (p = 0.86). The regional minorities proportion associated with each program was not found to influence scores (p = 0.088).

CONCLUSIONS

EDI elements were seldom addressed on Canadian ophthalmology residency program affiliated websites. We propose recommendations in 4 domains: showcasing team diversity, clear discourse in recruitment, inclusive resources, and program organization. Improving EDI visibility on program websites through revisions of existing websites or through creation of new promotional websites may foster recruitment of a diverse resident body.

Analysis of Reapplicants to Ophthalmology Residency: Factors Associated with Successful Matching

BACKGROUND

Ophthalmology residency programs are highly competitive and each year there are many unmatched reapplicants who must make time-sensitive decisions on how to prepare for their reapplication. Our analysis of reapplication factors will be the first evidence-based guide to reapplying ophthalmology.

OBJECTIVE

To determine the components of a reapplicants application that contribute and that do not contribute to ophthalmology residency match success.

METHODS

Eighty-eight total reapplicants to Dell Medical School's Department of Ophthalmology residency program for the year of 2022 and 2023 were analyzed in a retrospective review in 2023 based on their San Francisco match applications. We assessed match success in the context of demographics, academic performance, cognitive measures, letters of recommendation, personal statements, and extracurricular activities.

RESULTS

Of 84 reapplicants who completed the San Francisco Match, 41 matched successfully (48.8%). Factors that significantly affected match success include quantity of LORs written by ophthalmologists (P = .0143), choice of interim year activity (p = .0019), and Step 2 Clinical Knowledge score (P = .0017). Our findings demonstrate that noncontributory factors to reapplicant match rate include number of publications during research years, demographics, clinical grades, Alpha Omega Alpha (AOA) membership, LOR qualities, and Step 1 score.

CONCLUSIONS

Ophthalmology residency reapplicants should focus their efforts on finding a research year or established pre-ophthalmology residency program for their interim year and bolstering their letters of recommendations within the field. The quality of in-specialty relationships and letters of recommendation are more impactful than the number of publications during a research gap year. Step 2 CK score is a significant factor but would have already been determined by the time of reapplication. Metrics such as Step 1 scores, AOA status, number of volunteering experiences, and LOR and personal statement qualities had no effect on a reapplicant's chance of matching.

Predictors of Ophthalmology Resident Performance From Medical Student Application Materials

OBJECTIVE

To determine whether elements in ophthalmology residency applications are predictors of future resident performance.

DESIGN

This multi-institutional, cross-sectional, observational study retrospectively reviewed the residency application materials of ophthalmology residents who graduated from residency from 2006 through 2018. Resident performance was scored by 2 faculty reviewers in 4 domains (clinical, surgical, academic, and global performance). Correlation between specific elements of the residency application and resident performance was assessed by Spearman correlation coefficients (univariate) and linear regression (multivariate) for continuous variables and logistic regression (multivariate) for categorical variables.

SETTING

Seven ophthalmology residency programs in the US.

PARTICIPANTS

Ophthalmology residents who graduated from their residency program.

RESULTS

High-performing residents were a diverse group, in terms of sex, ethnicity, visa status, and educational background. Residents with United States Medical Licensing Examination Step 1 scores higher than the national average for that year had significantly higher scores in all 4 performance domains than those who scored at or below the mean (all domains P < 0.05). Residents who had honors in at least 4 core clerkships and who were members of Alpha Omega Alpha Medical Honor Society also had higher scores in all 4 performance domains (all domains P ≤ 0.04). Step 1 score (ρ=0.26, P < 0.001) and the difference between Step 1 score and the national average for that year (ρ=0.19, P = 0.009) positively correlated with total resident performance scores. Residents who passed the American Board of Ophthalmology Written Qualifying Examination or Oral Examination on their first attempt had significantly higher Step 1/2 scores (P ≤ 0.005), Ophthalmology Knowledge Assessment Program scores (P = 0.001), and resident performance scores (P ≤ 0.004).

CONCLUSIONS

In this new landscape of increasing numbers of applicants to residency programs and changing of the Step 1 score to pass/fail, our findings may help guide selection committees as they holistically review applicants to select exceptional future residents in ophthalmology.

An Analysis of Medical Student PubMed-Indexed Research Productivity and Factors Associated With Matching at Top-Ranked Ophthalmology Residency Programs

Background Limited information exists regarding the research productivity of matched ophthalmology applicants given that ophthalmology residencies do not participate in the National Residency Match Program. Objectives This study determines the research productivity characteristics of matched ophthalmology applicants and variables associated with matching to higher-tier ophthalmology residency programs. Methods Medical school, matched residency program, and applicant-specific PubMed-indexed research productivity (including consideration for first-author publications, relevance to ophthalmology, and acceptance before application submission date) for 2020-2021 matched ophthalmology applicants were collected from publicly available websites. Statistical analysis was conducted using chi-squared tests and t-tests to compare research productivity between groups (gender, medical school tier, and ophthalmology residency program Doximity rank). Multivariate regression was used to analyze research factors associated with matching at the top 20 Doximity-ranked ophthalmology residency programs. Results Three hundred ninety-three matched ophthalmology applicants for the 2020-2021 academic year were analyzed with an average of 2.4 ± 3.1 (median: 1 {0-3}) publications, 1.0 ± 2.1 (median: 0 {0-1}) ophthalmic publications, 0.8 ± 1.4 (median: 0 {0-1}) first-author publications, and 0.5 ± 1.1 (median: 0 {0-1}) ophthalmic first-author publications. The applicants who matched at the top 20 Doximity-ranked ophthalmology residency programs were more likely to matriculate from the top 40 medical schools (63% versus 22%, p < 0.001), have more first-author publications at the time of application submission (1.1 ± 1.6 versus 0.7 ± 1.3, p = 0.044), and have more projects resulting in publications after submission (2.0 ± 2.5 versus 1.4 ± 2.3, p = 0.048). In multivariate regression, attendance at a top 40 medical school (adjusted odds ratio {aOR} = 6.07, 95% confidence interval {CI}: 3.56-10.5, p < 0.001) was a significant predictor of matching at a top 20 Doximity ophthalmology residency program, and no variables associated with research productivity were significant predictors. Conclusions There has been a consistent increase in research productivity among matched ophthalmology applicants. However, in multivariate analysis, the medical school tier was the only significant variable for matching at top-tier programs. More nuanced studies regarding the effect of research productivity on ophthalmology applicants are needed.

'Selection into training will always be an inexact process': A survey of Directors of Physician Education on selection into Basic Physician Training in Australia and New Zealand

BACKGROUND

Despite being one of the largest medical specialty training programmes in Australasia, there is no standardised method for selection into Basic Physician Training (BPT), and limited data exist regarding current practices.

AIMS

To address existing knowledge gaps, we aimed to create a 'snapshot' of current BPT selection practices and explore the perspectives of Directors of Physician Education (DPEs) regarding trainee selection.

METHODS

An electronic survey of DPEs from adult and paediatric medicine BPT sites in Australia and New Zealand was undertaken in January-February 2022. A combination of free text, multiple-choice and yes/no answers were analysed using descriptive statistics and qualitative content analysis.

RESULTS

A total of 70 responses were received, achieving a response rate of 35% (70/198). Selection practices were found to be heterogenous across BPT sites. Respondents had varying opinions regarding the utility of selection tools and desirable candidate attributes. A heavy reliance upon interviews and the reported use of subjective assessments raise concerns for selection process bias.

CONCLUSION

BPT sites should critically evaluate their selection methods, and more research in this field is needed to establish best practice.

Identification of Factors that May Predict Career Trajectory Among Neuro-Ophthalmology Fellows

BACKGROUND

There is modest literature regarding fellowship applicant factors that may predict future career achievement. We aim to characterize neuro-ophthalmology fellows and identify and analyze characteristics that may predict future career trajectory.

METHODS

Data, including demographic information, academic background, scholarly activities, and practice information, were collected using publicly available sources, on individuals who completed neuro-ophthalmology fellowships from 2015 to 2021. Summary statistics describing the cohort were calculated. Prefellowship characteristics were compared with postfellowship characteristics to evaluate which prefellowship characteristics may predict postfellowship academic productivity and career achievement.

RESULTS

Data were collected on 174 individuals (41.6% men, 58.4% women). Sixty-five percent were residency-trained in ophthalmology, 31% neurology, 1.7% both, and 1.7% pediatric neurology. Fifty-eight percent completed residency in the US, 8% in Canada, 32% internationally, and 2% in multiple locations. Among those practicing in the US/Canada, 63.8% practice at academic centers, 35.3% private practice, and 0.9% at both. Thirty-one percent completed additional subspecialty training and 17.8% additional graduate degrees. Completion of additional fellowship training or graduate degrees, and publication of more papers before fellowship, correlated with later academic productivity. There were no significant correlations between completion of an additional fellowship or graduate degree with current practice environment or attainment of leadership roles. There were no significant correlations between total publishing productivity prefellowship and practice environment or leadership roles postfellowship.

CONCLUSIONS

Additional graduate degrees/subspecialty training, and prefellowship academic productivity, correlated with later academic productivity among neuro-ophthalmologists, suggesting that these metrics may be helpful in predicting future academic performance among fellowship applicants.

Barriers in Ophthalmology Residency Applications for Students Identifying as Underrepresented in Medicine: A San Francisco Match Analysis

OBJECTIVE

There is a significant lack of ophthalmologists who self-identify as underrepresented in medicine (URiM) in the physician workforce. Prior literature has revealed bias in traditional metrics for selection relied on by resident programs such as United States Medical Licensing Examination (USMLE) scores, letters of recommendation (LOR), and induction into medical honors societies such as Alpha Omega Alpha (AOA). The purpose of this study was to elucidate race-based differences in word usage within ophthalmology residency letters of recommendation that may disproportionately affect URiM applicants.

DESIGN

This was a retrospective, cohort study.

SETTING

This was a multicenter study across the Wilmer Eye Institute at Johns Hopkins, the University of California San Francisco, and the University of North Carolina at Chapel Hill.

PARTICIPANTS

San Francisco (SF) Match applications submitted to three ophthalmology residency programs between 2018 and 2020 were reviewed. URiM status, USMLE Step 1 score, and AOA membership were recorded. Letters of recommendation were analyzed using text analysis software. T-tests and chi-squared or Fisher's exact tests were used to compare continuous and categorical variables, respectively. Frequency of word/summary term usage in letters of recommendation were the main outcome measures.

RESULTS

Relative to non-URiM applicants, URiM applicants had lower USMLE Step 1 scores (mean difference=7.0; p<0.001). Non-URiM letters of recommendation were more likely to describe applicants as "dependable" (p=0.009) and highlight "research" (p=0.046). URiM letters were more likely to describe applicants as "warm" (p=0.02) and "caring" (p=0.02).

CONCLUSIONS

This study identified potential barriers for URiM ophthalmology residency applicants which can help guide future interventions to increase workforce diversity.

Attending Perceptions on the Use of Preference Signaling in the Ophthalmology Residency Application Process

Objective  This article characterizes perspectives of ophthalmologists involved in the residency selection process regarding the potential impact of preference signaling on the ophthalmology residency match. Methods  An anonymous online questionnaire generated from SurveyMonkey was approved by the Association of University Professors of Ophthalmology (AUPO) Data Resource Committee for distribution to 391 individuals from the AUPO Departmental Chairs, Program Directors, and Directors of Medical Student Education email listservs in August 2022. Results  A total of 96 (24.6%) ophthalmology faculty completed the questionnaire. The majority ( n  = 76, 79.2%) agreed or strongly agreed that preference signaling should be implemented in the ophthalmology residency application system. Most respondents agreed or strongly agreed that preference signaling will allow for more holistic reviews of applications ( n  = 55, 57.3%), agreed or strongly agreed that it will benefit applicants who do not have connections to home programs or faculty that can reach out to desired programs ( n  = 81, 84.4%), and agreed or strongly agreed that it will improve the distribution of interviews to applicants ( n  = 76, 79.2%). Participants agreed or strongly agreed that applicants who have signaled interest in their program will receive preference when offering interviews ( n  = 59, 61.5%), and those signals will be used as a tiebreaker for similar applications ( n  = 75, 78.1%). The majority of participants believed that the ideal number of preference signals' applicants should be given three to four signals ( n  = 35, 36.0%) or five to six signals ( n  = 29, 30.2%). Conclusion  A majority of ophthalmology faculty surveyed support the integration of preference signaling into the ophthalmology residency match.

Gender-Based Differences in Language Used by Students to Describe Their Noteworthy Characteristics in Medical Student Performance Evaluations

PURPOSE

The noteworthy characteristic (NC) section of the medical student performance evaluation (MSPE) was introduced to facilitate holistic review of residency applications and mitigate biases. The student-written aspect of the characteristics, however, may introduce biases resulting from gender differences in self-promotion behaviors. The authors conducted an exploratory analysis of potential gender-based differences in language used in NCs.

METHOD

The authors performed a single-center cohort analysis of all student-written NCs at the Perelman School of Medicine (2018-2022). NCs were converted into single words and characterized into word categories: ability (e.g., "talent"), standout ("best"), grindstone ("meticulous"), communal ("caring"), or agentic ("ambitious"). The authors qualitatively analyzed NC topic characteristics (i.e., focused on scholarship, community service). Logistic regression was used to identify gender differences in word categories and topics used in NCs.

RESULTS

The cohort included 2,084 characteristics from 783 MSPEs (47.5%, n = 371 written by women). After adjusting for underrepresented in medicine status, honor society membership, and intended specialty, men were more likely to use standout (OR = 2.00; 95% confidence interval [CI] = 1.35, 2.96; P = .001) and communal (OR = 1.40; 95% CI = 1.03, 1.90; P = .03) words in their NCs compared with women but less likely to use grindstone words (OR = 0.72; 95% CI = 0.53, 0.98; P = .04). Men were more likely than women to discuss scholarship (OR = 2.03; 95% CI = 1.27, 3.23; P = .003), hobbies (OR = 1.45; 95% CI = 1.07, 1.96; P = .02), and/or awards (OR = 1.59; 95% CI = 1.16, 2.16; P = .004) and less likely to highlight community service (OR = 0.66; 95% CI = 0.48, 0.92; P = .02).

CONCLUSIONS

The self-written nature of NCs permits language differences that may contribute to gender bias in residency application.

Resident Selection Criteria in Veterinary Medicine

With the continued rise of interest and need for veterinary specialists, information regarding optimal selection criteria for successful residency candidates has been lacking in veterinary medicine. A 28-question online survey was developed to determine prioritized resident selection criteria, the importance of formal interviews, and residency supervisor satisfaction with the current selection process. This survey was sent to all programs listed by the Veterinary Internship and Residency Matching Program (VIRMP) for the 2019-2020 program year. Overall, the most important aspects of the residency application process were (1) letters of recommendation, (2) performance during the interview, (3) personal contact/recommendation from a colleague, (4) personal statement, and (5) demonstrated interest in the residency specialty. While measures of academic performance including GPA and veterinary class rank may play a role in sorting of candidates in more competitive specialties, this does not necessarily exclude them from the ranking process. This information should be helpful to candidates and program directors alike in understanding the success of the current residency candidate selection process.

Fate of Research Articles Reported by Ophthalmology Residency Applicants: What Happens to "Submitted" and "In Preparation" Articles?

OBJECTIVE

As coronavirus disease 2019 affects clinical training opportunities and with the transition of U.S. Medical Licensing Examination Step 1 to pass-fail, research may become increasingly important for evaluating ophthalmology residency applicants. Though publication misrepresentation has been studied among ophthalmology residency applicants, eventual publication rates of incomplete articles remain unknown. We aimed to determine publication rates for manuscripts listed as "submitted" or "in preparation" on ophthalmology residency applications and identify factors associated with unpublished manuscripts.

DESIGN

San Francisco Match applications to the Wilmer Eye Institute for the 2019 ophthalmology residency cycle were retrospectively reviewed. Each applicant's number of "published," "submitted," and "in preparation" manuscripts was recorded, then verified 1.5 years later through PubMed, Google Scholar, or journal websites. Unverifiable manuscripts were deemed "unpublished."

SETTING

Single academic institution (Wilmer Eye Institute, Baltimore, MD, USA) PARTICIPANTS: All 458 medical students who applied to the Wilmer Eye Institute for the 2019 ophthalmology residency cycle through the San Francisco Match.

RESULTS

A total of 458 applications were reviewed. Of 428 "submitted" publications, 126 (29.4%) remained unpublished after 1.5 years. Of 324 manuscripts "in preparation," 215 (66.4%) remained unpublished. In a multivariate model, AOA was associated with not having an unpublished manuscript compared to applicants without AOA (OR: 0.423, p = 0.0163). Gender, Step 1 score, additional degrees, and a research year had no association.

CONCLUSIONS

Nearly two-thirds of manuscripts listed as "in preparation" remained unpublished. Specific guidance from research mentors may help applicants better represent their publications in residency applications.

Integrated vascular surgery applicants' perspectives of virtual residency interviews during the coronavirus disease 2019 application cycle

BACKGROUND

In agreement with Association of American Medical Colleges guidelines, the 2020 to 2021 integrated vascular surgery (I-VS) residency interviews were conducted virtually. In the present study, we collected data about the virtual interview process from the applicant's perspective, including preferences for interview format and the virtual resources they found most helpful.

METHODS

An anonymous, online survey study of medical students who had been accepted into I-VS residencies during the 2020 to 2021 application cycle was performed. The survey contained questions about applicant behavior during the virtual application cycle, their perception of the virtual interviews, the utility of the virtual format, their preferences, and the virtual resources they found the most helpful for determining their rank list.

RESULTS

Of 72 applicants, 38 (18 women, 19 men, and 1 declined to answer) had completed the survey for a 57.2% response rate. The average number of programs interviewed was 25 to 30 (31%). More than one-half (55%) of the respondents had responded that they had interviewed at more programs than they would have had the interviews been in person. More than one-half of the applicants (55%) reported that they preferred remote interviews with the option to visit in-person at their top choice programs compared with the use of all remote interviews (21%) or all in-person interviews (18%). Most had somewhat or strongly agreed (79%) that virtual interviews allowed them to properly gauge a residency program and provided adequate opportunities to interact with the residents (65%). The online resources the applicants found the most helpful in determining their rank lists were attending educational conferences, prerecorded videos from faculty and residents, and the program's social media platforms.

CONCLUSIONS

The results from the present study have illuminated the current trends and attitudes of I-VS applicants for virtual interviews, including the virtual resources they found the most useful. Virtual interviews were a preferred method of interviewing and allowed applicants to assess a residency program and interact with the current residents. These variables should be considered by the program leadership when developing protocols for upcoming application cycles.

Association Between Preresidency Peer-reviewed Publications and Future Academic Productivity or Career Choice Among Ophthalmology Residency Applicants

Importance

Ophthalmology-residency selection committees require robust metrics to review applicants. Participation in research activities is a core component of the application process for its perceived association with future academic productivity.

Objective

To evaluate the correlation between the number of preresidency peer-reviewed publications (PPPs) and subsequent peer-reviewed publications or career choices of ophthalmology residency graduates.

Design, Setting, and Participants

In this cross-sectional study, names of ophthalmology residency graduates were obtained. PubMed-indexed publication records were generated and publications were categorized as preresidency, intraresidency, and postresidency. First author and journal publications with an impact factor (IF) score of 3 or more were recorded. Current academic and community-based career statuses were designated. Names were obtained from cohort and alumni lists on residency program websites or by emailing program directors. Participants included US Accreditation Council for Graduate Medical Education-accredited ophthalmology residency graduates from 2013 to 2016.

Main Outcomes and Measures

The primary outcome measure was association of PPPs with later publications, first authorship, and journal publications with an IF score of 3 or more. The secondary outcome measure was difference in characteristics associated with academic vs community-based ophthalmologist.

Results

A total of 964 ophthalmologists (52% of graduates) were studied and most (85.5%) had PubMed-indexed publications. First authorship (ρ = 0.71; 95% CI, 0.67-0.74; P < .001) had a strong positive correlation with intraresidency publications, while journal publications with an IF score of 3 or more (ρ = 0.56; 95% CI, 0.51-0.60; P < .001) and PPPs (ρ = 0.38; 95% CI, 0.32-0.43; P < .001) had moderate and weak positive correlations, respectively. For postresidency publications, journal publications with an IF score of 3 or more (ρ = 0.86; 95% CI, 0.84-0.87; P < .001) had the strongest positive correlation followed by first authorship (ρ = 0.77; 95% CI, 0.74-0.79; P < .001) and PPPs (ρ = 0.26; 95% CI, 0.20-0.31; P < .001). Preresidency (t = 3.3; P = .001), intraresidency (t = 4.1; P < .001), postresidency (t = 7.5; P < .001), first author (t = 6.6; P < .001), and journal publications with an IF score of 3 or more (t = 5.9; P < .001) were greater for academic ophthalmologists compared with community-based ophthalmologists.

Conclusions and Relevance

Preresidency publication history is at least weakly correlated with future publications or work in an academic setting among ophthalmologists. Multiple factors associated with academic productivity were evaluated; however, adjustment for multiple analyses was not done and further testing is required to prove whether these factors are predictive.

Standardized Interview Scoring Methodology for Neurosurgical Residency Applicant Selection

Interviews are critical to the neurosurgery resident application process. The COVID-19 pandemic forced residency interview activities are conducted virtually. To maintain a degree of control during a period of uncertainty, our department implemented a standardized survey for interviewers to evaluate the noncognitive attributes and program compatibility of applicants. Our objective was to assess the reliability and biases associated with our standardized interviewer survey implemented in neurosurgical residency interviews. A 5-question interviewer survey to assess applicant interview performance and program compatibility was implemented during the 2020 to 2021 interview season. After the application cycle, survey metrics were retrospectively reviewed. Multiple cohort analyses were performed by dividing interviewers into cohorts based on status (faculty or resident) and sex. Applicant scores were assessed within sex subgroups for each aforementioned interviewer cohort. Intraclass correlation coefficients (ICCs) were calculated to assess survey reliability. Fifteen interviewers (8 faculty and 7 residents) and 35 applicants were included. Female applicants (17%) and interviewers (20%) comprised the minority. There were no differences between resident and faculty reviewer scores; however, female reviewers gave higher overall scores than male reviewers ( P = .003). There was no difference in total scores between female and male applicants when evaluating all reviewers or subgroups of faculty, residents, females, or males. ICC analysis demonstrated good (ICC 0.75-0.90) or excellent (ICC > 0.90) reliability for all questions and overall score. The standardized interviewer survey was a feasible and reliable method for evaluating noncognitive attributes during neurosurgery residency interviews. There was no perceptible evidence of sex bias in our single-program experience.

Enhancing Diversity in the Ophthalmology Workforce

Health care teams are most effective at addressing complex problems and improving health outcomes for underserved populations when team members bring diverse life experiences and perspectives to the effort. With rates of visual impairment expected to increase in the United States by 2050, especially among minority populations, diversification of the ophthalmology workforce will be critical in reducing disparities in access to and quality of vision health care. Currently, ophthalmology is less diverse with respect to race, ethnicity, and gender than graduating medical classes and other medical specialties, as well as the general US population. In addition, data on diversity in sexual orientation and gender identity, socioeconomic status, and disability are lacking in ophthalmology. The Minority Ophthalmology Mentoring and Rabb-Venable Excellence in Ophthalmology Programs are examples of initiatives to increase racial and ethnic diversity in the workforce and can serve as models for increasing other aspects of inclusiveness. Other strategies for improving vision health care for all Americans include continuing to support existing diversity programs and creating new ones; addressing unconscious and implicit bias in medical school, residency, and faculty selections; conducting holistic reviews of medical school and residency applications; diversifying selection committees and leadership; and encouraging faculty development of underrepresented groups.

Predictive Factors of Research Productivity among Ophthalmology Residents: A Benchmark Analysis

Introduction Positive and negative associations between prior publications and future research productivity is described in other fields, but no such analysis exists for ophthalmology. We conducted a study to determine characteristics of residents exhibiting research productivity during residency.

Methods Using San Francisco Match and Program Web sites, a roster of ophthalmology residents in 2019 to 2020 was compiled, and publication data was collected via PubMed and Google Scholar on a random sample of 100 third-year residents.

Results The median number of publications generated by ophthalmology residents before residency is 2 (range 0–13). Thirty-seven, 23, and 40 residents had zero, one, and two or more papers published during residency, respectively, with a median of 1 (range 0–14). On univariate analysis, compared with residents who published zero or one paper, those who published ≥ 2 were more likely to have more preresidency publications (odds ratio [OR] 1.30; p = 0.005), attend a top-25 ranked residency program by multiple metrics including Doximity reputation (OR 4.92; p < 0.001), and have attended a top-25 ranked medical school program by U.S. News and World Report (OR 3.24; p = 0.03). However, on adjusted analyses, the only factor that remained significant for predicting publications in residency was whether the residency program attended was top 25 ranked (OR 3.54; p = 0.009).

Discussion/Conclusion With the advent of the United States Medical Licensing Examination Step 1 pass/fail system, greater emphasis will be placed on other metrics, including research. This is the first benchmark analysis examining factors predictive of publication productivity in ophthalmology residents. Our study suggests that the residency program attended, not the medical school attended or prior publication history, plays an influential role in the number of publications produced during residency, highlighting the importance of factors to support research on the institutional level, such as mentorship and funding, rather than historical factors in research productivity by the resident.

Factors Associated with Early Career Research Productivity after Ophthalmology Residency

Background  Few studies have evaluated associations between ophthalmology trainee characteristics and performance with postgraduate research productivity. Purpose  This article evaluates factors associated with post-residency research productivity among U.S. ophthalmology graduates. Methods  Publicly available information of residents graduating between 2009 and 2014 from 30 randomly selected U.S. ophthalmology programs was collected from June to September 2020. Differences in publications between the 5 years post-residency and pre-residency/residency period were used as metrics of productivity. Residents with incomplete records were excluded. Results  A total of 758 of 768 residents, 306 females (40.4%) and 452 males (59.6%), met inclusion criteria. The mean (standard deviation [SD]) number of pre-residency publications was 1.7 (4.0), residency was 1.3 (2.2), and post-residency was 4.0 (7.3). Mean (SD) H-index was 4.2 (4.9). Top-ranked residency ( p  = 0.001), Alpha Omega Alpha (AOA) medical honor status ( p  = 0.002), U.S. medical school graduates ( p  < 0.001), and academic career ( p  < 0.001) were all associated with higher pre-/post-residency mean publication difference. Pursuing fellowship training also was associated with higher total publications ( p  < 0.001). Of all pre-residency degrees, PhD had the greatest odds of high postgraduate publications (defined as > 4). There was a positive correlation between both pre-residency/residency and post-residency publications (rho = 0.441; p  < 0.001) and between mean difference of pre-residency/post-residency publications for residents at a program and that program's Doximity rank (rho = 0.497; p  < 0.001). Multivariate logistic regression revealed, academic career choice (odds ratio [OR] = 3.38; p  < 0.001), Heed fellowship (OR = 3.12; p  = 0.031), > 2 residency publications (OR = 2.89; p  < 0.001), AOA status (OR = 2.0; p  = 0.004), and top-ranked residency programs (OR = 1.89; p  = 0.007), had greatest odds of > 4 postgraduation publications. Conclusion  Higher post-residency productivity was associated with multiple factors, with choice of an academic career, Heed fellowship, and residency productivity playing key roles.

Trends in Cornea Fellowship Applications and Applicant Characteristics: A San Francisco Match Analysis

Purpose We investigate trends in cornea fellowship positions filled over time and applicant characteristics associated with matching into cornea fellowship.

Methods Characteristics of cornea fellowship applicants were assessed using deidentified 2010 to 2017 San Francisco (SF) Match data. Publicly available SF Match cornea fellowship data including the number of participating programs, number of positions offered, number of positions filled, percentage of positions filled, and number of vacancies from 2014 to 2019 were also analyzed as data from 2010 to 2013 were unavailable.

Results From 2014 to 2019, the number of cornea fellowship programs increased by 11.3% (mean 2.3% per year, p = 0.006) and the number of positions offered increased by 7.7% (mean 1.4% per year, p = 0.065). Of 1,390 applicants from 2010 to 2017, 589 (42.4%) matched into cornea. After controlling for potential covariates, graduation from a U.S residency program (odds ratio [OR]: 6.15, 95% confidence interval [CI]: 4.05–9.35, p < 0.001) and a greater number of interviews completed (OR: 1.35, 95% CI: 1.29–1.42, p < 0.001) were associated with increased odds of cornea fellowship match. A greater number of applied programs (OR: 0.97, 95% CI: 0.95–0.98, p < 0.001) was associated with decreased odds of matching into cornea fellowship. The proportion of applicants matching into cornea fellowship increased until 30 applications.

Conclusions The number of cornea fellowship programs and positions increased from 2014 to 2019. Graduation from a U.S residency program and a greater number of interviews completed were associated with an increased likelihood of cornea fellowship match. Unlike applying to any ophthalmology subspecialty fellowship, applying to greater than 30 cornea fellowship programs was associated with decreased odds of matching.

Exploring Potential Schedule-Related and Gender Biases in Ophthalmology Residency Interview Scores

Purpose Prior studies have revealed grading discrepancies in evaluation of personal statements and letters of recommendation based on candidate's race and gender. Fatigue and the end-of-day phenomenon can negatively impact task performance but have not been studied in the residency selection process. Our primary objective is to determine whether factors related to interview time and day as well as candidate's and interviewer's gender have a significant effect on residency interview scores.

Methods Seven years of ophthalmology residency candidate evaluation scores from 2013 to 2019 were collected at a single academic institution, standardized by interviewer into relative percentiles (0–100 point grading scale), and grouped into the following categories for comparisons: different interview days (Day 1 vs. Day 2), morning versus afternoon (AM vs. PM), interview session (Day 1 AM/PM vs. Day 2 AM/PM), before and after breaks (morning break, lunch break, and afternoon break), residency candidate's gender, and interviewer's gender.

Results Candidates in the morning sessions were found to have higher scores than afternoon sessions (52.75 vs. 49.28, p < 0.001). Interview scores in the early morning, late morning, and early afternoon were higher than late afternoon scores (54.47, 53.01, 52.15 vs. 46.74, p < 0.001). Across all interview years, there were no differences in scores received before and after morning breaks (51.71 vs. 52.83, p = 0.49), lunch breaks (53.01 vs. 52.15, p = 0.58), and afternoon breaks (50.35 vs. 48.30, p = 0.21). No differences were found in scores received by female versus male candidates (51.55 vs. 50.49, p = 0.21) or scores given by female versus male interviewers (51.31 vs. 50.84, p = 0.58).

Conclusion Afternoon residency candidate interview scores, especially late afternoon, were significantly lower than morning scores, suggesting the need to further study the effects of interviewer's fatigue in the residency interview process. The interview day, presence of break times, candidate's gender, and interviewer's gender had no significant effects on interview score.

Factors involved in applicant interview selection and ranking for chronic pain medicine fellowship

INTRODUCTION

Applicants to chronic pain medicine fellowship programs often express confusion regarding the importance of various selection criteria. This study sought to elucidate program directors' considerations in applicant selection for fellowship interviews and ranking and to correlate these criteria with match statistics to provide a guide for prospective candidates.

METHODS

An electronic survey was sent to all Accreditation Council for Graduate Medical Education-accredited chronic pain fellowship directors. The importance of various applicant characteristics were evaluated and compared with recent match data.

RESULTS

Fifty-seven program directors completed the survey. The most important factors involved in applicant interview selection were perceived commitment to the specialty, letters of recommendation from pain faculty, scholarly activities, and leadership experiences. Although completion of a pain rotation was valued highly, experience with procedures was of relatively low importance. There was no preference if rotations were completed within the responders' department. Variability was noted when considering internal applicants or the applicant's geographic location. When citing main factors in ranking applicants, interpersonal skills, interview impression and applicant's fit within the institution were highly ranked by most responders.

DISCUSSION

Assessment of an applicant's commitment to chronic pain is challenging. Most responders prioritize the applicant's commitment to chronic pain as a specialty, scholarly activity, participation in chronic pain rotations, pain-related conferences and letters of recommendation from pain faculty. Chronic pain medicine fellowship candidates should establish a progressive pattern of genuine interest and involvement within the specialty during residency training to optimize their fellowship match potential.

Current Trends of Research Productivity among Students Matching at Top Ophthalmology Programs

Importance San Francisco Match publishes no data on the research output of matched applicants to an ophthalmology residency.

Objective The aim of this study was to examine the temporal trends in publication volume by medical students who successfully matched into a top ophthalmology residency.

Methods This retrospective case series compared all residents in the top 30 ophthalmology residency programs from the class of 2022 and 2017. Publication volume from before September 15th of the residents' fourth year of medical school was recorded using PubMed and Google Scholar. We recorded total number of publications (any authorship), first/second author publications, and ophthalmology-specific publications. Using Welch's t-test, publication volumes were statistically compared against all others.

Results One-hundred sixty-one residents from the class of 2022 and 145 residents from the class of 2017 were included. Total publications per matched applicant (mean ± standard deviation) were 3.04 ± 0.35 for the class of 2022 and 1.67 ± 0.23 for the class of 2017. Mean publications in ophthalmology journals were 1.07 ± 0.20 (2022) and 0.58 ± 0.13 (2017); mean first author publications were 1.00 ± 0.13 (2022) and 0.64 ± 0.11 (2017) and mean second author publications were 0.70 ± 0.10 (2022) and 0.37 ± 0.06 (2017). Research productivity in all four metrics (total, ophthalmology journals, first author, and second author publications) was significantly higher for the class of 2022 than the class of 2017 (p = 0.001; p = 0.03; p = 0.03; p = 0.02, respectively) supporting the trend of increasing research output among students. Applicants with PhD degrees had statistically more total and first author publications in 2017 (p = 0.01; p = 0.045), but only more first author publications in 2022 (p = 0.01). International applicants produced significantly more total publications in 2022 (p < 0.001).

Conclusions Overall, after a 5-year period, the authors found matched applicants had significantly increased publications compared with those at the beginning of the period. We also identified several applicant factors that may have variable effects on research publication. This analysis emphasizes the growing importance of research in the match process and can help future applicants navigate the ophthalmology match.

The Effect of Changing Fellowship Interview Format on Candidate Ranking Variabilities: The COVID-19 Experience

Purpose

This study investigated the effect of interview format changes (in-person to virtual, one-to-one to multiple-to-one) necessitated by the COVID-19 travel restrictions on preliminary fellowship candidate ranking variabilities.

Design

Cross-sectional observational study.

Method

In 2018 and 2019, the glaucoma fellowship interviews were conducted in-person in a one-to-one format, whereas in 2020, interviews were virtual and in a multiple (interviewers)-to-one (candidate) format. We compared ranking ranges of interviewers within the same virtual room (WSR) and not within the same virtual room (NWSR) to assess the effect of WSR versus NWSR on ranking variabilities. We also compared ranking categories ("accept," "alternate," and "pass") agreements between 2018, 2019, and 2020 to assess the effect of virtual versus in-person interviews on ranking variabilities.

Results

NWSR and WSR mean rankings differed by 1.33 (95% confidence interval difference 0.61 to 2.04, p = 0.0003), with WSR interviewers having less variability than NWSR pairs. The variability between 2018/2019 (in-person interviews) and 2020 (virtual interviews) showed no differences between in-person and virtual interviews (weighted Kappa statistic 0.086 for 2018, 0.158 for 2019, and 0.101 for 2020; p < 0.05 for all years). The overall least attractive candidate has the lowest variability; the most attractive candidate has the second lowest variability.

Conclusion

Grouping interviewers WSR during the interview decreased ranking variabilities compared to NWSR, while a change from in-person to virtual interview format did not increase the ranking variabilities. This suggests that the decreased nonverbal interactions in virtual interviews do not decrease interviewers' perceptions as applied to preliminary rankings.

Can Behavior-Based Interviews Reduce Bias in Fellowship Applicant Assessment?

PURPOSE

Components of trainee applications may introduce bias based on race or gender. Behavior-based interviews (BBIs) rely on structured questions to elicit applicants' past experiences to predict future behavior. Our objective was to implement BBIs in one fellowship program and compare applicant assessment by race and gender when using a standardized assessment tool versus a BBI-based tool.

METHODS

In 2019 and 2020, we developed BBIs and BBI-specific assessments; 6 of 15 faculty were trained in this interview method. Applicants completed 6 interviews with either a BBI or unstructured format. All faculty completed a standardized assessment on applicants. BBI faculty also completed a BBI-specific assessment. Normalized average scores were calculated and used to rank applicants into quartiles. Race was categorized into White, underrepresented minorities (URMs; Black and Hispanics), and non-URMs (all others). Faculty and applicants were surveyed about BBIs.

RESULTS

Seventy-five applicants were interviewed. Significant differences were found in standardized assessment scores (White 1.01 +/- 0.09, non-URM 1.02 +/- 0.08, URM .94 +/- 0.07; P = .02) and quartiles by race (P = .05), but not for BBI scores (White 0.98 +/- 0.09, non-URM 1.03 +/- 0.09, URM 1.02 +/- 0.1; P = .18) or quartiles by race (P = .17). There were no significant differences in score or quartile by gender for either tool. The majority of faculty and applicant survey respondents commented positively about BBIs.

CONCLUSION

BBIs were successfully implemented and generally reviewed positively by faculty and applicants. BBIs reduced racial differences in applicant assessments. Applicant assessment may benefit from structured tools to mitigate potential biases.

Characteristics of First-Year Residents in Top-Ranked United States Ophthalmology Residency Programs

Objective  The aim of the study is to investigate the characteristics of first-year residents associated with attending a top-ranked United States (U.S.) ophthalmology residency program over the past decade. Methods  First-year ophthalmology residents in 2009, 2013, 2016, and 2019 were identified from institutional websites, Doximity, LinkedIn and the Wayback Machine. Publications were obtained from Scopus and Google Scholar; research productivity was measured using the h -index, and medical school region based on U.S. Census Bureau designations. Medical school and ophthalmology training program rankings were based on U.S. News & World Report (U.S. News) rankings and National Institutes of Health (NIH) funding. One-way ANOVA, Wilcoxon rank sum, χ 2 , and t -tests were used to analyze trends, and odds ratios (ORs) were calculated using logistic regression. Results  Data were obtained on 81% (1,496/1,850) of the residents; 43% were female; 5% were international medical graduates (IMGs); and 10% had other graduate degrees. Over the decade, the mean h -index increased (0.87-1.26; p <0.05) and the proportion of residents who attended a top 20 medical school decreased (28-18%; p <0.05). In a multivariate logistic regression model, characteristics associated with being a first-year resident in a top 20 program ranked by U.S. News were female gender [OR: 1.32, 95% CI: 1.02-1.72], having a Master's degree [OR: 2.28, 95% CI: 1.29-4.01] or PhD [OR: 2.23, 95% CI: 1.32-3.79], attending a top 20 [OR: 5.26, 95% CI: 3.66-7.55] or a top 40 medical school by NIH funding [OR: 2.45, 95% CI: 1.70-3.54], attending a medical school with a mean USMLE Step 2 score above 243 [OR: 1.64, 95% CI: 1.01-2.67] or located in the Northeast [OR: 2.00, 95% CI: 1.38-2.89] and having an h -index of one or more [OR: 1.92, 95% CI: 1.47-2.51]. Except for gender, these characteristics were also significantly associated with matching to a top 20 ophthalmology program by NIH funding. Conclusion  Female gender, graduate degrees, research productivity, and attending a medical school with high research productivity, high mean USMLE Step 2 score or in the Northeast were key characteristics of first-year residents in top-ranked U.S. ophthalmology residency programs.

Perceptions of a Virtual Interview Exercise for Ophthalmology Residency Applicants

Importance In-person interviews have traditionally been considered a crucial component of the residency application process. Virtual interviews (VIs) became the standard format for the 2020 to 2021 application cycle due to the novel coronavirus disease 2019 (COVID-19) pandemic. VIs offer a new perspective and challenge to this process which warrants unique considerations and further understanding of effects on applicants.

Objective This study aimed to assess the perceived efficacy of a VI preparedness exercise for ophthalmology residency applicants in the 2021 residency application cycle.

Design, Setting, and Participants A cross-sectional survey was distributed online. All participants in a mock VI exercise conducted via video-telecommunication technology were invited to complete the survey. Data collection occurred from October 12, 2020, to November 2, 2020. A follow-up survey after a match results released was distributed to all participants and data collection occurred from February 18, 2021, to February 25, 2021.

Main Outcome and Measures Applicant demographics, comfort, and attitudes toward VIs and VI practice were the primary measurements of this study.

Results Responses to the initial survey were received from all 35 participants (100%) in the VI mock interviews. There was a statistically significant difference between the pre- and postinterview responses for “How prepared do you feel for virtual interviews with residency programs?” (p = 0.0003) and “How likely are you to practice virtual interviews with someone you know?” (p = 0.0023). Participants reported feeling more prepared for VIs with residency programs after the mock interview (p = 0.002). A greater proportion of participants responded with “Very Likely” after the mock interview in comparison to before the interview to the questions “How likely are you to practice interviews with someone you know?” (p = 0.039) and “How likely are you to practice virtual interviews in the same room/area as you will during the official interview season?” (p = 0.021). Of the 35 original participants, 20 completed the follow-up survey. There were an equal number of participants who responded either “Helped Somewhat” (n = 9) or “Helped Greatly” (n = 9) to “How much did the VI mock exercise help you for the actual interview season?” in the follow-up survey. The majority of follow-up survey respondents (17/20) reported that they had additional practice in the virtual environment for interviews after the VI mock exercise. There was no significant difference in perceived helpfulness of the VI mock exercise during the actual interview season between matched and unmatched participants.

Conclusion and Relevance As residency applicants prepare for future VIs, practice and adequate preparation will be essential. In this study, implementation of a VI preparedness exercise had a positive impact on applicants' perception of their preparedness and intention to practice the format in the future.

Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency

Purpose Only from reviewing applications, it is difficult to identify which applicants will be successful ophthalmology residents. The change of the USMLE Step 1 scoring to “Pass/Fail” removes another quantitative metric. We aimed to identify application attributes correlated with successful residency performance. This study also used artificial intelligence (AI) to evaluate letters of recommendation (LOR), the Dean's letter (MSPE), and personal statement (PS).

Design Retrospective analysis of application characteristics versus residency performance was conducted.

Participants Residents who graduated from the Dean McGee Eye Institute/University of Oklahoma Ophthalmology residency from 2004 to 2019 were included in this study.

Methods Thirty-four attributes were recorded from each application. Residents were subjectively ranked into tertiles and top and bottom deciles based on residency performance by faculty present during their training. The Ophthalmic Knowledge Assessment Program (OKAP) examination scores were used as an objective performance metric. Analysis was performed to identify associations between application attributes and tertile/decile ranking. Additional analysis used AI and natural language processing to evaluate applicant LORs, MSPE, and PS.

Main Outcome Measures Characteristics from residency applications that correlate with resident performance were the primary outcome of this study.

Results Fifty-five residents and 21 faculty members were included. A grade of “A” or “Honors” in the obstetrics/gynecology (OB/GYN) clerkship and the presence of a home ophthalmology department were associated with ranking in the top tertile but not the top decile. Mean core clerkship grades, medical school ranking in the top 25 U.S. News and World Report (USNWR) primary care rankings, and postgraduate year (PGY)-2 and PGY-3 OKAP scores were predictive of being ranked in both the top tertile and the top decile. USMLE scores, alpha-omega-alpha (AOA) status, and number of publications did not correlate with subjective resident performance. AI analysis of LORs, MSPE, and PS did not identify any text features that correlated with resident performance.

Conclusions Many metrics traditionally felt to be predictive of residency success (USMLE scores, AOA status, and research) did not predict resident success in our study. We did confirm the importance of core clerkship grades and medical school ranking. Objective measures of success such as PGY-2 and PGY-3 OKAP scores were associated with high subjective ranking.

Predictors of future research productivity among ophthalmology residents

PURPOSE

To investigate the characteristics of first-year ophthalmology residents associated with high research productivity in twelve years.

METHODS

All first-year ophthalmology residents in the United States (US) in 2009 listed by institutional websites were identified. Publications were obtained from Scopus and Google Scholar, h-indices were calculated, and medical school characteristics such as research productivity, region, rank, and average USMLE Steps 1 and 2 scores were obtained from National Institutes of Health, US Census Bureau, and US News.

RESULTS

Data were obtained on 70% (316/453) of residents; most were male, had Doctor of Medicine (MD) degrees, and graduated from medical schools in the US or Canada. Having an additional graduate degree [OR: 3.05, 95% CI: 1.07-8.67], between 1 and 3 publications [OR: 4.16, 95% CI: 2.22-7.79], or 4 or more publications [OR: 14.27, 95% CI: 3.13-64.94] were associated with higher future research productivity.

CONCLUSION

Graduate degrees and publication count were key predictors of future research productivity among ophthalmology residents in the US.

Growth mindset in competency-based medical education

The ongoing adoption of competency-based medical education (CBME) across health professions training draws focus to learner-centred educational design and the importance of fostering a growth mindset in learners, teachers, and educational programs. An emerging body of literature addresses the instructional practices and features of learning environments that foster the skills and strategies necessary for trainees to be partners in their own learning and progression to competence and to develop skills for lifelong learning. Aligned with this emerging area is an interest in Dweck's self theory and the concept of the growth mindset. The growth mindset is an implicit belief held by an individual that intelligence and abilities are changeable, rather than fixed and immutable. In this paper, we present an overview of the growth mindset and how it aligns with the goals of CBME. We describe the challenges associated with shifting away from the fixed mindset of most traditional medical education assumptions and practices and discuss potential solutions and strategies at the individual, relational, and systems levels. Finally, we present future directions for research to better understand the growth mindset in the context of CBME.

Systems-Level Reforms to the US Resident Selection Process: A Scoping Review

BACKGROUND

Calls to reform the US resident selection process are growing, given increasing competition and inefficiencies of the current system. Though numerous reforms have been proposed, they have not been comprehensively cataloged.

OBJECTIVE

This scoping review was conducted to characterize and categorize literature proposing systems-level reforms to the resident selection process.

METHODS

Following Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines, searches of Embase, MEDLINE, Scopus, and Web of Science databases were performed for references published from January 2005 to February 2020. Articles were included if they proposed reforms that were applicable or generalizable to all applicants, medical schools, or residency programs. An inductive approach to qualitative content analysis was used to generate codes and higher-order categories.

RESULTS

Of 10 407 unique references screened, 116 met our inclusion criteria. Qualitative analysis generated 34 codes that were grouped into 14 categories according to the broad stages of resident selection: application submission, application review, interviews, and the Match. The most commonly proposed reforms were implementation of an application cap (n = 28), creation of a standardized program database (n = 21), utilization of standardized letters of evaluation (n = 20), and pre-interview screening (n = 13).

CONCLUSIONS

This scoping review collated and categorized proposed reforms to the resident selection process, developing a common language and framework to facilitate national conversations and change.

Virtual Versus In-Person Surgical Fellowship Interviews and Ranking Variability: the COVID-19 Experience

Objective

To investigate the effect of interview format changes (in-person to virtual, one-to-one to multiple-to-one) necessitated by the COVID-19 travel restrictions on candidate ranking variabilities.

Method

In 2018/2019, the glaucoma fellowship interviews were conducted in-person and one-to-one, whereas in 2020, interviews were virtual and multiple (interviewers)-to-one (candidate). We compared ranking ranges of interviewers within the same virtual room (WSR) and not within the same virtual room (NWSR) to assess the effect of this change on ranking variabilities. We also compared ranking categories ("accept," "alternate," and "pass") agreements between in-person and virtual interviews to assess the effect of this change on ranking variabilities.

Results

NWSR and WSR mean rankings differed by 1.33 (95% confidence interval difference 0.61 to 2.04, p = 0.0003), with WSR interviewers having less variability than NWSR pairs. The variability of in-person interviews and later virtual interviews showed no differences (weighted Kappa statistic 0.086 for 2018, 0.158 for 2019, and 0.101 for 2020; p < 0.05 for all years). The overall least attractive candidate has the lowest variability; the most attractive candidate has the second lowest variability.

Conclusion

Grouping interviewers decreased ranking variabilities, while a change from in-person to virtual interview format did not increase the ranking variabilities.

A virtual COVID-19 ophthalmology rotation

The coronavirus (COVID-19) pandemic temporarily suspended medical student involvement in clinical rotations, resulting in the need to develop virtual clinical experiences. The cancellation of clinical ophthalmology electives and away rotations reduces opportunities for exposure to the field, to network with faculty, conduct research, and prepare for residency applications. We review the literature and discuss the impact and consequences of COVID-19 on undergraduate medical education with an emphasis on ophthalmic undergraduate medical education. We also discuss innovative learning modalities used from medical schools around the world during the COVID-19 pandemic such as virtual didactics, online cases, and telehealth. Finally, we describe a novel, virtual neuro-ophthalmology elective created to educate medical students on neuro-ophthalmology foundational principles, provide research and presentation opportunities, and build relationships with faculty members. These innovative approaches represent a step forward in further improving medical education in ophthalmology during COVID-19 pandemic and beyond.

The Impact of Away Rotations on the Ophthalmology Residency Match

Background The ophthalmology residency match is highly competitive, and fourth-year medical students frequently complete away rotations at other institutions hoping to increase their chances of matching.

Objective The study aimed to determine the impact of away rotations on the ophthalmology residency selection process and to provide recommendations to faculty for counseling fourth-year medical students as they prepare for the ophthalmology match.

Methods A survey was sent in June 2020 to program directors (PDs), medical student educators (MSEs), and residents at 119 accredited ophthalmology residency programs via the Association of University Professors of Ophthalmology e-mail listserv. The 43 PDs and 38 MSEs who responded were asked questions about their views regarding away rotations and visiting students during the residency selection process. The 83 resident respondents were asked about their interview and match success after completing away rotations. Categorical variables were analyzed by Chi-square test.

Results The majority of PDs and MSEs typically recommend that fourth-year medical students complete away rotations (p = 0.022 and p = 0.194, respectively). The majority believe that an away rotation will increase the chance of the applicant matching with the host institution (p = 0.004 and p = 0.001, respectively). The vast majority of PDs were more likely to offer an interview to a visiting student compared with another equally qualified applicant (p < 0.001). However, only one-third of PDs were typically more likely to rank an away rotator higher on their rank list (p = 0.047). Two-thirds of residents completed at least one away rotation in medical school (mean = 1.7 rotations). The majority received interview invitations from at least some of their host institutions (p < 0.001). Approximately one-fifth ultimately matched at a host residency program (p < 0.001).

Conclusion The majority of ophthalmology PDs and MSEs surveyed recommend that fourth-year medical students complete away rotations and believe that these rotations will increase the chance of the applicant matching at a host institution. The PDs are more likely to offer an interview to a visiting student compared with another equally qualified applicant. For some programs, visiting students may also receive special consideration by being ranked higher on a program's rank list and being more likely to match.

Current Applicant Perceptions of the Ophthalmology Residency Match

Importance

The ophthalmology residency application process is critical for applicants and residency programs, and knowledge about the preferences of applicants would assist both groups in improving the process.

Objective

To evaluate the experiences and preferences of ophthalmology residency applicants.

Design, Setting, and Participants

This cross-sectional, nonvalidated survey was conducted online. All applicants to the Bascom Palmer Eye Institute ophthalmology residency program during the 2018-2019 application cycle were invited to complete the survey. Data collection occurred from April 1, 2019, to April 30, 2019.

Main Outcomes and Measures

Applicant demographics, application submissions, interview experiences, financial considerations, match results, and suggestions for improvement of the application process.

Results

Responses were received from 185 applicants (36.4%), including 77 women (41.6%). A successful match into an ophthalmology residency was achieved by 172 respondents (93.0%). There was a mean (SD) US Medical Licensing Examination Step 1 score of 245.8 (13.3) points. Respondents applied to a mean (SD) of 76.4 (23.5) ophthalmology residency programs, received 14.0 (9.0) invitations to interview, and attended 10.3 (4.4) interviews. Choices regarding applications and interviews were based mostly on program reputation, location, and advisor recommendation. A usual lead time of at least 3 weeks between the invitation and interview was reported by 126 respondents (69.2%), which was reduced to 14 respondents (15.1%) when a wait-list was involved. The ophthalmology residency application process cost a mean (SD) of $5704 ($2831) per applicant. Respondents reported that they were most able to reduce costs through housing choices (hotel stays or similar arrangements) and least able to reduce costs by limiting the number of programs to which they applied or at which they interviewed.

Conclusions and Relevance

The ophthalmology residency application process is complex and poses substantial challenges to applicants and residency programs. These findings suggest that many current applicants have difficulty selecting programs to apply to, and most respondents desired changes to the current system of interview invitations and scheduling.

Ophthalmology Program Directors' Perspectives on the Impact of the United States Medical Licensing Examination Step 1 Change to Pass-Fail Scoring

Purpose This article investigates the perspectives of ophthalmology residency program directors (PDs) regarding the impact of the United States Medical Licensing Examination (USMLE) Step 1 change from graded to pass-fail scoring on ophthalmology resident selection and medical education.

Methods The PDs of all United States ophthalmology residency programs accredited by the Accreditation Council for Graduate Medical Education were identified using a public, online database. An anonymous web-based survey constructed using REDCap was emailed to each PD in February 2020.

Results Surveys were completed by 64 (54.2%) PDs, with the majority (81.2%) disagreeing with the change to pass-fail scoring. The majority of PDs believe this change will negatively impact the ability to evaluate residency applicants (92.1%) and achieve a fair and meritocratic match process (76.6%), and will decrease medical students' basic science knowledge (75.0%). The factors identified most frequently by PDs as becoming more important in evaluating residency applicants as a result of the Step 1 scoring change include clerkship grades (90.6%), USMLE Step 2 Clinical Knowledge score (84.4%), and a rotation in the PD's department (79.7%). The majority of PDs believe the Step 1 grading change to pass-fail will benefit applicants from elite medical schools (60.9%), and disadvantage applicants from nonelite allopathic schools (82.8%), international medical graduate applicants (76.6%), and osteopathic applicants (54.7%).

Conclusion The majority of ophthalmology PDs disagree with the change in USMLE Step 1 scoring from graded to pass-fail and believe this change will negatively impact the ability to evaluate residency applicants and achieve a fair and meritocratic match process, and will decrease medical students' basic science knowledge.

Ophthalmology Program Director Perspectives of Scoring Step 1 Pass/Fail

Background Scoring for the United States Medical Licensing Examination (USMLE) Step 1 was recently announced to be reported as binary as early as 2022. The general perception among program directors (PDs) in all specialties has largely been negative, but the perspective within ophthalmology remains uncharacterized.

Objective This article characterizes ophthalmology residency PDs' perspectives regarding the impact of pass/fail USMLE Step 1 scoring on the residency application process.

Methods A validated 19-item anonymous survey was electronically distributed to 111 PDs of Accreditation Council for Graduate Medical Education-accredited ophthalmology training programs.

Results Fifty-six PDs (50.5%) completed the survey. The median age of respondents was 48 years and the majority were male (71.4%); the average tenure as PD was 7.1 years. Only 6 (10.7%) PDs reported the change of the USMLE Step 1 to pass/fail was a good idea. Most PDs (92.9%) indicated that this will make it more difficult to objectively compare applicants, and many (69.6%) did not agree that the change would improve medical student well-being. The majority (82.1%) indicated that there will be an increased emphasis on Step 2 Clinical Knowledge (CK) scores, and many (70.4%) felt that medical school reputation will be more important in application decisions.

Conclusion Most ophthalmology PDs who responded to the survey do not support binary Step 1 scoring. Many raised concerns regarding shifted overemphasis on Step 2 CK, uncertain impact on student well-being, and potential to disadvantage certain groups of medical students including international medical graduates. These concerns highlight the need for reform in the ophthalmology application process.

Reference letters for subspecialty medicine residency positions: are they valuable for decision-making? Results from a Canadian study

BACKGROUND

The letter of recommendation is currently an integral part of applicant selection for residency programs. Internal medicine residents will spend much time and expense completing sub-specialty away electives to obtain a letter of recommendation. The purpose of this study was 1) to examine a large sample of reference letters in order to define essential components of a high-quality letter, and 2) to elucidate the relationship between quality of reference letter and the letter writer.

METHODS

We conducted a two-phase study. In phase one, a large sample of letters of recommendation was examined using an audit tool as a coding framework. A 5-point composite endpoint of high-quality letter components was subsequently developed. In phase two, program director letters were compared to non-program director home institution and non-home institution elective letters based on inclusion of components of the 5-point composite endpoint using Chi square testing.

RESULTS

715 letters were examined (398 non-program director home institution letters, 201 program director letters, and 116 non-home institution elective letters). High-quality letter components were: nature of relationship, duration of relationship, In Training Evaluation Report information, research involvement and comments on areas for improvement. Program director letters had a significantly higher proportion (10.4%) of all 5 high-quality components, compared to 0% in both non-program director home institution letters and elective letters (p < 0.001). A significantly higher proportion of program director letters had 4-5 high-quality components (62.5%) compared to 2% of non-program director home institution letters and 0% of elective letters (p < 0.0001).

CONCLUSIONS

Letters of recommendation from elective rotations are of the poorest quality and such rotations should not be pursued for the sole purpose of obtaining a letter. The low quality of elective letters leads to the recommendation that writers should decline to write them, programs should not require them and trainees should not request them. Program directors write the highest quality letters and should be a resource for faculty development. Clinical supervisors can use the 5-point composite endpoint as a guide when writing letters for applicants.

Ophthalmology Applicant Perceptions of Two Residency Application Services: The San Francisco Match Central Application Service and Electronic Residency Application Service

Purpose Given ophthalmology residency programs are transitioning to include the internship year, either through “joint” or “integrated” 4-year programs, we set out to identify applicant preferences regarding the match and their experiences with two residency application systems: (1) the Central Application Service (CAS) and (2) the Electronic Residency Application Service (ERAS).

Design This study is designed as a retrospective repeated cross-sectional survey.

Methods A 15-question online survey was sent to 196 and 461 applicants to the 2019 and 2020 ophthalmology match cycles, respectively. Questions from the survey assessed user experiences with specific components of both application services and evaluated preferences regarding the future of the ophthalmology match.

Results Responses were received from 208 (32%) applicants. A majority of users had positive experiences with both application services; for CAS, 162 (78%) applicants had a positive experience, compared with 111 (53%) for ERAS. When compared directly, applicants favored the CAS (60%) to ERAS (21%). Furthermore, 108 (52%) respondents stated that they would prefer ophthalmology continue to use both the CAS and ERAS, while 47 (23%) respondents indicated a desire for the CAS to become the only application system for both matches.

Conclusion Although half of all respondents prefer that both the CAS and ERAS systems are utilized for the match process, many express a desire for a single matching program. As ophthalmology residency programs move to joint and integrated 4-year programs, the complexity of matching will increase. Further evaluation of applicant preferences during this transition phase is needed as applicants are required to apply to a variety of different joint and integrated internship and ophthalmology programs.

USMLE Step 2 CK: Best Predictor of Multimodal Performance in an Internal Medicine Residency

BACKGROUND

Internal medicine (IM) residency programs receive information about applicants via academic transcripts, but studies demonstrate wide variability in satisfaction with and usefulness of this information. In addition, many studies compare application materials to only 1 or 2 assessment metrics, usually standardized test scores and work-based observational faculty assessments.

OBJECTIVE

We sought to determine which application materials best predict performance across a broad array of residency assessment outcomes generated by standardized testing and a yearlong IM residency ambulatory long block.

METHODS

In 2019, we analyzed available Electronic Residency Application Service data for 167 categorical IM residents, including advanced degree status, research experience, failures during medical school, undergraduate medical education award status, and United States Medical Licensing Examination (USMLE) scores. We compared these with post-match residency multimodal performance, including standardized test scores and faculty member, peer, allied health professional, and patient-level assessment measures.

RESULTS

In multivariate analyses, USMLE Step 2 Clinical Knowledge (CK) scores were most predictive of performance across all residency performance domains measured. Having an advanced degree was associated with higher patient-level assessments (eg, physician listens, physician explains, etc). USMLE Step 1 scores were associated with in-training examination scores only. None of the other measured application materials predicted performance.

CONCLUSIONS

USMLE Step 2 CK scores were the highest predictors of residency performance across a broad array of performance measurements generated by standardized testing and an IM residency ambulatory long block.

Identifying and Promoting Best Practices in Residency Application and Selection in a Complex Academic Health Network

Medical education institutions have a social mandate to produce a diverse physician workforce that meets the public's needs. Recent reports have framed the admission process outcome of undergraduate and postgraduate medical education (UGME and PGME) programs as a key determinant of the collective contributions graduating cohorts will make to society, creating a sense of urgency around the issue of who gets accepted. The need for evidence-informed residency application and selection processes is growing because of the increasing size and diversity of the applicant pool and the need for equity, fairness, social accountability, and health human resource planning. The selection literature, however, is dominated by a UGME focus and emphasizes determination of desirable qualities of future physicians and selection instrument reliability and validity. Gaps remain regarding PGME selection, particularly the creation of specialty-specific selection criteria, suitable outcome measures, and reliable selection systems.In this Perspective, the authors describe the University of Toronto's centralized approach to defining system-level best practices for residency application and selection. Over the 2012-2013 academic year, the Best Practices in Application and Selection working group reviewed relevant literature and reports, consulted content experts, surveyed local practices, and conducted iterative stakeholder consultations on draft recommendations. Strong agreement arose around the resulting 13 principles and 24 best practices, which had either empirical support or face validity. These recommendations, which are shared in this article, have been adopted by the university's PGME advisory committee and will inform a national initiative to improve trainees' transition from UGME to PGME in Canada.

Use of the Interview in Resident Candidate Selection: A Review of the Literature

BACKGROUND

Although the resident candidate interview is costly and time-consuming for both applicants and programs, it is considered critically important for resident selection. Noncognitive attributes, including communication skills and professionalism, can be assessed by the personal interview.

OBJECTIVE

We conducted a review of the literature on the residency interview to identify the interview characteristics used for resident selection and to ascertain to what extent the interview yields information that predicts future performance.

METHODS

We searched PubMed and Scopus using the following search terms: residency, internship, interview, selection, and performance. We extracted information on characteristics of the interview process, including type of interview format, measures taken to minimize bias by interviewers, and testing of other clinical/surgical skills.

RESULTS

We identified 104 studies that pertained to the resident selection interview, with highly varied interview formats and assessment tools. A positive correlation was demonstrated between a medical school academic record and the interview, especially for unblinded interview formats. A total of 34 studies attempted to correlate interview score with performance in residency, with mixed results. We also identified a number of studies that included personality testing, clinical skills testing, or surgical skills testing.

CONCLUSIONS

Our review identified a wide variety of approaches to the selection interview and a range of factors that have been studied to assess its effectiveness. More research needs to be done not only to address and ascertain appropriate interview formats that predict positive performance in residency, but also to determine interview factors that can predict both residents' "success" and program attrition.

Characteristics That Define a Successful Pharmacy Resident as Perceived by Residency Programs

PURPOSE

To identify the subjective and objective characteristics that pharmacy residency programs use to define a successful resident and to determine what percentage of their 2009-2010 residency class they felt were successful.

METHODS

An electronic survey was sent via e-mail to all residency program directors (RPDs) of postgraduate year 1 (PGY1) and postgraduate year 2 (PGY2) pharmacy residency programs in the United States. A 3-part survey instrument was developed following validation of questions for clarity and reliability using a pilot survey. Respondents were asked to rank the importance of 20 subjective characteristics for a resident to possess in order to be considered successful and the importance of different objective measurements of accomplishment in the definition of a "successful" resident using a Likert scale where 1 = not at all important, 2 = some importance, 3 = very important, and 4 = critical.

RESULTS

Of the 1,081 surveys sent to RPDs, 473 respondents answered at least one question, yielding a response rate of 43.8%. The most critically important subjective characteristics in defining a successful resident as ranked among PGY1 residency programs are dependability, professionalism, self-motivation/initiative, and work ethic. PGY2 programs ranked clinical knowledge and skills, critical thinking, and dependability as the most important. The most critically important objective characteristic in defining a successful resident as ranked among both PGY1 and PGY2 programs is obtaining a clinical position. The majority of PGY1 and PGY2 respondents felt that 76% to 100% of their 2009-2010 residency class was successful based on the characteristics they rated most important.

CONCLUSION

Identification of the characteristics that pharmacy residency programs use to define success will allow them to identify predictors of success and optimal methods of selecting residents who possess these characteristics.