Development and Assessment of Early Utilization of the Standardized Letter of Recommendation for Use in the Radiology Residency Match

Perceptions and Expectations Regarding the Step 1 Score Change Among Surgical and Non-Surgical Program Directors: A Systematic Review and Meta-Analysis

There are currently no studies examining differences in perceptions and expected impact of the Step 1 score change to pass/fail between surgical and non-surgical program directors (PDs). We conducted a systematic review in May 2023 of PubMed, Scopus, Web of Science, and PSYCInfo to evaluate studies examining PDs' perspectives regarding the Step 1 score change. We performed random-effects meta-analyses to determine differences in perspectives among surgical and non-surgical PDs. Surgical PDs (76.8% [95% CI, 72.1%-82.0%], I2 = 52%) reported significantly greater rates of disagreement with the score change compared to non-surgical (65.1% [95% CI, 57.9%-73.1%], I2 = 69.7%) (P = .01). Surgical PDs also reported significantly greater rates of agreement that the score change will increase the difficulty in objectively comparing applicants (88.1% [95% CI, 84.6%-91.7%], I2 = 16.4%), compared to non-surgical (81.0% [95% CI, 75.6%-86.8%], I2 = 72.6%) (P = .04). There was less heterogeneity among non-surgical PDs (88.7% [95% CI, 86.2%-91.2%], I2 = 0%), compared to surgical (84.7% [95% CI, 79.0%-90.8%], I2 = 67.3%), regarding expected increases in emphasis on Step 2, although the difference in rates of agreement was not statistically significant. Overall, there is significant heterogeneity in the literature regarding expected changes in the residency application review process. Most PDs reported significant disagreement with the score change, greater expected difficulty in objectively evaluating applicants, and greater emphasis on Step 2, with surgical PDs reporting greater rates of disagreement, greater expected difficulty, and heterogeneity regarding expected increases in emphasis on Step 2, compared to non-surgical. Additionally, there is significant heterogeneity in the overall literature regarding expected changes in the residency application review process. Further research is needed to establish evidence-based guidelines that improve the overall residency application process for all stakeholders.

Components of effective letters of recommendation: A cross-sectional survey of academic faculty

INTRODUCTION

Conventional merit-based criteria, including standardized test scores and grade point averages, have become less available to residency programs to help distinguish applicants, making other components of the application, including letters of recommendation (LORs), important surrogate markers for performance. Despite their impact on applications, there is limited published data on LORs in the international setting.

METHODS

A cross-sectional survey of academic faculty was conducted between 9 January 2023 and 12 March 2023 at two large academic medical centers in the United Arab Emirates. Descriptive statistics were used to tabulate variable frequencies.

RESULTS

Of the 98 respondents, the majority were male (n = 67; 68.4%), Western-trained (n = 66; 67.3%), mid-career physicians (n = 46; 46.9%). Most respondents (n = 77; 78.6%) believed that the purpose of an LOR was to help an applicant match into their desired program. Letters rarely included important skills, such as leadership (n = 37; 37.8%), applicant involvement in research (n = 43; 43.9%), education (n = 38; 38.8%), or patient advocacy (n = 30; 30.6%). Most faculty (n = 81; 82.7%) were not familiar with standardized letters of recommendation. Only 7.3% (n = 7) of respondents previously received training in writing LORs, but 87.7% (n = 86) expressed an interest in this professional development opportunity.

CONCLUSION

There is variability in perceptions and practices related to LOR writing in our international setting, with several areas for improvement. Given the increasing importance of LORs to a candidate's application, faculty development is necessary.

Confidence of Faculty in Writing Letters of Recommendation for Pediatric Fellowship Applicants

BACKGROUND

The assessment of pediatric residents applying to subspecialty fellowship programs relies on faculty letters of recommendation (LOR). However, it is unclear if pediatric faculty are confident that their LOR are effective.

OBJECTIVE

This study aims to assess the confidence of pediatric faculty in writing an effective LOR for pediatric residents applying to subspecialty fellowship programs.

METHODS

Survey development was conducted using evidence-based best practices. Surveys were distributed via email in 2021 to all full-time pediatric faculty members who taught pediatric residents in a large academic medical center. Categorical values were compared by chi-square test.

RESULTS

Eighty-five out of 150 (57%) faculty members completed the survey. Forty-one percent of participants were very confident that their LOR provided adequate content to assess residents during the application process. Confidence was associated with higher academic rank (p=0.02), frequent contact with residents (p=0.01), and writing >2 LOR in the last five years (p=0.0002). Confident LOR writers were more likely to describe their own background, details about the resident's scholarly activity, and the resident's ability to work as part of a team. Thirty-five percent of respondents reported never considering gender bias when writing LOR, whereas 28% reported always considering gender bias. Eighty-seven percent of respondents reported an interest in receiving LOR writing guidelines.

CONCLUSION

Half of the faculty respondents were not very confident in their ability to write an effective LOR for pediatric residents applying for a fellowship. Faculty development and standardized instructions on writing effective LOR may be helpful both at the institutional and national levels, including the importance of considering gender bias when writing LOR.