Improving the Standardized Letter of Recommendation in the Plastic Surgery Resident Selection Process

Diversity, Equity, and Inclusion in Plastic Surgery: Where Do We Stand? Highlighting Overlooked Issues in Efforts to Improve Representation

OBJECTIVE

The lack of diversity among applicants successfully matching into integrated plastic surgery residency continues to be a topic of significant discussion, yet substantial improvements in representation remain elusive. This article aims to highlight the systemic barriers contributing to the decline in diverse plastic surgery trainees.

DESIGN

This perspective summarizes current and existing challenges in improving recruitment efforts of underrepresented in medicine (URiM) trainees and those from socioeconomically disadvantaged backgrounds.

CONCLUSIONS

While acknowledging current DEI initiatives, the authors point out those underappreciated modifiable and nonmodifiable obstacles that perpetuate the lack of equity and inclusivity in the match process. We emphasize the importance of continued commitment from program directors and chairs to these initiatives and its generational augmentative effects on achieving true and lasting diversity and equity.

Do certain phrases in letters of recommendation predict success in fellowship performance?

BACKGROUND

Letters of recommendation (LORs) are a valued, yet imperfect tool. Program directors (PDs) score phrases such as give my highest recommendation and top 5 to 10% of students as positive. Although positive phrases are valued by PDs, there is no evidence that these phrases predict performance. We attempt to identify whether 12 specific phrases found in letters of recommendation predict future performance of fellows.

METHODS

LORs were evaluated for 12 select phrases and statements. Alpha Omega Alpha (AOA) status, Step 2 Clinical Knowledge (CK) score, and whether the letter writer was personally known to our admission's committee were also categorized. Logistic regressions were performed to evaluate the relationship of the independent variables with fellow performance.

RESULTS

Using multivariate logistic regression, one of the best residents (OR = 4.02, 95% CI (1.0, 15.9), p < 0.05), exceeds expectations (OR = 4.74, 95% CI (1.4, 16.3), p = 0.01), and give my highest recommendation (OR = 3.87, 95% CI (1.3, 11.7), p = 0.02) predicted positive performance. Highly recommend (OR = 0.31, 95% CI (0.1, 1.0), p < 0.05) and top 5 to 10% (OR = 0.05, 95% CI (0.0, 0.6), p = 0.02) predicted negative performance. The remaining phrases did not correlate to fellowship performance.

CONCLUSION

The current LOR evaluation process may place undo importance on phrases that have limited bearing on a candidate's success in training. Training both letter readers and writers to avoid using coded language or avoid assigning improper importance to select phrases may help improve the candidate selection process.

Optimization of a standardized letter of recommendation for faculty who wish to support candidates applying to surgical training programs

Letters of recommendation (LORs) play an important role in applicant selection for graduate medical education programs. LORs may be of increasing importance in the evaluation of applicants given the recent change of the USMLE Step 1 to pass/fail scoring and the relative lack of other objective measures by which to differentiate and stratify applicants. Narrative letters of recommendation (NLORs), although widely used, have certain limitations, namely variability in interpretation, introduction of gender/race bias, and performance inflation. Standardized letters of recommendation (SLOR) have been proposed as a potential corrective to these limitations. We conducted a series of semi-structured interviews and focus groups to gather perspectives from letter writers and readers to inform methods for improving information elicited by SLORs from which we collected and analyzed data using the constant comparative method and a process of iterative coding. We applied our findings to the development of a novel SLOR for use in surgical residency program applications and were subsequently invited to help revise existing SLORs for a surgical post-graduate training program.

Language Differences by Race on Letters of Recommendation for the Pediatric Surgery Match

OBJECTIVE

This study aims to evaluate if there is any significant linguistic difference in LoR based on applicant's race/ethnicity.

DESIGN

Retrospective review of applications to pediatric surgery fellowship at a single institution (2016-2020). Race was self-reported by applicants. LoR were analyzed via the Linguistic Inquiry and Word Count (LIWC) software program.

SETTING

Johns Hopkins All Children's Hospital, St. Petersburg, Florida USA. A free-standing tertiary pediatric hospital.

PARTICIPANTS

Pediatric surgery fellowship applicants from 2016 to 2020.

RESULTS

A total of 1086 LoR from 280 applicants (52% female) were analyzed. Racial distribution was Caucasians 62.1%, Asian 12.1%, Hispanics 7.1%, multiracial 6.4% African Americans 5%, and other/unknown 7.1%. Letter writers were largely male (84%), pediatric surgeons (63%) and professors (57%). There was no difference in LoR word count across races. LoR for female multiracial candidates contained higher use of affiliation and negative emotion terms compared to Hispanic females (p = 0.002 and 0.048, respectively), and past focus terms when compared to Caucasian and Asian female applicants (p < 0.001 and p = 0.003, respectively). Religion terms were more common in LoR for Asian females when compared to Caucasian females (p < 0.001).

CONCLUSION

This study demonstrates linguistic differences in LoR for pediatric surgery training programs based on applicant race/ethnicity. While differences are present, these do not suggest overt bias based on applicants race or ethnicity.

A SWOT Analysis of Hot Topics in Plastic Surgery Resident Education: Consensus From the ACAPS 10th Annual Winter Meeting

Background

With the aim of facilitating a critical self-reflection on how to align plastic surgery education with making excellent plastic surgeons, a rotating small-group session followed by live interactive audience polling was used to perform a SWOT (strengths, weaknesses, opportunities, and threats) analysis at the 10th Annual American Council of Academic Plastic Surgeons Winter Meeting.

Methods

The final day of the conference included a 3-hour session of rotating small groups followed by live interactive audience polls discussing the following six relevant educational topics: the Plastic Surgery Common Application and resident selection, aesthetic surgery education, leadership development and business education, embedded fellowships and focused training, mentorship, and faculty retention.

Results

A total of 60 individuals participated in the activity. A SWOT analysis was successfully performed for each educational topic, and a minimum of four opportunities were identified per topic to help guide future endeavors. Examples of opportunities include releasing recommendations for the implementation of holistic review; developing formal guidelines for aesthetic surgery education in residency via collaboration between ACAPS, American Society of Plastic Surgeons, and The Aesthetic Society; creating extended focused elective rotations; integrating business education into formal curricula for all training levels; enforcing transparency regarding position expectations and offerings including salary, call schedule, and current challenges; and more.

Conclusion

The results of this study will help guide future initiatives by the ACAPS to improve resident education and academic retention.

The Utility of the Plastic Surgery Standardized Letter of Recommendation Form in Predicting Residency Match Outcomes

BACKGROUND

Letters of recommendation play an important role in resident selection. While plastic surgery's Standardized Letter of Recommendation (SLOR) form most commonly serves as an adjunct to narrative letters, the SLOR provides objective data in the review process and could eventually replace narrative letters. The utility of the SLOR in predicting Match outcomes has not been studied.

METHODS

Applicant data from 225 first-time residency applicants in 2020-21 were collected. Logistic regression modeling was used to predict Match outcomes. This model was validated using 100 randomly selected applicants from 2019-20.

RESULTS

Rank placement (SLOR Question 6) was the most important factor when predicting Match outcomes (p<0.0001). All other SLOR questions were not predictive and subject to notable score inflation. No SLOR score differences were noted based on race; female applicants were rated higher in two of ten domains (p<0.05).

CONCLUSIONS

One question on the plastic surgery SLOR was highly predictive of an applicant matching. However, the remaining SLOR questions had little utility and were subject to gross score inflation. Further work should be done to optimize the utility of the SLOR in differentiating applicants. This has important implications in ensuring the selection of professional, competent residents according to the aims of the Accreditation Council of Graduate Medical Education.

Matching into Plastic Surgery: Insights into the Data

Matching into plastic and reconstructive surgery residency, whether integrated or independent, is a competitive process. This article serves to examine, review, and summarize data published on both residency application processes. Through the results summarized, applicants should carefully review the objective and subjective factors used in evaluation to strategize how to make themselves as competitive as possible. Specifically, though, early review of the data presented may allow applicants time to focus on what modifiable factors still remain before their respective application seasons.

Standardized Letters of Recommendation in Plastic Surgery: The Impact of Gender and Race

BACKGROUND

In the last few decades, standardized letters of recommendation have seen increasing popularity among various medical specialties and have been shown to demonstrate score inflation and gender bias in some select specialties. Through analysis of standardized letter of recommendation scoring patterns, this study evaluated the impact of race and gender on performance in standardized letters of recommendation and provides suggestions for programs to optimally interpret standardized letters of recommendation and minimize bias when writing letters of recommendation.

METHODS

Available standardized letters of recommendation were those from interviewed applicants to the integrated plastic surgery program between 2015 and 2018 (82 total applicants) and all applicants in the year of 2019 (241 total applicants). Various standardized letters of recommendation component scores were analyzed by demographics of both applicants and letter writers, including descriptive statistics, analysis of variance, t testing, and Fleiss kappa for interrater reliability using SPSS software (IBM, Armonk, N.Y.).

RESULTS

No scoring category had an average score below the eighty-fifth percentile. Male letter writers scored male applicants significantly higher in Overall, Conscientiousness, Self-Initiative, and Academic Performance components, whereas female letter writers scored female applicants higher in Communication and Overall components. In the 2019 cohort, applicants of a minority race received lower scores on average in nine out of 10 categories, a difference that was significant in the Team Player category.

CONCLUSIONS

Score inflation pervades standardized letters of recommendation scoring in plastic surgery. Work must be done by letter writers to help mitigate racial and gender biases that influence recommendations for resident selection in plastic surgery.

Attitudes towards and impact of letters of recommendation for anesthesiology residency applicants

Background: This survey aims to identify the relative value and the critical components of anesthesiology letters of recommendation(LORs) from the perspective of Program Directors (PDs) and Associate/Assistant Program Directors (APDs). Knowledge and insights originating from this survey might add to the understanding of the anesthesiology residency selection process and mitigate unintended linguistic biases.Methodology: Anonymous online surveys were sent to anesthesiology PDs/APDs from the Accreditation Council for Graduate Medical Education (ACGME) accredited anesthesiology residency Programs in the USA (US), as listed on the ACGME website and the American Medical Association Fellowship and Residency Electronic Interactive Database (AMA FREIDA) Residency Program Database. The survey authors were blinded to the identity of the respondents.Results: 62 out of 183 (33.8%) invited anesthesiology PDs/APDs completed the survey anonymously. In our survey, LORs are reported as more important in granting an interview than in making the rank list. 64% of respondents prefer narrative LORs. 77.4% of respondents look for specific keywords in LORs. Keywords such as 'top % of students' and 'we are recruiting this candidate' indicate a strong letter of recommendation while keywords such as 'I recommend to your program' or non-superlative descriptions indicate a weak letter of recommendation. Other key components of LORs include the specialty of the letter-writer, according to 84% of respondents, with anesthesiology as the most valuable specialty. Although narrative LORs are preferred, 55.1% of respondents are not satisfied with the content of narrative LORs.Conclusion: LORs containing specific keywords play an important role in the application to anesthesiology residency, particularly when submitted by an anesthesiologist. While narrative LORs are still the preferred format, most of our respondents feel they need improvements. The authors suggest specific LOR improvements including creating formalized LOR training, adding a style guide, and applying comparative scales, with standardized vocabulary in the narrative LOR.

Staying Close to Home: The Effects of COVID-19 on the Plastic Surgery Residency Match

The 2019 novel coronavirus created unique challenges for the integrated plastic surgery match. The goal of this study was to evaluate the trends of the 2020 and 2021 integrated plastic surgery match specifically related to the 2019 novel coronavirus.