Standardized Letters of Recommendation in Plastic Surgery: The Impact of Gender and Race

Assessment of Gender Differences in Letters of Recommendation for Physical Therapy Residency Applications

INTRODUCTION

Letters of recommendation (LOR) are an integral component of physical therapy residency applications. Identifying the influence of applicant and writer gender in LOR will help identify whether potential implicit gender bias exists in physical therapy residency application processes.

REVIEW OF LITERATURE

Several medical and surgical residency education programs have reported positive, neutral, or negative LOR female gender bias among applicants and writers. Little research exists on gender differences in LOR to physical therapy education programs or physical therapy residency programs.

SUBJECTS

Seven hundred sixty-eight LOR were analyzed from 256 applications to 3 physical therapy residency programs (neurologic, orthopaedic, sports) at one institution from 2014 to 2020.

METHODS

Thematic categories were developed to identify themes in a sample of LOR. Associations between writer and applicant gender were analyzed using summary statistics, word counts, thematic and psycholinguistic extraction, and rule-based and deep learning Natural Language Processing .

RESULTS

No significant difference in LOR word counts were found based on writer or applicant gender. Increased word counts were seen in sports residency LOR compared with the orthopaedic residency. Thematic analysis showed LOR gender differences with male applicants receiving more positive generalized recommendations and female applicants receiving more comments regarding interpersonal relationship skills. No thematic or psycholinguistic gender differences were seen by LOR writer. Male applicants were 1.9 times more likely to select all male LOR writers, whereas female applicants were 2.1 times more likely to choose all female LOR writers.

DISCUSSION AND CONCLUSION

Gender differences in LORs for physical therapy residencies were found using a comprehensive Natural Language Processing approach that identified both a positive recommendation male applicant gender bias and a positive interpersonal relationship skill female applicant gender bias. Applicants were not harmed nor helped by selecting LOR writers of the opposite gender. Admissions committees and LOR writers should be mindful of potential implicit gender biases in LOR submitted to physical therapy residency programs.

Diversity, Equity, and Inclusion in Plastic Surgery: A Systematic Review

BACKGROUND

The number of publications on the subject of diversity, equity, and inclusion has surged in the last 5 years. However, a systematic review of this topic has not been published.

METHOD

Six top plastic surgery journals were queried from 2018 to 2023 using the search term "diversity." Methods, conclusions, and recommendations were tabulated.

RESULTS

A total of 138 publications were identified; 68 studies presented data suitable for analysis. All studies were retrospective. Currently, over 40% of plastic surgery residency applicants are women. The proportion of women in integrated plastic surgery residents is now 43%. In 2021 and 2022, the percentage of female first-year residents exceeded men. The percentage of female presenters at meetings (34%) is double the number in the workforce (17%). Twenty-five percent of academic faculty positions and 22% of program director positions are now held by women. Underrepresented minorities account for fewer than 10% of applicants to integrated plastic surgery residencies.

DISCUSSION

The proportion of Black and Hispanic applicants to integrated plastic surgery residencies (6% and 8%, respectively) mirrors the proportion of Black and Hispanic medical students (7% and 6%, respectively). Numerous recommendations have been made to increase the proportion of underrepresented minorities in plastic surgery programs.

CONCLUSIONS

The representation of women in plastic surgery has increased dramatically. A lack of Hispanics and Blacks reflects a small pool of applicants, as opposed to a "leaky pipeline."

Deconstructing the Excellent Plastic Surgeon: A Survey of Key Attributes

Background

Most plastic surgeons practice in nonacademic settings, leaving a small subset of academic plastic surgeons with the responsibility of selecting the future generation of plastic surgeons without representation from a majority of our field. This raises questions as to whether the academic attributes valued during residency selection are valid predictive markers of who will become an excellent plastic surgeon. A survey was conducted of both academic and nonacademic plastic surgeons, as well as trainees, to determine what traits are considered most essential to being an excellent plastic surgeon.

Methods

An electronic survey was distributed before the American Council of Academic Plastic Surgeons 10th Annual Winter Meeting. Demographics and information regarding the respondents' training and academic status were collected. Respondents were asked to select five traits that they considered most important to be an excellent plastic surgeon from a list of 20 preselected traits. Chi-square and Fisher exact tests were used to perform subgroup analyses.

Results

A total of 187 responses were received from meeting attendees, representing an 89.0% response rate. Overall, the five values endorsed as most important for a plastic surgeon were being technically sound (53%), collaborative (48%), ethical (44%), compassionate (37%), and emotionally intelligent (33%). However, the emphasis placed on these different attributes differed significantly amongst different demographic groups.

Conclusion

It is important that we use methods such as holistic review when evaluating plastic surgery applicants to ensure our selection process is congruent with the traits we value.

A SWOT Analysis of Hot Topics in Plastic Surgery Resident Education: Consensus From the ACAPS 10th Annual Winter Meeting

Background

With the aim of facilitating a critical self-reflection on how to align plastic surgery education with making excellent plastic surgeons, a rotating small-group session followed by live interactive audience polling was used to perform a SWOT (strengths, weaknesses, opportunities, and threats) analysis at the 10th Annual American Council of Academic Plastic Surgeons Winter Meeting.

Methods

The final day of the conference included a 3-hour session of rotating small groups followed by live interactive audience polls discussing the following six relevant educational topics: the Plastic Surgery Common Application and resident selection, aesthetic surgery education, leadership development and business education, embedded fellowships and focused training, mentorship, and faculty retention.

Results

A total of 60 individuals participated in the activity. A SWOT analysis was successfully performed for each educational topic, and a minimum of four opportunities were identified per topic to help guide future endeavors. Examples of opportunities include releasing recommendations for the implementation of holistic review; developing formal guidelines for aesthetic surgery education in residency via collaboration between ACAPS, American Society of Plastic Surgeons, and The Aesthetic Society; creating extended focused elective rotations; integrating business education into formal curricula for all training levels; enforcing transparency regarding position expectations and offerings including salary, call schedule, and current challenges; and more.

Conclusion

The results of this study will help guide future initiatives by the ACAPS to improve resident education and academic retention.

Confidence of Faculty in Writing Letters of Recommendation for Pediatric Fellowship Applicants

BACKGROUND

The assessment of pediatric residents applying to subspecialty fellowship programs relies on faculty letters of recommendation (LOR). However, it is unclear if pediatric faculty are confident that their LOR are effective.

OBJECTIVE

This study aims to assess the confidence of pediatric faculty in writing an effective LOR for pediatric residents applying to subspecialty fellowship programs.

METHODS

Survey development was conducted using evidence-based best practices. Surveys were distributed via email in 2021 to all full-time pediatric faculty members who taught pediatric residents in a large academic medical center. Categorical values were compared by chi-square test.

RESULTS

Eighty-five out of 150 (57%) faculty members completed the survey. Forty-one percent of participants were very confident that their LOR provided adequate content to assess residents during the application process. Confidence was associated with higher academic rank (p=0.02), frequent contact with residents (p=0.01), and writing >2 LOR in the last five years (p=0.0002). Confident LOR writers were more likely to describe their own background, details about the resident's scholarly activity, and the resident's ability to work as part of a team. Thirty-five percent of respondents reported never considering gender bias when writing LOR, whereas 28% reported always considering gender bias. Eighty-seven percent of respondents reported an interest in receiving LOR writing guidelines.

CONCLUSION

Half of the faculty respondents were not very confident in their ability to write an effective LOR for pediatric residents applying for a fellowship. Faculty development and standardized instructions on writing effective LOR may be helpful both at the institutional and national levels, including the importance of considering gender bias when writing LOR.

Letters of recommendation for gynecologic oncology fellowship differ by gender and race

INTRODUCTION

Letters of recommendation (LOR) play a significant role in applicant selection for gynecologic oncology (GO) fellowship. LORs can be agentic, associated with competence, independence and strength, or communal, associated with being accommodating, cooperative and empathetic. Agentic LORs have been shown to reflect favorably on applicants, making them more likely to be selected for interview. The primary goal of this study was to determine linguistic differences in LORs for applicants applying to GO fellowship based on applicant gender and race.

METHODS

All applications to a single academic institution from the years 2018-2020 were analyzed. LORs were extracted from all applicants who self-reported their gender and race/ethnicity. Analysis was performed using Linguistic Inquiry Word Count (LIWC) in terms of agentic and communal language. All analysis was done using SAS version 9.4.

RESULTS

From 2018 to 2020, there were 239 GO fellowship applications to a single academic institution. There were 186 (78.1%) applicants who identified as female and 52 (21.8%) who identified as male. Female applicants were more likely to have more research (p = 0.047) and volunteer activities (p = 0.02) than male applicants. There were no differences between female and male applicants in terms of age, geographic location of residency, USMLE scores, AOA status, number of publications or attending a Doximitiy-rated top 20 residency program. There were 139 (60.7%) applicants who identified as White, 46 (20.1%) as Asian, 16 (7.0%) as Black, 19 (8.3%) as Latinx and 9 (3.9%) as other. On multivariable analysis, LORs written for male applicants had higher agentic scores compared to females (p < 0.05), and LORs of female applicants received higher communal scores (p < 0.05). LORs for applicants who identified as Black or Latinx were found to have lower agentic scores compared to individuals who identified as White(p < 0.05 for both).

CONCLUSION

There were demonstrated linguistic differences in LOR for GO fellowship based on applicant gender and race/ethnicity. LORs written for white, male applicants were more likely to have higher agentic scores compared to minority, female applicants. LORs written for female applicants were more likely to have higher communal scores. As LORs have been shown to be instrumental in GO fellowship interview selection and linguistic differences exist based on applicant gender and race/ethnicity, this study highlights the need for a more objective GO fellowship application process.

DISCLOSURES

Portion of this paper was presented at the Society for Gynecologic Oncology as part of a focus plenary.

The Association Between Interview Day and Rank Order in Plastic Surgery Match: Is Recency Effect to Blame?

INTRODUCTION

Plastic and reconstructive surgery is among the most competitive specialties in the National Resident Matching Program match. Though efforts to institute unbiased and equitable measures of an applicant's success have been made, many barriers still hinder suitable applicants from successfully matching. We sought to identify whether interview day influenced applicants' likelihood of being ranked favorably in both independent and integrated plastic surgery residency programs at a single academic institution.

METHODS

Data from 10 years of independent plastic surgery applicants and 8 years of integrated plastic surgery applicants were queried. Data regarding whether applicants were interviewed on day 1, day 2, or during subinternships (integrated cohort only) and what number they were on the programs rank list were included in the analysis.

RESULTS

A total of 226 independent applicants 237 integrated applicants were identified. For integrated applicants, those who interviewed on day 1 were weighted toward worse rank scores. Applicants who interviewed during their subinternship had a bimodal distribution either ranking favorably or poorly. Integrated applicants who interviewed on the second day were more likely to be ranked in the first quartile. For those who interviewed on day 1, the odds of being ranked in the last quartile was 2.34 times higher than those who interviewed on day 2 (p = 0.02).

CONCLUSIONS

Our results demonstrating that interview day may influence an applicant's final rank in the MATCH. Further study is needed to determine if this effect is can be observed in other academic plastic surgery programs.

The Utility of the Plastic Surgery Standardized Letter of Recommendation Form in Predicting Residency Match Outcomes

BACKGROUND

Letters of recommendation play an important role in resident selection. While plastic surgery's Standardized Letter of Recommendation (SLOR) form most commonly serves as an adjunct to narrative letters, the SLOR provides objective data in the review process and could eventually replace narrative letters. The utility of the SLOR in predicting Match outcomes has not been studied.

METHODS

Applicant data from 225 first-time residency applicants in 2020-21 were collected. Logistic regression modeling was used to predict Match outcomes. This model was validated using 100 randomly selected applicants from 2019-20.

RESULTS

Rank placement (SLOR Question 6) was the most important factor when predicting Match outcomes (p<0.0001). All other SLOR questions were not predictive and subject to notable score inflation. No SLOR score differences were noted based on race; female applicants were rated higher in two of ten domains (p<0.05).

CONCLUSIONS

One question on the plastic surgery SLOR was highly predictive of an applicant matching. However, the remaining SLOR questions had little utility and were subject to gross score inflation. Further work should be done to optimize the utility of the SLOR in differentiating applicants. This has important implications in ensuring the selection of professional, competent residents according to the aims of the Accreditation Council of Graduate Medical Education.

Objective Residency Applicant Assessment Using a Linear Rank Model

OBJECTIVES

Residency applicant assessment is imperfect, with little objectivity built into the process, which, unfortunately, impacts recruitment diversity. Linear rank modeling (LRM) is an algorithm that standardizes applicant assessment to model expert judgment. Over the last 5 years, we have used LRM to assist with screening and ranking integrated plastic surgery (PRS) residency applicants. This study's primary objective was to determine if LRM scores are predictive of match success and, secondarily, to compare LRM scores between gender and self-identified race categories.

DESIGN

Data was collected on applicant demographics, traditional application metrics, global intuition rank, and match success. LRM scores were calculated for screened and interviewed applicants, and scores were compared by demographic groups. Univariate logistic regression was used to evaluate the association of LRM scores and traditional application metrics with match success.

SETTING

University of Wisconsin, Division of Plastic and Reconstructive Surgery. Academic institution.

PARTICIPANTS

Six hundred seventeen candidates who applied to a single institution over 4 application cycles (2019-2022).

RESULTS

Using area under the curve modeling, LRM score was the most predictive indicator for match success. With every one-point increase in LRM score, there was an 11% and 8.3% increase in the likelihood of screened and interviewed applicant match success (p < 0.001). An algorithm was developed to estimate the probability of match success based on LRM score. No significant differences in LRM scores were appreciated for interviewed applicant gender or self-identified race groups.

CONCLUSIONS

LRM score is the most predictive indicator of match success for PRS applicants and can be used to estimate an applicant's probability of successfully matching into an integrated PRS residency. Furthermore, it provides a holistic evaluation of the applicant that can streamline the application process and improve recruitment diversity. In the future, this model could be applied to assist in the match process for other specialties.

Approaches to address bias in letters of recommendation

Letters of recommendation are ubiquitous in the research enterprise. Requesting, writing, and reviewing letters of recommendation are all fraught with bias, especially for individuals from groups historically excluded from research environments. We detail how letter reviewers, requesters, and writers can make letters of recommendation a more equitable tool to evaluate scientists.

Recruitment of the Next Generation of Diverse Hand Surgeons

Hand surgery encompasses a diaspora of pathology and patients, but the surgeons treating this population are not commensurately diverse. A physician population that reflects the population it treats consistently leads to improved patient outcomes. Despite increasing diversity amongst surgeons entering into pipeline specialties such as General Surgery, Plastic Surgery, and Orthopaedic Surgery, the overall makeup of practicing hand surgeons remains largely homogenous. This article outlines organizations, such as the Perry Initiative, which have increased recruitment of women and underrepresented minorities into pipeline programs. Techniques of minimizing bias and increasing opportunities for underrepresented groups are also discussed.

Gender Differences in Letters of Recommendation for Gynecology Surgical Fellowship Applicants: Analysis of One Program's Recruitment Cycle

STUDY OBJECTIVE

To examine whether there are gender differences in letters of recommendation (LORs) written for residents applying to gynecology surgical fellowships.

DESIGN

Retrospective study.

SETTING

Single, academic institution.

PATIENTS

LORs for applicants to gynecology oncology, urogynecology, and minimally invasive gynecology fellowships during the 2019-2020 application cycle.

INTERVENTIONS

Not applicable.

MEASUREMENTS AND MAIN RESULTS

We analyzed the linguistic content of the letters for the presence of 4 summary variables and 21 word categories based on previous studies using validated computerized text analysis software. We used multivariable analysis using linear mixed models to compare linguistic characteristics of letters by applicant gender. We performed qualitative content analysis on letters and compared the frequency of code themes by gender. The mixed-method design was planned to allow for analysis of domains not captured in text analysis. Among 680 letters written for 186 applicants, 124 (18.2%) were written for men, and 556 (81.8%) were written for women. There were no differences in the least square mean (standard error) word counts for LORs written for men and women applicants, 465 (20.0) vs 458(9.4) words, p = .74. In multivariable analysis, LORs written for men were found to have higher authentic tone and more risk words (p = .005 and p = .03, respectively). LORs written for women contained more communal (relationship-oriented) words (p = .006). The qualitative analysis demonstrated that ability, interpersonal traits, surgical skills, and research were the most often mentioned themes. Comments about compassion/empathy, leadership potential, teaching, interpersonal skills, and patient rapport were found more often in letters for men. More doubt raisers (words that raise doubt or concern) were present in letters for men, but letters for both genders had similar levels of negative criticism. In contrast, comments on ability, being "drama-free," and self-awareness were found more often in letters for women.

CONCLUSION

There were gender differences in LORs written for obstetrics and gynecology surgical subspecialty fellowship applicants indicating the presence of gender bias.

Standardized Letter of Recommendation: Can Everyone Be Awesome?

OBJECTIVE

Standardized letters of recommendation (SLOR) are hypothesized to decrease bias and provide consistent domains for evaluation. However, their ability to differentiate among applicants is unknown. The utilization and functionality of SLOR and the impact of SLOR domain rating on matching for colon and rectal surgery (CRS) residency applicants have yet to be assessed.

DESIGN

Descriptive statistics and bivariate analysis were employed. Applicants were categorized into 3 groups; Top-tier(TT): applicants rated 100% Excellent/Very Good; Mid-Tier(MT) applicants rated 80-99% Excellent/Very Good; and Non-Top Tier(NTT) applicants rated <80% Excellent/Very Good.

SETTING

University of Hospitals Cleveland Medical Center.

PARTICIPANTS

SLORs submitted to a single colorectal surgery residency in 2019 were analyzed RESULTS: A total of 101 applicants were included, 54 (53.5%) of the applicants were male. 75 (74.2%) applicants who applied to our residency matched into a CRS residency, compared to the national rate of 66%. Of the 101 applicants with SLOR, 54 (53.5%) were categorized as TT, 26 (25.7%) as MT, and 21(20.8%) as NTT. The univariable analysis demonstrated a statistically significant difference in research experience (p=0.029) and match status (p=0.01) between applicant tiers. There were no statistically significant differences between applicant-tier and demographics, foreign medical graduates (FMG), H-indices, ABSITE scores, type of residency, preliminary year, completing an unaccredited CRS, and applicants with an additional degree. On multivariable analysis age (OR=0.65; CI=0.48-0.87) and FMG applicants (OR=0.05; CI=0.01-0.44) were inversely associated with successfully matching. Compared to TT applicants, MT (OR=0.07; CI=0.01-0.57) and NTT (OR=0.04; CI=0.01-0.34) applicants were inversely associated with a successful match. Individuals who completed research prior to residency but after medical school was associated with successfully matching (p=0.009).

CONCLUSIONS

The presence of MT and NTT ratings is associated with failure to match and may represent an area of concern for CRS programs rather than a tool to discern differences between candidates.

Trends in utilization of standardized letters of recommendation in the 2021-2022 neurosurgery application process

OBJECTIVE

Standardized letters of recommendation (SLORs) were introduced during the 2020-2021 neurosurgery residency application cycle, but suffered from poor interrater reliability and grade inflation. Changes were made to the SLOR template and utilization patterns in response to these shortcomings. The authors examined the second year of SLOR utilization. They hypothesized that grade inflation and interrater reliability would be improved from the first iteration. They also hypothesized that increased numbers of letters by single writers would correlate with broader rating distributions.

METHODS

This cross-sectional study analyzed all SLORs submitted to a single neurosurgery residency program over the 2021-2022 cycle. Data from 7 competency domains and the overall rating were recorded and stratified by academic category of the letter writer. Interrater reliability was evaluated using Krippendorff's alpha. The frequency of letters written was evaluated using the Kruskal-Wallis H test.

RESULTS

Ninety percent of SLORs rated applicants among the top 25%, but there was a significant decrease in the usage of the top 1% and top 2%-5% ratings. Interrater reliability was poor across all competencies. Writers who completed 1 SLOR rated applicants higher and had a narrower range than those who completed multiple SLORs.

CONCLUSIONS

Changes in the format and subsequent utilization patterns of SLORs have slightly decreased grade inflation; however, interrater reliability remains poor. The most wide-ranging evaluators submitted the highest number of SLORs, suggesting that future evaluation and usage of SLORs should emphasize letter-writer characteristics and numbers of SLORs written. Overall, SLORs have been well and broadly accepted with subtle improvements in the second year of utilization.