Letters of Recommendation: Association with Interviewers' Perceptions and Preferences

The ADVICER Template for Faculty Reviewer Letters for Promotion and Appointment

RATIONALE AND OBJECTIVES

Promotion is an important milestone in the career of academic radiologists. Appointments, Promotion and Tenure (APT) committees require multiple letters of support from both internal and external referees. Traditional narrative letters are highly subjective, have high inter-reader variability, are time-intensive, and vulnerable to gender and other biases. The Alliance of Directors and Vice Chairs of Education in Radiology (ADVICER) recognized the need for a standardized template to assist academic faculty, letter writers, and APT committees.

MATERIALS AND METHODS

An ADVICER ad hoc committee of six educators with experience serving as external referees was convened to create a standardized template. Committee members performed a search of the relevant literature and internet sites, spoke with stakeholders such as APT chairs, and ultimately developed a template for faculty reviewer letters using the common clinician-educator pathway as a focal point.

RESULTS

An open source, modifiable, standardized, template was produced. The template has been made available to ADVICER members and is available on the Association of University Radiologists (AUR) website at: https://www.aur.org/resources/Template-for-Faculty-Reviewer-Letters-for-Promotion-and-Appointment CONCLUSION: This external referee template has the potential to reduce subjectivity, eliminate bias, and provide a flexible, modifiable, comprehensive faculty review letter template which will be useful for academic faculty, letter writers, and promotions committees.

Improving the Standardized Letter of Recommendation in the Plastic Surgery Resident Selection Process

OBJECTIVE

The American Council of Academic Plastic Surgeons (ACAPS) Plastic Surgery Recommendation Form is a standardized letter of recommendation (SLOR) used in the plastic surgery resident selection process since 2012. This study aims to evaluate this form and use a multistep survey to optimize this SLOR.

DESIGN

A 16 question survey was sent via Survey Monkey to 510 ACAPS faculty. The results of this survey were followed up with a live-response survey at the ACAPS Spring Retreat. Consensus was used to modify and create a more optimal SLOR.

SETTING

This study was performed by the Division of Plastic and Reconstructive Surgery at the University of California, San Diego.

PARTICIPANTS

Survey respondents represented over five hundred plastic surgery faculty from a variety of subspecialties and academic programs nation-wide.

RESULTS

The response rate to the online survey was 36%. On a scale of 1 (not useful) to 5 (very useful), letter readers and letter writers found the SLOR 3.88 and 3.90 respectively. Most respondents believed that the SLOR should be used as an adjunct to the narrative LOR. Forty-two faculties participated as respondents to the live audience questions, although over a 100 faculty were present for the discussion. Most participants agreed that independent program applicants should have a separate SLOR form, with at least 1 filled out by the Program Director of the residents' primary board. The majority (61%) of respondents supported the addition of 1-line characteristics to each applicant quality, and the change to a percentile format for assessment.

CONCLUSIONS

Letters of recommendation are vital to the resident selection process, and thus it is incumbent upon fields of medicine to consciously work on improving how these letters are written. This study outlines a 2-step process by which a SLOR was evaluated and optimized for faculty in the field of plastic surgery.

Ideas and Innovations to Improve the Otolaryngology–Head and Neck Surgery Residency Application and Selection Process

Objective

To (1) summarize strategies proposed in the recent otolaryngology–head and neck surgery (OTO-HNS) literature for improving the residency application and selection process, (2) evaluate the effects of recently implemented changes to the OTO-HNS match, and (3) discuss recommendations for future changes to the OTO-HNS residency application and selection process.

Data Sources

PubMed, Medline Ovid database, and article reference lists.

Review Methods

A structured literature search was performed to identify current English language articles relating to the objectives of this study using the aforementioned data sources, in accordance with the Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) guidelines. The search was limited to submissions published between January 1, 2015, and January 1, 2020.

Conclusions

Numerous proposals have been made for improving the otolaryngology residency application and selection process and addressing the competitive nature of the Match. These proposals include but are not limited to mandating a secondary essay, implementing consortia and early match processes, using a signaling system, conducting regional and web-based interviews, offering early engagement with interest groups, instituting a hard cap on applications, increasing costs of applying, counseling self-restraint to prospective applicants, and creating application filters.

Implications for Practice

As the volume of literature surrounding the OTO-HNS Match continues to increase, this review aims to provide a summary of past proposals and serve as a guide for possible future innovations. We propose 3 initiatives that may improve the residency application and selection process for both program and resident, with minimal impact to the current National Resident Matching Program (NRMP) Match structure.

Physician Workforce Disparities and Patient Care: A Narrative Review

Background: Ensuring the strength of the physician workforce is essential to optimizing patient care. Challenges that undermine the profession include inequities in advancement, high levels of burnout, reduced career duration, and elevated risk for mental health problems, including suicide. This narrative review explores whether physicians within four subpopulations represented in the workforce at levels lower than predicted from their numbers in the general population—women, racial and ethnic minorities in medicine, sexual and gender minorities, and people with disabilities—are at elevated risk for these problems, and if present, how these problems might be addressed to support patient care. In essence, the underlying question this narrative review explores is as follows: Do physician workforce disparities affect patient care? While numerous articles and high-profile reports have examined the relationship between workforce diversity and patient care, to our knowledge, this is the first review to examine the important relationship between diversity-related workforce disparities and patient care.

Methods: Five databases (PubMed, the Cochrane Library of Systematic Reviews, EMBASE, Web of Knowledge, and EBSCO Discovery Service) were searched by a librarian. Additional resources were included by authors, as deemed relevant to the investigation.

Results: The initial database searches identified 440 potentially relevant articles. Articles were categorized according to subtopics, including (1) underrepresented physicians and support for vulnerable patient populations; (2) factors that could exacerbate the projected physician deficit; (3) methods of addressing disparities among underrepresented physicians to support patient care; or (4) excluded (n=155). The authors identified another 220 potentially relevant articles. Of 505 potentially relevant articles, 199 (39.4%) were included in this review.

Conclusions: This report demonstrates an important gap in the literature regarding the impact of physician workforce disparities and their effect on patient care. This is a critical public health issue and should be urgently addressed in future research and considered in clinical practice and policy decision-making.

Are Standardized Letters of Recommendation in Residency Applications Correlated With Objective Data?

OBJECTIVE

To evaluate whether the standardized letter of recommendation (SLOR), commonly used for medical students applying to otolaryngology residency, correlates with objective data in the application.

BACKGROUND

Standardized letters of recommendation using Likert-type scales for different attributes are commonly used by evaluators because of their high interrater reliability and efficiency in preparation and interpretation. Given that these are subjectively scored, it is unknown how well these correlate with objective data.

METHODS

Applications to the University of Cincinnati otolaryngology residency were evaluated in the academic cycle of 2017-2018. Standardized letters of recommendation were scored to determine whether certain attributes were correlated with objective data (United States Medical Licensing Examination [USMLE] scores and number of presentations/publications) provided in their application. Spearman correlations were used to evaluate the strength of the relationship between the subjective score in certain attributes with objective data.

RESULTS

There were 217 applications to the University of Cincinnati that contained SLORs. Of these applications, 474 standardized letters were scored in categories of medical knowledge, research, and commitment to academic medicine. Total publications and presentations were weakly correlated with commitment to academic medicine (0.35, P < .0001, n = 369) and with research (0.44, P < .0001, n = 355). Medical knowledge was weakly correlated with Step 1 scores (0.20, P < .0001) and Step 2 scores (0.18, P = .0002).

CONCLUSIONS

Subjective research and commitment to academic medicine rating scores were weakly correlated with greater academic productivity. Similarly, medical knowledge scores were weakly correlated with Step 1/2 scores. Further research may be needed to assess how to interpret SLOR scores in addition to the information available in an otolaryngology application.

LEVEL OF EVIDENCE

4.

Using word clouds to Re-envision letters of recommendation for residency applicants

OBJECTIVES/HYPOTHESIS

To develop a "word cloud"-based visual letter of recommendation (VLOR) and to evaluate its efficiency in discerning applicant quality compared to narrative letters of recommendation (NLORs).

STUDY DESIGN

Cross-sectional cohort study.

METHODS

NLORs for 48 otolaryngology residency applicants interviewed from the 2016 application cycle were identified and mined for descriptive terms to generate a word cloud, referred to as a VLOR. Eight individuals reviewed and rated a total of 187 blinded NLORs and 48 VLORs on a four-point scale (negative to exceptional). Median VLOR and NLOR scores and the time to review for each candidate were compared using the Wilcoxon signed rank test.

RESULTS

It took significantly more time to review the NLORs in comparison to the VLORs (67 seconds, interquartile range [IQR]: 41-98 seconds vs. 17 seconds, IQR: 11-26 seconds, P < .001). There was no significant difference between median scores for VLORs and NLORs (P = .136). Review time and score correlated positively for VLORs and was statistically significant (ρ = 0.459, P = .001), indicating that more time spent reviewing equates to higher scores. The same relationship appeared with NLORs, but was not statistically significant (ρ = 0.276, P = .058).

CONCLUSIONS

VLORs are a novel and efficient additive tool for screening candidates for otolaryngology residency interview slots. Their scores do not significantly vary from NLOR scores and are significantly faster to evaluate.

LEVEL OF EVIDENCE

2b Laryngoscope, 129:2026-2030, 2019.

Women physicians underrepresented in American Academy of Neurology recognition awards

OBJECTIVE

To investigate representation by gender among recipients of physician recognition awards presented by the American Academy of Neurology (AAN).

METHODS

We analyzed lists of individual recipients over the 63-year history of the AAN recognition awards. Included were awards intended primarily for physician recipients that recognized a body of work over the course of a career. The primary outcome measures were total numbers and proportions of men and women physician award recipients.

RESULTS

During the period studied, the proportion of women increased from 18% (1996) to 31.5% (2016) among AAN US neurologist members and from 18.6% (1992) to 35% (2015) in academia, and the AAN presented 323 awards to physician recipients. Of these recipients, 264 (81.7%) were men and 59 (18.3%) were women. During the most recent 10-year period studied (2008-2017), the proportion of women increased from 24.7% (2008) to 31.5% (2016) among AAN US neurologist members and from 28% (2009) to 35% (2015) in academia, and the AAN presented 187 awards to physician recipients, comprising 146 men (78.1%) and 41 women (21.9%). Although it has been more than 2 decades since the proportion of women among US neurologist members of the AAN was lower than 18%, 1 in 4 AAN award categories demonstrated 0% to 18% representation of women among physician recipients during the most recent decade. Moreover, for highly prestigious awards, underrepresentation was more pronounced.

CONCLUSION

Although the reasons why are not clear, women were often underrepresented among individual physician recognition award recipient lists, particularly for highly prestigious awards.