Plagiarised letters of recommendation submitted for the National Resident Matching Program

Artificial Intelligence-Generated Writing in the ERAS Personal Statement: An Emerging Quandary for Post-graduate Medical Education

OBJECTIVE

This study was designed to investigate if artificial intelligence (AI) detection software can determine the use of AI in personal statements for residency applications.

METHOD

Previously written personal statements were collected from physicians who had already matched to residency through the Electronic Residency Application System. Physicians were recruited for the study through collegial relationships and were given study information via email. The study team constructed five parallel statements from the shared personal statements to prompt AI to create a personal statement of similar content. An online AI detection tool, GPTZero, was used to assess all the personal statements. Statistical analyses were conducted using R. Descriptive statistics, t-tests, and Pearson correlations were used to assess the data.

RESULTS

Eight physicians' statements were compared to eight AI-generated statements. GPTZero was able to correctly identify AI-generated writing, assigning them significantly higher AI probability scores compared to human-authored essays. Human-generated statements were considered more readable, used shorter words with fewer syllables, and had more sentences compared to AI-generated essays. Longer average sentence length, low readability scores, and high SAT word percentages were strongly associated with AI-generated essays.

CONCLUSIONS

This study shows the capacity of GPTZero to distinguish human-created versus AI-generated writing. Use of AI can pose significant ethical challenges and carries a risk of inadvertent harm to certain applicants and erosion of trust in the application process. Authors suggest standardization of protocol regarding the use of AI prior to its integration in post-graduate medical education.

Components of effective letters of recommendation: A cross-sectional survey of academic faculty

INTRODUCTION

Conventional merit-based criteria, including standardized test scores and grade point averages, have become less available to residency programs to help distinguish applicants, making other components of the application, including letters of recommendation (LORs), important surrogate markers for performance. Despite their impact on applications, there is limited published data on LORs in the international setting.

METHODS

A cross-sectional survey of academic faculty was conducted between 9 January 2023 and 12 March 2023 at two large academic medical centers in the United Arab Emirates. Descriptive statistics were used to tabulate variable frequencies.

RESULTS

Of the 98 respondents, the majority were male (n = 67; 68.4%), Western-trained (n = 66; 67.3%), mid-career physicians (n = 46; 46.9%). Most respondents (n = 77; 78.6%) believed that the purpose of an LOR was to help an applicant match into their desired program. Letters rarely included important skills, such as leadership (n = 37; 37.8%), applicant involvement in research (n = 43; 43.9%), education (n = 38; 38.8%), or patient advocacy (n = 30; 30.6%). Most faculty (n = 81; 82.7%) were not familiar with standardized letters of recommendation. Only 7.3% (n = 7) of respondents previously received training in writing LORs, but 87.7% (n = 86) expressed an interest in this professional development opportunity.

CONCLUSION

There is variability in perceptions and practices related to LOR writing in our international setting, with several areas for improvement. Given the increasing importance of LORs to a candidate's application, faculty development is necessary.

Gender bias in colorectal surgery fellowship letters of recommendation

BACKGROUND

As leaders strive to create equitable surgical pipelines, one process under scrutiny is letters of recommendation (LORs). We sought to review the Colon and Rectal Surgery (CRS) Resident Candidate Assessment questionnaire and LORs for gendered differences.

METHODS

This retrospective observational study of letters of recommendation to CRS fellowship during the 2018-2019 application cycle utilized linguistic Inquiry and Word Count (LIWC2015) software to assess letter length and themes comparing differences by applicant and referee gender.

RESULTS

103 applicants (35 ​% women) with 363 LORs (16 ​% written by women) were included. Short answer responses were longer for women applicants, while LORs were longer for men applicants (368 vs 325 words p ​= ​0.03). Men applicants' strengths had more technical skill descriptors, while women applicants' strengths had more emotional language and cognitive and perceptual words.

CONCLUSIONS

This study found significant differences between LORs written for CRS fellowship applicants based on gender.

From Personal Statement to Mission Statement

In recent years, the application process for residency education has come under increased review. Step 1 scores will be pass/fail starting in 2022. There has been controversy about grade inflation or the lack of grades altogether at many medical schools. Faculty letters of recommendation have been criticized that they often exhibit implicit bias against women and under-represented minorities or simply are too vague or generic to be useful. Given this difficult landscape, the personal statement carries increased importance to highlight our applicants’ unique motivations and interests. We propose a model that residency and fellowship applicants may employ to craft their personal statements.

To the Point: advising students applying to Obstetrics and Gynecology residency in 2020 and beyond

This article, from the "To the Point" series by the Undergraduate Medical Education Committee of the Association of Professors of Gynecology and Obstetrics, is a guide for advising medical students applying to Obstetrics and Gynecology residency programs. The residency application process is changing rapidly in response to an increasingly complex and competitive atmosphere, with a wider recognition of the stress, expense, and difficulty of matching into graduate training programs. The coronavirus disease 2019 pandemic and societal upheaval make this application cycle more challenging than ever before. Medical students need reliable, accurate, and honest advising from the faculty in their field of choice to apply successfully to residency. The authors outline a model for faculty career advisors, distinct from mentors or general academic advisors. The faculty career advisor has detailed knowledge about the field, an in-depth understanding of the application process, and what constitutes a strong application. The faculty career advisor provides accurate information regarding residency programs within the specialty, helping students to strategically apply to programs where the student is likely to match, decreasing anxiety, expense, and overapplication. Faculty career advisor teams advise students throughout the application process with periodic review of student portfolios and are available for support and advice throughout the process. The authors provide a guide for the faculty career advisor in Obstetrics and Gynecology, including faculty development and quality improvement.

Letters of recommendation for surgical fellowships - Does gender make a difference?

BACKGROUND

Females comprise 1/3 of general surgery residents, 1/4 of surgical faculty and 10% of full professors. Inadequate sponsorship is one proposed mechanism for this decline. This study evaluated letters of recommendation (LOR) among applicants applying to a complex general surgical oncology (CGSO) fellowship.

METHODS

Linguistic analysis of LOR for CGSO applicants was conducted. Demographics of authors and features of the LOR were extracted. Differences by gender of the applicant were analyzed.

RESULTS

Among 340 letters, 67% were written for male and 33% written for female applicants. Males authored 84% of letters reviewed. Female authors used more grindstone adjectives than males (3.61 v 2.90). However, this difference was seen only among letters written for male applicants (3.82 v. 2.73). All other linguistic features were similar, aside from mention of physical appearance which was significantly more common in letters written about female applicants (4% v. 1%).

CONCLUSIONS

Female authors write substantively different letters than males. Physical appearance is a small but important difference in letters for female applicants.