Have personal statements become impersonal? An evaluation of personal statements in anesthesiology residency applications

Can Artificial Intelligence Fool Residency Selection Committees? Analysis of Personal Statements by Real Applicants and Generative AI, a Randomized, Single-Blind Multicenter Study

Introduction

The potential capabilities of generative artificial intelligence (AI) tools have been relatively unexplored, particularly in the realm of creating personalized statements for medical students applying to residencies. This study aimed to investigate the ability of generative AI, specifically ChatGPT and Google BARD, to generate personal statements and assess whether faculty on residency selection committees could (1) evaluate differences between real and AI statements and (2) determine differences based on 13 defined and specific metrics of a personal statement.

Methods

Fifteen real personal statements were used to generate 15 unique and distinct personal statements from ChatGPT and BARD each, resulting in a total of 45 statements. Statements were then randomized, blinded, and presented to a group of faculty reviewers on residency selection committees. Reviewers assessed the statements by 14 metrics including if the personal statement was AI-generated or real. Comparison of all metrics was performed.

Results

Faculty correctly identified 88% (79/90) real statements, 90% (81/90) BARD, and 44% (40/90) ChatGPT statements. Accuracy of identifying real and BARD statements was 89%, but this dropped to 74% when including ChatGPT. In addition, the accuracy did not increase as faculty members reviewed more personal statements (area under the curve [AUC] 0.498, p = 0.966). BARD performed poorer than both real and ChatGPT across all metrics (p < 0.001). Comparing real with ChatGPT, there was no difference in most metrics, except for Personal Interests, Reasons for Choosing Residency, Career Goals, Compelling Nature and Originality, and all favoring the real personal statements (p = 0.001, p = 0.002, p < 0.001, p < 0.001, and p < 0.001, respectively).

Conclusion

Faculty members accurately identified real and BARD statements, but ChatGPT deceived them 56% of the time. Although AI can craft convincing statements that are sometimes indistinguishable from real ones, replicating the humanistic experience, personal nuances, and individualistic elements found in real personal statements is difficult. Residency selection committees might want to prioritize these particular metrics while assessing personal statements, given the growing capabilities of AI in this arena.

Clinical Relevance

Residency selection committees may want to prioritize certain metrics unique to the human element such as personal interests, reasons for choosing residency, career goals, compelling nature, and originality when evaluating personal statements.

Evaluation of commitment to equity, diversity, and inclusion on Canadian ophthalmology residency program websites

OBJECTIVE

Program websites play a crucial role in providing information to prospective residency applicants. This research assesses the extent to which Canadian ophthalmology residency programs demonstrate their commitment to equity, diversity, and inclusion (EDI) through their websites and identifies areas for potential improvement.

METHODS

A cross-sectional analysis of websites affiliated with 15 accredited Canadian ophthalmology residency training programs was performed. A score was assigned based on 16 EDI elements derived from recent literature. The relationship between program website EDI elements with language, geography, and regional population diversity was explored.

RESULTS

Of the 15 Canadian ophthalmology residency programs, 12 (80%) met at least 1 of 16 EDI elements, with a mean program score of 2.4 (SD = 1.92). The program that attained the highest score met 7 of 16 EDI elements. Anglophone programs had a higher mean score (2.92; SD = 1.78) when compared with their Francophone counterparts (0.33; SD = 0.58; p = 0.023). No significant difference was found between programs in western, central, and eastern Canada (p = 0.86). The regional minorities proportion associated with each program was not found to influence scores (p = 0.088).

CONCLUSIONS

EDI elements were seldom addressed on Canadian ophthalmology residency program affiliated websites. We propose recommendations in 4 domains: showcasing team diversity, clear discourse in recruitment, inclusive resources, and program organization. Improving EDI visibility on program websites through revisions of existing websites or through creation of new promotional websites may foster recruitment of a diverse resident body.

Detecting Artificial Intelligence-Generated Personal Statements in Professional Physical Therapist Education Program Applications: A Lexical Analysis

OBJECTIVE

The objective of this study was to compare the lexical sophistication of personal statements submitted by professional physical therapist education program applicants with those generated by OpenAI's Chat Generative Pretrained Transformer (ChatGPT).

METHODS

Personal statements from 152 applicants and 20 generated by ChatGPT were collected, all in response to a standardized prompt. These statements were coded numerically, then analyzed with recurrence quantification analyses (RQAs). RQA indices including recurrence, determinism, max line, mean line, and entropy were compared with t-tests. A receiver operating characteristic curve analysis was used to examine discriminative validity of RQA indices to distinguish between ChatGPT and human-generated personal statements.

RESULTS

ChatGPT-generated personal statements exhibited higher recurrence, determinism, mean line, and entropy values than did human-generated personal statements. The strongest discriminator was a 13.04% determinism rate, which differentiated ChatGPT from human-generated writing samples with 70% sensitivity and 91.4% specificity (positive likelihood ratio = 8.14). Personal statements with determinism rates exceeding 13% were 8 times more likely to have been ChatGPT than human generated.

CONCLUSION

Although RQA can distinguish artificial intelligence (AI)-generated text from human-generated text, it is not absolute. Thus, AI introduces additional challenges to the authenticity and utility of personal statements. Admissions committees along with organizations providing guidelines in professional physical therapist education program admissions should reevaluate the role of personal statements in applications.

IMPACT

As AI-driven chatbots like ChatGPT complicate the evaluation of personal statements, RQA emerges as a potential tool for admissions committees to detect AI-generated statements.

Orthopaedic Personal Statement Thematic Review: Does Topic Matter?

Introduction

In the process of applying into medical residency, the Electronic Residency Application Service (ERAS) requires critical documents including a personal statement. Utility of personal statements are questioned based on suspected congruity of the content within personal statements among those who apply into orthopaedic surgery. The goal of this study was to identify and categorize the thematic elements found within the 2021 to 2022 personal statements of orthopaedic surgery applicants at a single institution and assess a correlation to interview invitation.

Methods

Deidentified personal statements among 2021 to 2022 ERAS applicants were reviewed by the research staff and categorized into one of the proposed themes. Three hundred ninty-four applications passed initial screening filters, and 49 applicants were granted an interview. Proposed themes that were collected included: family of physician, working with hands, history of injury/disease, prior professional setting, immigration/travel, athlete/sports, reapplication, previous clinical experience, and other. χ2 test was used to analyze categorical themes and additional univariate group-to-group comparisons. Multivariate principal component analysis was performed to determine which themes were associated with interview invitation.

Results

There was a significant difference in theme selection for an applicant's personal statement (χ2 = 209.5, p < 0.001), but no statistical difference was observed between each individual categorical themes and interview selection. Univariate group-to-group comparison demonstrated greater interview rates between immigration/travel compared with reapplicant and family of physician compared with working with hands. Multivariate principal component analysis demonstrated immigration/travel as the theme with a positive correlation toward an interview invitation.

Conclusion

Despite a significant focus of the application process into orthopaedic surgery residencies, our single-institution study did observe specific themes that were more prevalent. There was an increased interview rate between applicant's themes for immigration/travel and family of physician when comparing groups. Immigration/travel was also identified as the only significant theme associated with interview invitation which may be due to the recent emphasis on promoting diversity within orthopaedic surgery.

Making the Personal Statement "Truly Personal": Recommendations From a Qualitative Case Study of Internal Medicine Program and Associate Program Directors

BACKGROUND

The personal statement is an integral part of a residency application but little guidance exists for medical students about what content to include.

OBJECTIVE

We use the framework of isomorphism, the process by which institutions model themselves after one another, to understand what internal medicine program directors (PDs) and associate program directors (APDs) recommend be included in the personal statement and how programs use personal statements in their selection of applicants to interview and rank.

METHODS

Semi-structured phone interviews were conducted between August and October 2020 with 13 academic PDs and APDs, who were selected for participation based on program size and geographic location. Interviews were recorded, transcribed, and coded using content analysis.

RESULTS

Effective personal statements should be well-written, present unique information, and demonstrate fit with a residency program. PDs and APDs recommended against expressing lack of interest in a program or highlighting negative personal characteristics. PDs and APDs used personal statements to distinguish between applicants and noted that personal statements help programs form an impression of the applicant. Consensus among PDs and APDs about what personal statements should include and how they are used indicates that isomorphism influences the match process.

CONCLUSIONS

Our study found that the personal statement is a valued part of the residency application when it includes unique attributes and reveals personal values that align with that of the program. Additionally, PDs and APDs noted that when applicants highlight their unique characteristics, it can help distinguish themselves from others.

Why Do Physicians Choose Pain as a Specialty?

OBJECTIVE

Numerous forces shape the practice of pain management: scientific advances, technical advances, societal expectations, public health crises, reimbursement factors, and the parameters of who gets trained and what motivates the trainees. In this observational study, we sought to determine expressed motivations for entering the subspecialty of pain management, and in particular whether applicants were more interested in procedural skills (our hypothesis) or rehabilitative and cognitive practices.

METHODS

We analyzed the personal statements of 142 applicants to the University of Minnesota's pain medicine fellowship program. In addition to those themes, the personal statements were scrutinized for other themes such as interest in teaching and research. Comprehensive vs interventional/procedural interests were coded by a group of four research assistants who were trained using practice essays until they achieved high interrater reliability (alpha > 0.8). Two of the researchers coded for additional themes on a two-point scale indicating presence or absence of a particular theme. When they did not agree, a third researcher broke the tie. Theme prevalence was compared by specialty and gender.

RESULTS

Residents expressed interest in interventional and comprehensive pain practice without significant differences; however, there were specialty and gender differences in other themes such as teaching, research, and leadership in program development.

CONCLUSIONS

If pain specialty training is going to meet the needs of prospective residents, patients, and society, we should do more to attract women and neurology and psychiatry residents. We should include more opportunities for research and the flexibility to educate trainees who may not pursue a procedural practice.

Prioritizing the Interview in Selecting Resident Applicants: Behavioral Interviews to Determine Goodness of Fit

From our initial screening of applications, we assess that the 10% to 15% of applicants whom we will interview are all academically qualified to complete our residency training program. This initial screening to select applicants to interview includes a personality assessment provided by the personal statement, Dean's letter, and letters of recommendation that, taken together, begin our evaluation of the applicant's cultural fit for our program. While the numerical scoring ranks applicants preinterview, the final ranking into best fit categories is determined solely on the interview day at a consensus conference by faculty and residents. We analyzed data of 819 applicants from 2005 to 2017. Most candidates were US medical graduates (62.5%) with 23.7% international medical graduates, 11.7% Doctors of Osteopathic Medicine (DO), and 2.1% Caribbean medical graduates. Given that personality assessment began with application review, there was excellent correlation between the preinterview composite score and the final categorical ranking in all 4 categories. For most comparisons, higher scores and categorical rankings were associated with applicants subsequently working in academia versus private practice. We found no problem in using our 3-step process employing virtual interviews during the COVID pandemic.

Gender-Based Differences in Urology Residency Applicant Personal Statements

OBJECTIVE

To gain insight into the perceptions of urology held by medical students as they enter the field, we analyzed the linguistic characteristics and gender differences in personal statements written by urology residency program applicants.

METHODS

Personal statements were abstracted from residency applications to a urology residency program. Linguistic Inquiry and Word Count, a validated text analysis software, characterized the linguistic content of the statements. Analyzed statements were compared according to gender of the applicant using multivariate analysis, examining the association of applicant gender and statement characteristics. Multivariate analysis was also performed to determine the association of personal statement characteristics with matching into urology residency.

RESULTS

Of 342 analyzed personal statements, no significant difference was found in statement characteristics between matched and unmatched applicants. Male and female applicants wrote with the same degree of overall analytical thinking, authenticity, and emotional tone. Clout, a measure of portrayed confidence, was low for both genders. Female applicants used more social and affective process words. Male applicants used more words indicating a sense of community and acceptance. Female applicants had more references to women within their statements.

CONCLUSION

Significant linguistic differences exist among personal statements written by men and women applying to urology residency. Word usage differences follow societal gender norms. Statement content demonstrates a difference between genders in perceived sense of belonging, highlighting the importance of gender concordant mentorship within the field.

Characteristics of internal medicine residents who successfully match into cardiology fellowships

BACKGROUND

The unique traits of residents who matriculate into subspecialty fellowships are poorly understood. We sought to identify characteristics of internal medicine (IM) residents who match into cardiovascular (CV) fellowships.

METHODS

We conducted a retrospective cohort study of 8 classes of IM residents who matriculated into residency from 2007 to 2014. The primary outcome was successful match to a CV fellowship within 1 year of completing IM residency. Independent variables included residents' licensing exam scores, research publications, medical school reputation, Alpha Omega Alpha (AOA) membership, declaration of intent to pursue CV in the residency application personal statement, clinical evaluation scores, mini-clinical evaluation exercise scores, in-training examination (ITE) performance, and exposure to CV during residency.

RESULTS

Of the 339 included residents (59% male; mean age 27) from 120 medical schools, 73 (22%) matched to CV fellowship. At the time of residency application, 104 (31%) had ≥1 publication, 38 (11%) declared intention to pursue CV in their residency application personal statement, and 104 (31%) were members of AOA. Prior to fellowship application, 111 (33%) completed a CV elective rotation. At the completion of residency training, 108 (32%) had ≥3 publications. In an adjusted logistic regression analysis, declaration of intention to pursue CV (OR 6.4, 99% CI 1.7-23.4; p < 0.001), completion of a CV elective (OR 7.3, 99% CI 2.8-19.0; p < 0.001), score on the CV portion of the PGY-2 ITE (OR 1.05, 99% CI 1.02-1.08; p < 0.001), and publication of ≥3 manuscripts (OR 4.7, 99% CI 1.1-20.5; p = 0.007) were positively associated with matching to a CV fellowship. Overall PGY-2 ITE score was negatively associated (OR 0.93, 99% CI 0.90-0.97; p < 0.001) with matching to a CV fellowship.

CONCLUSIONS

Residents' matriculation into CV fellowships was associated with declaration of CV career intent, completion of a CV elective rotation, CV medical knowledge, and research publications during residency. These findings may be useful when advising residents about pursuing careers in CV. They may also help residents understand factors associated with a successful match to a CV fellowship. The negative association between matching into CV fellowship and overall ITE score may indicate excessive subspecialty focus during IM residency.

What Do Program Directors Value in Personal Statements? A Qualitative Analysis

Background: All applicants to accredited training programs must write a personal statement as part of the application process. This may provoke anxiety on the part of the applicant and can result in an impersonal product that does not enhance his or her application. Little has been written about what program directors are seeking in personal statements.

Objective: To gain a better understanding of how pulmonary and critical care fellowship program directors view and interpret these essays and to help applicants create more effective personal statements and make the writing process less stressful.

Methods: We surveyed the membership of the Association of Pulmonary and Critical Care Medicine Program Directors in 2018. Quantitative data were collected regarding the importance of the personal statement in the candidate selection process. Qualitative data exploring the characteristics of personal statements, what the personal statement reveals about applicants, and advice for writing them were also collected. Comparative analysis was used for coding and analysis of qualitative data.

Results: Surveys were completed by 114 out of 344 possible respondents (33%). More than half of the respondents believed that the personal statement is at least moderately important when deciding to offer an interview, and 40% believed it is at least moderately important when deciding rank order. A qualitative analysis revealed consistent themes: communication skills, provision of information not found elsewhere, applicant characteristics, and things to avoid.

Conclusion: The respondents view the personal statement as moderately important in the application process. They value succinct, quality writing that reveals personal details not noted elsewhere. The information presented may help reduce anxiety associated with writing the personal statement and result in making the personal statement a more meaningful part of the application.

Linguistic analysis of pediatric residency personal statements: gender differences

BACKGROUND

All US residency programs require applicants to submit personal statements. Prior studies showed gender differences in personal statement writing, which has implications for gender bias in the application process, but previous studies have not considered the dual influence of specialty-specific values on personal statement writing by applicants of each gender.

OBJECTIVE

To understand gender differences in pediatric residency personal statements.

METHODS

From 2017 to 2018, we performed linguistic analysis of personal statements written by interviewees at a mid-size US pediatrics residency during two prior academic years. We assessed writing tone, communal language, and agentic language. We performed t-tests to evaluate for gender differences, p < 0.05.

RESULTS

We analyzed personal statements from 85 male and 85 female interviewees. Average word count was 676 words. Personal statements demonstrated analytic writing style with authentic and positive emotional tone. We found no gender differences in communal language for social affiliation (p = 0.31), adjectives (p = 0.49), or orientation (p = 0.48), which deviates from typical gender norms for male language use. Males used agentic language of reward more frequently (p = 0.02).

CONCLUSIONS

Findings suggest that social language is valued in pediatrics, a predominantly female specialty, regardless of applicant gender. Use of reward language by males is consistent with previous findings. Future studies should evaluate gender differences in residency applications across specialties to advance understanding of the role gender plays in the application process.

Thematic Analysis of Emergency Medicine Applicants' Personal Statements

BACKGROUND

The personal statement is an important part of the residency application. Although guidance exists, the composition of personal statements is not standardized; each statement reflects an applicant's unique personality. In emergency medicine (EM), the personal statement could thus provide insight into why applicants are choosing EM and what they hope to accomplish in the field that could guide advisors and applicants.

OBJECTIVE

To perform a thematic analysis of personal statements from applicants accepted into an academic EM residency program to gain insight into what successful applicants include in their personal statements, why applicants are pursuing careers in emergency medicine, and anticipated career goals.

METHODS

Thematic analysis was performed on ten randomly selected personal statements from matched allopathic, U.S. applicants at a single, large, urban 3-year EM residency program between 2008 and 2015. Themes and sub-themes were identified and analyzed for frequency.

RESULTS

Ten personal statements were analyzed. Thirty-one (31) unique themes were identified and grouped into five main themes: personal characteristics related to a career in EM (38.3%, 116/303), why I love EM (36%, 109/303), my story (13.5%, 41/303), my career in EM (8.9%, 27/303), and ideal characteristics of a residency program (3.3%, 10/303). The most common personal characteristics described were altruism and the ability to work well under pressure. Applicants love EM due to the diversity of patients and disease presentations and the ability to perform procedures.

CONCLUSIONS

Thematic analysis of EM applicants' personal statements highlights the uniqueness of EM as a specialty and what draws applicants to EM.

A Narrative Review of the Evidence Supporting Factors Used by Residency Program Directors to Select Applicants for Interviews

BACKGROUND

Residency applicants feel increasing pressure to maximize their chances of successfully matching into the program of their choice, and are applying to more programs than ever before.

OBJECTIVE

In this narrative review, we examined the most common and highly rated factors used to select applicants for interviews. We also examined the literature surrounding those factors to illuminate the advantages and disadvantages of using them as differentiating elements in interviewee selection.

METHODS

Using the 2018 NRMP Program Director Survey as a framework, we examined the last 10 years of literature to ascertain how residency directors are using these common factors to grant residency interviews, and whether these factors are predictive of success in residency.

RESULTS

Residency program directors identified 12 factors that contribute substantially to the decision to invite applicants for interviews. Although United States Medical Licensing Examination (USMLE) Step 1 is often used as a comparative factor, most studies do not demonstrate its predictive value for resident performance, except in the case of test failure. We also found that structured letters of recommendation from within a specialty carry increased benefit when compared with generic letters. Failing USMLE Step 1 or 2 and unprofessional behavior predicted lower performance in residency.

CONCLUSIONS

We found that the evidence basis for the factors most commonly used by residency directors is decidedly mixed in terms of predicting success in residency and beyond. Given these limitations, program directors should be skeptical of making summative decisions based on any one factor.

Analysis of Small Animal Rotating Internship Applicants' Personal Statements

The primary purpose of this study was to identify themes that are consistent across veterinary internship applicants' personal statements and that are correlated with the statements' perceived overall quality. A secondary purpose was to investigate the reliability in personal statement quality scoring among six experienced internship candidate evaluators. One hundred applications to the University of Georgia Small Animal Rotating Internship program were evaluated. Each evaluator wrote a description of what he or she values in personal statements and his or her beliefs about content and presentation in high- and low-quality statements. After statement de-identification, each evaluator reviewed 15 randomly selected personal statements from internship applicants and assigned each a score ranging from 1 to 4 according to the following criteria: 1 = would not rank for an internship; 2 = would rank in the bottom third; 3 = would rank in the middle third; and 4 = would rank in the top third. A subset of these scored personal statements was chosen for qualitative analysis. A qualitative document analysis using grounded theory was performed for both the evaluators' descriptions of preferences in personal statements and the subset of personal statements. Agreement among evaluators' assigned scores was slight (Fleiss's κ = 0.11). Analysis of the evaluator statements and the scored candidate statements indicated that important factors in a personal statement include the applicant's ability to articulate experiences, to convey maturity, to demonstrate understanding of what an internship entails, and to describe reasons for pursuing an internship.

Textual Analysis of General Surgery Residency Personal Statements: Topics and Gender Differences

PURPOSE

Applicants to US general surgery residency training programs submit standardized applications. Applicants use the personal statement to express their individual rationale for a career in surgery. Our research explores common topics and gender differences within the personal statements of general surgery applicants.

METHODS

We analyzed the electronic residency application service personal statements of 578 applicants (containing 3,82,405 words) from Liaison Committee on Medical Education-accredited medical schools to a single ACGME-accredited general surgery program using an automated textual analysis program to identify common topics and gender differences. Using a recursive algorithm, the program identified common words and clusters, grouping them into topic classes, which are internally validated.

RESULTS

We identified and labeled 8 statistically significant topic classes through independent review: "my story," "the art of surgery," "clinical vignettes," "why I love surgery," "residency program characteristics," "working as a team," "academics and research," and "global health and policy." Although some classes were common to all applications, we also identified gender-specific differences. Notably, women were significantly more likely than men to be represented within the class of "working as a team." (p < 0.01) Furthermore, men were significantly more likely than women to be represented within the class of "clinical vignettes" (p < 0.01).

CONCLUSIONS

Applying textual analysis to a national cohort, we identified common narrative topics in the personal statements of aspiring general surgeons, noting differences between the statements of men and women. Women were more likely to discuss surgery as a team endeavor while men were more likely to focus on the details of their surgical experiences. Our work mirrors what has been found in social psychology research on gender-based differences in how men and women communicate their career goals and aspirations in other competitive professional situations.

Utility of selection methods for specialist medical training: A BEME (best evidence medical education) systematic review: BEME guide no. 45

BACKGROUND

Selection into specialty training is a high-stakes and resource-intensive process. While substantial literature exists on selection into medical schools, and there are individual studies in postgraduate settings, there seems to be paucity of evidence concerning selection systems and the utility of selection tools in postgraduate training environments.

AIM

To explore, analyze and synthesize the evidence related to selection into postgraduate medical specialty training.

METHOD

Core bibliographic databases including PubMed; Ovid Medline; Embase, CINAHL; ERIC and PsycINFO were searched, and a total of 2640 abstracts were retrieved. After removing duplicates and screening against the inclusion criteria, 202 full papers were coded, of which 116 were included.

RESULTS

Gaps in underlying selection frameworks were illuminated. Frameworks defined by locally derived selection criteria, and heavily weighed on academic parameters seem to be giving way to the evidencing of competency-based selection approaches in some settings. Regarding selection tools, we found favorable psychometric evidence for multiple mini-interviews, situational judgment tests and clinical problem-solving tests, although the bulk of evidence was mostly limited to the United Kingdom. The evidence around the robustness of curriculum vitae, letters of recommendation and personal statements was equivocal. The findings on the predictors of past performance were limited to academic criteria with paucity of long-term evaluations. The evidence around nonacademic criteria was inadequate to make an informed judgment.

CONCLUSIONS

While much has been gained in understanding the utility of individual selection methods, though the evidence around many of them is equivocal, the underlying theoretical and conceptual frameworks for designing holistic and equitable selection systems are yet to be developed.

Preparing for Fellowship in Internal Medicine. Steps for Success with a Focus on Pulmonary and/or Critical Care Programs

This paper outlines specific tips for those applying to pulmonary and/or critical care medicine fellowship training in the United States using the PAIR-Match steps: preparation, application, interview, ranking, and match. Preparation for fellowship begins long before the application process with an assessment of one's long-term goals (to the extent that these are known). The cornerstone of the application is the curriculum vitae, which should highlight applicants' pulmonary and critical care-related experiences and scholarly work. Applicants should obtain letters of recommendation from faculty members who know them well and can write a letter that speaks to their strengths in clinical, scholarly, or leadership areas. The personal statement is an opportunity to share experiences not otherwise shared in the application and is an opportunity to explain any breaks in training or performance lapses. When selecting programs to which they will apply, applicants should pay close attention to the areas of education and curriculum, clinical experience, scholarly opportunity, and personal factors. Preparing for interviews should include a review of the program at which one is interviewing and development of relevant questions regarding details of the program. The interview day is the applicant's opportunity to see the "personality" of the program by meeting with the program director, faculty, and current fellows and to assess whether the program is a good fit for their goals. Applicants should only rank those programs they are willing to attend, in order of preference; they should be aware that the match process is binding.

Identifying and Promoting Best Practices in Residency Application and Selection in a Complex Academic Health Network

Medical education institutions have a social mandate to produce a diverse physician workforce that meets the public's needs. Recent reports have framed the admission process outcome of undergraduate and postgraduate medical education (UGME and PGME) programs as a key determinant of the collective contributions graduating cohorts will make to society, creating a sense of urgency around the issue of who gets accepted. The need for evidence-informed residency application and selection processes is growing because of the increasing size and diversity of the applicant pool and the need for equity, fairness, social accountability, and health human resource planning. The selection literature, however, is dominated by a UGME focus and emphasizes determination of desirable qualities of future physicians and selection instrument reliability and validity. Gaps remain regarding PGME selection, particularly the creation of specialty-specific selection criteria, suitable outcome measures, and reliable selection systems.In this Perspective, the authors describe the University of Toronto's centralized approach to defining system-level best practices for residency application and selection. Over the 2012-2013 academic year, the Best Practices in Application and Selection working group reviewed relevant literature and reports, consulted content experts, surveyed local practices, and conducted iterative stakeholder consultations on draft recommendations. Strong agreement arose around the resulting 13 principles and 24 best practices, which had either empirical support or face validity. These recommendations, which are shared in this article, have been adopted by the university's PGME advisory committee and will inform a national initiative to improve trainees' transition from UGME to PGME in Canada.

Selection of subspecialty fellows in anesthesia for pediatric anesthesia: a national survey of program directors in the United States

BACKGROUND

Several studies have investigated the selection process for medical students, residents, and fellows across different specialties. Currently, the selection criteria used by pediatric anesthesia program directors to select prospective pediatric anesthesia fellows are unknown.

OBJECTIVE

To evaluate the criteria used by program directors in the selection of prospective pediatric anesthesia fellows in the United States.

METHODS

Survey responses were solicited from 51 pediatric anesthesia fellowship directors in the United States. Questions evaluated department demographic information, the importance of fellowship selection criteria, and program satisfaction with the Match process.

RESULTS

Forty-two of 51 fellowship directors responded to the survey (a response rate of 82%). The personal statement was reported as extremely/very important for 30 of 42 (71%) program directors, while research experience and prior publications were reported as extremely/very important for 16 of 41 (39%) and 16 of 41 (39%) program directors, respectively. Programs that offered all the spots through the Match were smaller, median (IQR) number of fellows of 3 (2-6) compared to the ones that did not offer all the spots through the Match, 8 (2-12), P = 0.03. In contrast, no difference in program directors' satisfaction with the Match was detected between smaller (≤ 3 fellows) and larger programs (>3 fellows), P = 0.25.

CONCLUSION

A minority of pediatric fellowship program directors rated prior research experience as an important factor in the selection of prospective fellows. The lack of emphasis on research qualities for fellows is potentially concerning for the future of our specialty.

The influences of background on beginning medical students' perceptions of rural medical practice

BACKGROUND

Access to health care is an ongoing problem for underserved populations in rural and remote regions of Australia and North America. Despite medical schools educating more medical doctors, this maldistribution continues. While students entering medical programs with a rural focus purport to have an interest in rural medicine, their understanding of possible future rural practice is unclear. This study explored the differences in perception of rural practice between beginning medical students from rural and urban backgrounds to gain an indication of the usefulness of our selection process to meet the rural workforce mandate.

METHODS

Beginning medical students completed a writing exercise about the life and work of a rural medical doctor as a test of their academic writing skills. After completing the task and receiving feedback, students were invited to submit their work for analysis. Template analysis using themes from a study of rural medical registrars was used to analyse 103 scripts.

RESULTS

Students demonstrated foundational insight into some of the realities of rural life and practice. However, differences were noted in perspectives between rural background students and urban background students. Rural background students used everyday language to describe the practicalities of rural life, medical practice and the implications for families and communities. Urban background students generally used complex language and more negative descriptors.

CONCLUSIONS

Beginning medical students from urban and rural backgrounds differ in their perceptions and expression of rural practice. These outcomes are important for medical schools that use interviews in their selection process. Rural background applicants' suitability may be overlooked because of the interviewer's expectations of language, while urban background applicants may score higher related to complex language and use of key phrases. Interviewer training should address this likely bias thereby increasing the potential to recruit rural background students.

The predictive value of application variables on the global rating of applicants to a general surgery residency program

OBJECTIVE

Selection of applicants to residency programs can involve a great deal of variability. The purpose of this study was to determine the relationship between different subjective and objective application variables and the global rating score (GRS) of applicants to a general surgery residency program.

DESIGN

This was a retrospective analysis of data collected from the Electronic Residency Application Service on 188 applicants to a general surgery residency program from 2010 to 2013. Subjective variables including letters of recommendation (LORs), personal statements (PSs), and volunteer work were blindly assessed by raters using a literature-based method of evaluation. Objective data included several variables, such as United States Medical Licensing Examination (USMLE) scores. Each applicant received a GRS, which was a faculty-given numerical value reflecting both interview performance and overall application strength. The effect of subjective and objective variables on the GRS was determined.

SETTING

The Geisinger Medical Center, a rural moderate-sized general surgery residency program.

RESULTS

Of all the application variables examined, bivariate analysis indicated that having no prior residency (p = 0.0023), prior medical work (p = 0.0329), higher USMLE Step II Clinical Knowledge scores (p = 0.0021), higher overall PS score (p = 0.0125) and PS Written Expression score (p = 0.0007), and LORs from surgeons in leadership positions (p = 0.0029) have a significant (p < 0.05) effect on the GRS. Of these factors, USMLE Step II Clinical Knowledge score, PS Written Expression score, no prior residency, prior medical work, and LORs from surgeon in lead position had a significant effect on GRS based on multivariate stepwise regression analysis.

CONCLUSIONS

Our analysis identifies specific surgical resident applicant variables that are predictive of the GRS. Interestingly, most of these factors were objective. This may allow for the development of a more quantitative tool for selection of applicants.

Textual analysis of internal medicine residency personal statements: themes and gender differences

CONTEXT

Applicants to US residency training programmes are required to submit a personal statement, the content of which is flexible but often requires them to describe their career goals and aspirations. Despite their importance, no systematic research has explored common themes and gender differences inherent to these statements.

OBJECTIVES

This study was conducted to analyse US applicants' Electronic Residency Application Service (ERAS) personal statements using two automated textual analysis programs, and to assess for common themes and gender-associated differences.

METHODS

A retrospective cohort study of 2138 personal statements (containing 1,485,255 words) from candidates from 377 national and international medical schools applying to US internal medicine (IM) residency programmes through ERAS was conducted. A mathematical analysis of text segments using a recursive algorithm was performed; two different specifications of the text segments were used to conduct an internal validation.

RESULTS

Five statistically significant thematic classes were identified through independent review by the researchers. These were best defined as referring to: the appeal of the residency programme; memorable patients; health care as public policy; research and academia, and family inspiration. Some themes were common to all applications. However, important gender-specific differences were identified. Notably, men were more likely to describe personal attributes and to self-promote, whereas women more frequently expressed the communicative and team-based aspects of doctoring. The results were externally validated using a second software program. Although these data comprise part of the national pool, they represent applicants to a single specialty at a single institution.

CONCLUSIONS

By applying textual analysis to material derived from a national cohort, we identified common narrative themes in the personal statements of future US physicians, noting differences between men and women. Together, these data provide novel insight into the dominant discourse of doctoring in this generation of students applying for further training in US IM residency programmes, and depict a diverse group of applicants with multiple motivations, desires and goals. Furthermore, differences seen between men and women add to the growing understanding of bias in medical education. Training programmes may benefit by adapting curricula to foster such diverse interests.

A critical review of personal statements submitted by dermatology residency applicants

Background. A strong personal statement is deemed favorable in the overall application review process. However, research on the role of personal statements in the application process is lacking. Objective. To determine if personal statements from matched applicants differ from unmatched applicants. Methods. All dermatology residency applications (n = 332) submitted to UC Davis Dermatology in the year of 2012 were evaluated. Two investigators identified the characteristics and recurring themes of content present in the personal statements. Then, both investigators individually evaluated the content of these personal statements in order to determine if any of the defined themes was present. Chi-square, Fisher's exact, and reliability tests were used. Results. The following themes were emphasized more often by the matched applicants than the unmatched applicants as their reasons for going into dermatology are to study the cutaneous manifestations of systemic disease (33.8% versus 22.8%), to contribute to the literature gap (8.3% versus 1.1%), and to study the pathophysiology of skin diseases (8.3% versus 2.2%; P ≤ 0.05 for all). Conclusion. The prevalence of certain themes in personal statements of dermatology applicants differs according to match status; nevertheless, whether certain themes impact match outcome needs to be further elucidated.

The personal statement for residency application: review and guidance

All applicants to US residency programs are required to write a personal statement. Recent reports of plagiarism and homogeneity in these freeform essays suggest the need for better guidance in this process. The authors review the historical and current role of the personal statement and provide a practical framework for writing a unique and effective personal statement, which will help both applicants and residency directors to maximize their chances of a successful match.