Surgical Residency Recruitment-Opportunities for Improvement

Exploring Characteristics of Academic General Surgery Residency Applicants: A Group Concept-Mapping Approach

OBJECTIVE

Holistic review, which emphasizes qualitative attributes over objective measures, has been proposed as a method for selecting candidates for surgical residency in order to improve diversity in graduate medical education, and, ultimately, the field of surgery. This study seeks to articulate desirable traits of applicants as a first-step in standardizing the holistic review process.

DESIGN

Using Group Concept Mapping, a web-based mixed-methods participatory research methodology, residency selection committee members were asked to 1) list desirable characteristics of applicants, 2) group these into categories, 3) rate their importance to academic/clinical success on a 5-point Likert scale (1 = not at all important, 5 = extremely important), and 4) rate the degree to which each characteristic is feasible to assess on a 3-point Likert scale (1 = not at all feasible, 3 = very feasible). Grouped characteristics submitted to hierarchical cluster analysis depicted committee's consensus about desirable qualities/criteria for applicants. Bivariate scatter-plots and pattern-matching graphics demonstrated which of these criteria were most important and reliably assessed.

SETTING

A single academic general surgery residency training program in Western Pennsylvania.

PARTICIPANTS

Members of the selection committee for the UPMC General Surgery Residency program who had participated in at least 1 prior cycle of applicant selection.

RESULTS

Desirable characteristics of highly qualified applicants into an academic general surgery residency were clustered into domains of 1) scholarly work and research, 2) grades/formal assessments, 3) program fit, 4) behavioral assets, and 5) aspiration. Behavioral assets, which was felt to be the most important to clinical and academic success were considered to be the least feasible to reliably assess. Within this domain, initiative, being self-motivated, intellectual curiosity, work ethic, communication skills, maturity and self-awareness, and thoughtfulness were viewed as most frequently reliably assessed from the application and interview process.

CONCLUSIONS

High quality applicants possess several behavioral assets that faculty deem are important to academic and clinical success. Adapting validated metrics for assessing these assets, may provide a solution for addressing subjectivity and other challenges scrutinized by critics of holistic review.

The Virtual Recruitment Onion: Peeling Back the Layers of the Interview Season During the COVID-era

OBJECTIVE

We sought to characterize General Surgery residency program directors' (PDs) baseline perspective on how the COVID-19 mandated changes to the recruitment and interview processes impacted how the PDs evaluated and recruited the applicants.

DESIGN

An anonymous cross-sectional questionnaire survey.

SETTING

A large, mid-western academic general surgery residency program.

PARTICIPANTS

47 PDs of Accreditation Council for Graduate Medical Education (ACGME) accredited General Surgery residency programs.

RESULTS

During the virtual-only interviews during the COVID-19 pandemic-era 2020-21 General Surgery residency application cycle, PDs shifted their focus to virtual outreach efforts and bolstered social media presences to recruit strong applicants. Also, our study found statistically significant changes to the increased value of letters of recommendation (LORs) for the PDs when assessing an applicant's commitment to surgery. These findings suggest that the necessity of adapting to the virtual-only interview format significantly altered how the PDs recruited and evaluated applicants for the General Surgery residency match.

CONCLUSIONS

A complete replacement of the in-person interviews with virtual-only interviews may be challenging unless buy-in exists from key stakeholders in the surgical community. Our study highlights the PDs' hesitation in assessing candidates' commitment to surgery from virtual interviews alone. Incorporating virtual interviews as a part of the screening process for applicants may serve as an avenue to maximize the benefits of the virtual interview format. Furthermore, COVID-19 pandemic has normalized the growing social media presence of residency programs, adding to the changing landscape of recruiting and interviewing applicants for General Surgery residency match.

General Surgery Residency Match: Time for More than a Virtual Change

The 2020-2021 General Surgery Residency Match presents unique challenges in the setting of the COVID-19 pandemic and highlights pre-existing concerns. In order to move toward an equitable and manageable surgical residency application process for both programs and applicants, systemic change is warranted.

Program Director Opinions of Virtual Interviews: Whatever Makes my Partners Happy

OBJECTIVE

To assess the processes and outcomes of 2021 colon and rectal surgery match season: one of the first National Resident Matching Program (NRMP) match to conduct uniformly virtual interviews for all programs and candidates due to the Covid-19 pandemic. Since this if the first-year interviews were held entirely virtual for a (NRMP) match season, we sought to determine: (1) How did program directors (PDs) in this year's fellowship conduct their virtual interviews? (2) Were any of these conduct decisions associated with the PD satisfaction with the resulting match? (3) What is the PDs opinion of how interviews should occur next year if COVID-19 is not a factor?

DESIGN AND SETTING

The authors sent an anonymous survey to the PDs of all programs participating in the 2021 colon and rectal surgery residency match directly following match day 2020.

PARTICIPANTS

Forty-one colon and rectal residency PDs (70% response rate) responded to the survey (78% Male) representing a range of experience (M = 7.61, SD = 5.66, years as PD at current institution), and program type (77.5% Academic, 7.5% Independent Academic Medical Center, 15% Nonacademic).

RESULTS

While programs utilized several different platforms, conducted various forms of training for their faculty, and provided applicants with different types of information, interview day(s) across the specialty are reported to have proceeded smoothly. PDs as a whole were very satisfied with their match results this year (M = 4.65, SD = .66), and this satisfaction was not impacted by virtual interview decisions or processes. However, only 55% of PDs agree or strongly agree that next year's interviews should be virtual regardless of COVID-19, a judgement solely influenced by the opinion of other program faculty on virtual interviews, regardless of satisfaction with match or comfort with technology.

CONCLUSION

While PDs report high satisfaction with virtual interview processes and outcomes, there is less agreement that colon and rectal surgery residency interviews should move to a solely virtual platform.

Virtual interviews - Utilizing technological affordances as a predictor of applicant confidence

PURPOSE

In the midst of a pandemic, residency interviews transitioned to a virtual format for the first time. Little is known about the effect this will have on the match process. The study aim is to evaluate resident application processes and perceived outcomes.

METHODS

An electronic survey was distributed to 142 colon and rectal surgery residency applicants (95% of total).

RESULTS

A total of 77 applicants responded to the survey (54% response rate). Applicants reported high levels of satisfaction with virtual interviews but less comfort. Utilizing the mute button and using notes in a different way from face-to-face interviews were significantly associated with applicant confidence that they ranked the right program highest. A majority of applicants (73%) would recommend virtual interviews next year even if COVID-19 is not a factor.

CONCLUSION

While applicants appear generally satisfied with virtual interviews, they also reported less comfort. Applicant confidence was predicted by utilizing the unique technological affordances offered by the virtual platform.

An Early Interview Day with Feedback to Prepare General Surgery Applicants for Residency Interviews

BACKGROUND

Success in the residency interview is important in obtaining a position in general surgery residency. Preparing applicants for interviews is imperative yet can be resource intensive. The primary objective of our study was to investigate whether an early interview day with feedback (IDWF) provides interview preparation to internal applicants to general surgery residency without negatively impacting their rank list position at our program.

METHODS

Internal applicants to the general surgery residency program at a single academic institution were invited to an early interview day after which they received individualized feedback and attended a workshop on interview techniques. Applicants were anonymously surveyed after The Match to measure their experiences with the IDWF. Two years of post-intervention program rank lists were compared to those from the five years pre-intervention to assess for change in rank list position of internal applicants. Participants included the 16 of 20 internal applicants who completed the survey. De-identified rank order data of internal applicants between 2014 and 2020 included 48 applicants.

RESULTS

All applicants believed the IDWF should be continued, and over half reported improved confidence and made changes from feedback. Rank list analysis demonstrated no statistically significant change in the proportion of internal applicants who ranked in the top 40 nor the average rank position of internal applicants.

CONCLUSIONS

An early interview day with feedback provides interview preparation in a resource-efficient manner without harming rank list position. The IDWF may be generalizable to other institutions to provide interview preparation to general surgery residency applicants.

A Model for Exploring Compatibility Between Applicants and Residency Programs: Right Resident, Right Program

Holistic review of residency applications is touted as the gold standard for selection, yet vast application numbers leave programs reliant on screening using filters such as United States Medical Licensing Examination scores that do not reliably predict resident performance and may threaten diversity. Applicants struggle to identify which programs to apply to, and devote attention to these processes throughout most of the fourth year, distracting from their clinical education. In this perspective, educators across the undergraduate and graduate medical education continuum propose new models for student-program compatibility based on design thinking sessions with stakeholders in obstetrics and gynecology education from a broad range of training environments. First, we describe a framework for applicant-program compatibility based on applicant priorities and program offerings, including clinical training, academic training, practice setting, residency culture, personal life, and professional goals. Second, a conceptual model for applicant screening based on metrics, experiences, attributes, and alignment with program priorities is presented that might facilitate holistic review. We call for design and validation of novel metrics, such as situational judgment tests for professionalism. Together, these steps could improve the transparency, efficiency and fidelity of the residency application process. The models presented can be adapted to the priorities and values of other specialties.

Determining Grades in the Internal Medicine Clerkship: Results of a National Survey of Clerkship Directors

PURPOSE

Trust in and comparability of assessments are essential in clerkships in undergraduate medical education for many reasons, including ensuring competency in clinical skills and application of knowledge important for the transition to residency and throughout students' careers. The authors examined how assessments are used to determine internal medicine (IM) core clerkship grades across U.S. medical schools.

METHODS

A multisection web-based survey of core IM clerkship directors at 134 U.S. medical schools with membership in the Clerkship Directors in Internal Medicine was conducted in October through November 2018. The survey included a section on assessment practices to characterize current grading scales used, who determines students' final clerkship grades, the nature/type of summative assessments, and how assessments are weighted. Respondents were asked about perceptions of the influence of the National Board of Medical Examiners (NBME) Medicine Subject Examination (MSE) on students' priorities during the clerkship.

RESULTS

The response rate was 82.1% (110/134). There was considerable variability in the summative assessments and their weighting in determining final grades. The NBME MSE (91.8%), clinical performance (90.9%), professionalism (70.9%), and written notes (60.0%) were the most commonly used assessments. Clinical performance assessments and the NBME MSE accounted for the largest percentage of the total grade (on average 52.8% and 23.5%, respectively). Eighty-seven percent of respondents were concerned that students' focus on the NBME MSE performance detracted from patient care learning.

CONCLUSIONS

There was considerable variability in what IM clerkships assessed and how those assessments were translated into grades. The NBME MSE was a major contributor to the final grade despite concerns about the impact on patient care learning. These findings underscore the difficulty in comparing learners across institutions and serve to advance discussions for how to improve accuracy and comparability of grading in the clinical environment.

Prioritizing the Interview in Selecting Resident Applicants: Behavioral Interviews to Determine Goodness of Fit

From our initial screening of applications, we assess that the 10% to 15% of applicants whom we will interview are all academically qualified to complete our residency training program. This initial screening to select applicants to interview includes a personality assessment provided by the personal statement, Dean's letter, and letters of recommendation that, taken together, begin our evaluation of the applicant's cultural fit for our program. While the numerical scoring ranks applicants preinterview, the final ranking into best fit categories is determined solely on the interview day at a consensus conference by faculty and residents. We analyzed data of 819 applicants from 2005 to 2017. Most candidates were US medical graduates (62.5%) with 23.7% international medical graduates, 11.7% Doctors of Osteopathic Medicine (DO), and 2.1% Caribbean medical graduates. Given that personality assessment began with application review, there was excellent correlation between the preinterview composite score and the final categorical ranking in all 4 categories. For most comparisons, higher scores and categorical rankings were associated with applicants subsequently working in academia versus private practice. We found no problem in using our 3-step process employing virtual interviews during the COVID pandemic.

Trends in Representation of Female Applicants and Matriculants in Canadian Residency Programs Across Specialties, 1995 to 2019

IMPORTANCE

Disparities in representation between sexes have been shown at multiple career stages in medicine despite increasing representation in the overall physician workforce.

OBJECTIVE

To assess sex representation of applicants to the Canadian R-1 entry match for postgraduate training programs from 1995 to 2019, comparing distribution between different specialties as well as applied vs matched applicants.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional analysis of aggregate data provided by the Canadian Resident Matching Service between 1995 and 2019 analyzed aggregate data for the Canadian R-1 residency match from 1995 through 2019.

EXPOSURES

Applicant sex as reported in the Canadian Resident Matching Service database.

MAIN OUTCOMES AND MEASURES

The sex representation of applicants was compared and the longitudinal trends in sex representation were analyzed by specialty between 1995 and 2019. The sex representation of overall applicants to the Canadian R-1 entry match were compared with matched applicants, and both were stratified by specialty.

RESULTS

A total of 48 424 applicants were identified (26 407 [54.5%] female applicants), of which 41 037 were matched applicants. Using specialty groupings, female applicants were most highly represented in obstetrics and gynecology (1776 of 2090 [85.0%]) and least represented in radiology (658 of 2055 [32.0%]). Within individual subspecialties, female applicants had the lowest representation in neurosurgery (90 of 394 [22.8%]). While female applicants represented an increasing proportion of the overall applicant population between 1995 and 2019 (z = 2.71; P = .007), significant increases were seen in some, but not all, individual specialties. Differences by sex were found among Canadian medical graduate match rates to their top-ranked specialty: female applicants had a lower likelihood of being rejected for family medicine (rejection of male applicants: OR, 0.46; 95% CI, 0.39-0.54; P < .001) and psychiatry (OR, 0.59; 95% CI, 0.46-0.76; P < .001) and were more likely to be rejected for all-encompassing surgery (acceptance of male applicants: OR, 1.19; 95% CI, 1.10-1.28; P < .001).

CONCLUSIONS AND RELEVANCE

Increasing representation of female residency applicants over time was seen in some, but not all, medical specialties in Canada, and sex-based differences in successful match rates were observed in some specialties. The reasons for these disparities require further investigation for corrective strategies to be identified.

Personal Statements in General Surgery: An Unrecognized Role in the Ranking Process

OBJECTIVE

Personal statements are a requirement of general surgery residency applications. Yet, their role in an applicant's final rank within a program remains unclear. This study explores the language used in personal statements to differentiate applicants in the general surgery residency ranking process.

DESIGN

A textual analysis of general surgery residency applicant personal statements was performed. Using inductive coding and grounded theory, 3 main themes from personal statements were identified: my story, my future, my goals. These themes were utilized to build a dictionary consisting of over 400 descriptive terms in multiple categories. Data was extracted using the Linguistics Inquiry and Word Count program, which can linguistically determine basic characteristics from text. The data was stratified according to final rank and gender for analysis, using correlation and descriptive statistics.

SETTING

Large, urban, academic general surgery residency program.

PARTICIPANTS

One hundred nineteen personal statements during the 2018 to 2019 application cycle were analyzed. All applicants were interviewed and considered for inclusion on our final rank list.

RESULTS

There were 68 (57%) females on the final rank list with no difference in the distribution of gender between those in the top and bottom halves (p = 0.11). Overall, personal statements for the top applicants scored higher in grit than those in the bottom half (median 0.42% vs 0.35%, p = 0.03). Males ranked in the top half had less use of agentic (p = 0.04) and efficient/organized (p = 0.03) words when compared with males ranked in the bottom half. In contrast, females ranked in the top half used more grit words compared to those in the bottom half (median 0.45% vs 0.35%, p = 0.004).

CONCLUSIONS

Linguistic differences existed in the personal statements of top- and bottom-ranked applicants to a general surgery residency program. These findings provide an adjunctive tool for differentiating applicants based on this underutilized component of the ranking process.

Evaluating the Standardized Letter of Recommendation Form in Applicants to Orthopaedic Surgery Residency

INTRODUCTION

A standardized letter of recommendation (SLOR) form for orthopaedic surgery residency programs has recently been adopted for use, but it has not been scientifically evaluated. The purpose of this study is to investigate the usefulness of the SLOR form in the selection process.

METHODS

All SLOR forms submitted to our institution over a single application cycle were extracted and analyzed. The United States Medical Licensing Examination Step 1 scores, grades in clinical rotations, Alpha Omega Alpha status, and the number of publications were recorded for each applicant. Correlations were calculated with Spearman rho, and inter-rater reliability was evaluated by calculating intraclass correlation coefficients.

RESULTS

One thousand one hundred thirty-seven SLOR forms were analyzed for 513 applicants. There was substantial rank inflation with the SLOR form; the majority (92%) of applicants were rated as either ranked to match or in the top one-third of their rank list. Objective applicant factors such as grades and Step 1 scores demonstrated a very weak to nonexistent correlation with the summative rank (rho 0.07 to 0.13, P ≤ 0.012). Poor inter-rater reliability was observed with the intraclass correlation coefficient ranging from 0.22 to 0.33 (P < 0.001).

CONCLUSIONS

The usefulness of the SLOR form is limited by the very high ratings observed for all questions, and in particular, the final summative rank. Measures to reduce rank inflation must be implemented to improve the discriminant ability of the SLOR form, and if this cannot be accomplished, perhaps the form should be abandoned.

LEVEL OF EVIDENCE

Level III Retrospective.

The neurosurgery residency interview: assessing applicant perspectives on question content, utility, and stress

OBJECTIVE

Residency interviews are integral to the recruitment process yet imperfect. Through surveys of neurosurgery residency applicants, the authors describe interview content and the perceived utility and stress of topics from the applicant's perspective.

METHODS

All 2018-2019 neurosurgery resident applicants applying to three particular programs were surveyed. Across 10 interview topics, survey questions assessed topic frequency and the applicant's opinion of the utility and stress of each topic (Likert scale 1-5). Analyses included descriptive statistics, Spearman's rank correlation, and logistic regression.

RESULTS

One hundred thirty-three of 265 surveyed US residency applicants (50%) responded. Extracurricular activities, research, future career, non-medicine interests, and small talk were discussed in all interviews. The least frequent topics included neurosurgical knowledge assessment (79%) and manual dexterity tests (45%). The most useful topics according to respondents were future career objectives (4.78 ± 0.49) and prior research (4.76 ± 0.50); the least useful were neurosurgical knowledge assessment (2.67 ± 1.09) and manual dexterity tests (2.95 ± 1.05). The most stressful topics were neurosurgical knowledge assessment (3.66 ± 1.23) and ethical/behavioral scenarios (2.94 ± 1.28). The utility and stress of manual dexterity tests and neurosurgical knowledge assessments were inversely correlated (r = -0.40, p < 0.01; r = -0.36, p < 0.01), whereas no such correlation existed for ethical/behavioral questions (r = -0.12, p = 0.18), indicating that ethical/behavioral questions may have been stressful but were potentially useful topics. Respondents who attended ≥ 15 interviews were more likely to be asked about the three most stressful topics (each p < 0.05). Respondents with children were less likely to be asked about ethical/behavioral scenarios (OR 0.13, 95% CI 0.03-0.52, p < 0.01).

CONCLUSIONS

Applicants found several of the most frequently discussed topics to be less useful, indicating a potential disconnect between applicant opinion and the faculty's preferred questions. Ethical/behavioral scenarios were rated as stressful but still useful, representing a potentially worthwhile type of question. These data provide several avenues for potential standardization and improvement of the interview process.

Can Better Selection Tools Help Us Achieve Our Diversity Goals in Postgraduate Medical Education? Comparing Use of USMLE Step 1 Scores and Situational Judgment Tests at 7 Surgical Residencies

PURPOSE

Use of the United States Medical Licensing Examination (USMLE) for residency selection has been criticized for its inability to predict clinical performance and potential bias against underrepresented minorities (URMs). This study explored the impact of altering traditional USMLE cutoffs and adopting more evidence-based applicant screening tools on inclusion of URMs in the surgical residency selection process.

METHOD

Multimethod job analyses were conducted at 7 U.S. general surgical residency programs during the 2018-2019 application cycle to gather validity evidence for developing selection assessments. Unique situational judgment tests (SJTs) and scoring algorithms were created to assess applicant competencies and fit. Programs lowered their traditional USMLE Step 1 cutoffs and invited candidates to take their unique SJT. URM status (woman, racial/ethnic minority) of candidates who would have been considered for interview using traditional USMLE Step 1 cutoffs was compared with the candidate pool considered based on SJT performance.

RESULTS

A total of 2,742 general surgery applicants were invited to take an online SJT by at least 1 of the 7 programs. Approximately 35% of applicants who were invited to take the SJT would not have met traditional USMLE Step 1 cutoffs. Comparison of USMLE-driven versus SJT-driven assessment results demonstrated statistically different percentages of URMs recommended, and including the SJT allowed an average of 8% more URMs offered an interview invitation (P < .01).

CONCLUSIONS

Reliance on USMLE Step 1 as a primary screening tool precludes URMs from being considered for residency positions at higher rate than non-URMs. Developing screening tools to measure a wider array of candidate competencies can help create a more equitable surgical workforce.

The Emergency Medicine Group Standardized Letter of Evaluation as a Workplace-based Assessment: The Validity Is in the Detail

Introduction

Interest is growing in specialty-specific assessments of student candidates based on clinical clerkship performance to assist in the selection process for postgraduate training. The most established and extensively used is the emergency medicine (EM) Standardized Letter of Evaluation (SLOE), serving as a substitute for the letter of recommendation. Typically developed by a program’s leadership, the group SLOE strives to provide a unified institutional perspective on performance. The group SLOE lacks guidelines to direct its development raising questions regarding the assessments, processes, and standardization programs employ. This study surveys EM programs to gather validity evidence regarding the inputs and processes involved in developing group SLOEs.

Methods

A structured telephone interview was administered to assess the input data and processes employed by United States EM programs when generating group SLOEs.

Results

With 156/178 (87.6%) of Accreditation Council of Graduate Medical Education-approved programs responding, 146 (93.6%) reported developing group SLOEs. Issues identified in development include the following: (1) 84.9% (124/146) of programs limit the consensus process by not employing rigorous methodology; (2) several stakeholder groups (nurses, patients) do not participate in candidate assessment placing final decisions at risk for construct under-representation; and (3) clinical shift assessments don’t reflect the task-specific expertise of each stakeholder group nor has the validity of each been assessed.

Conclusion

Success of the group SLOE in its role as a summative workplace-based assessment is dependent upon valid input data and appropriate processes. This study of current program practices provides specific recommendations that would strengthen the validity arguments for the group SLOE.

Utilization of a Novel Program-Specific Evaluation Tool Results in a Decidedly Different Interview Pool Than Traditional Application Review

BACKGROUND

There are almost twice as many applicants as there are general surgery internships, each utilizing a common application with standard components. These elements are frequently not useful in determining affinity for a program or overall ability, and resultant poor fit may be partially responsible for program attrition. Alternative evaluation instruments would be beneficial to both programs and applicants.

METHODS

An application review committee comprised of resident representatives, faculty representing all program-affiliated institutions, and program leadership completed a written evaluation developed by a third party (SurgWise Consulting) that specializes in industrial and organizational psychology. The responses were compiled to create a standardized assessment tool. This assessment was sent to applicants who were subsequently ranked according to fit with our program. The pool of applicants was separately evaluated using our traditional application review. Two residents independently graded each applicant on a 5-point Likert scale to evaluate common application elements; applicants were subsequently assigned an overall score.

RESULTS

The assessment was completed by 507 (99%) of 512 qualifying applicants. Separately, 378 applications were reviewed by the traditional method for a total of 756 reviews. Of the 96 applicants identified by the assessment tool to invite for interviews, 22 (23%) qualified for interview invitations according to the traditional review method. The assessment produced 74 applicants that otherwise would not have been interviewed.

CONCLUSION

Traditional application review strategies have many shortcomings. A competency-based assessment tool in the residency application selection process identifies a pool of applicants not identified by traditional review methods.

A Narrative Review of the Evidence Supporting Factors Used by Residency Program Directors to Select Applicants for Interviews

BACKGROUND

Residency applicants feel increasing pressure to maximize their chances of successfully matching into the program of their choice, and are applying to more programs than ever before.

OBJECTIVE

In this narrative review, we examined the most common and highly rated factors used to select applicants for interviews. We also examined the literature surrounding those factors to illuminate the advantages and disadvantages of using them as differentiating elements in interviewee selection.

METHODS

Using the 2018 NRMP Program Director Survey as a framework, we examined the last 10 years of literature to ascertain how residency directors are using these common factors to grant residency interviews, and whether these factors are predictive of success in residency.

RESULTS

Residency program directors identified 12 factors that contribute substantially to the decision to invite applicants for interviews. Although United States Medical Licensing Examination (USMLE) Step 1 is often used as a comparative factor, most studies do not demonstrate its predictive value for resident performance, except in the case of test failure. We also found that structured letters of recommendation from within a specialty carry increased benefit when compared with generic letters. Failing USMLE Step 1 or 2 and unprofessional behavior predicted lower performance in residency.

CONCLUSIONS

We found that the evidence basis for the factors most commonly used by residency directors is decidedly mixed in terms of predicting success in residency and beyond. Given these limitations, program directors should be skeptical of making summative decisions based on any one factor.

Computerized Residency Interview Scheduling: A Randomized Controlled Trial of Categorical General Surgery Applicants

OBJECTIVE

Scheduling interviews can be stressful and time-intensive for general surgery applicants and program coordinators. The objectives of this study were to determine whether computerized scheduling program (CSP) would decrease time to schedule interviews, reduce workload for residency coordinators, and improve applicant satisfaction.

DESIGN

A prospective randomized controlled trial of 2 interview-scheduling methods was conducted. All categorical general surgery applicants selected to interview for the 2017 match were randomized to either standard e-mail/phone scheduling or CSP using InterviewBroker. Time required to schedule an interview, number of communications, reschedules, withdrawals, and cancellations were all recorded. Additionally, applicants completed a voluntary, anonymous 9-question paper survey on their interview date. The program director and interviewers were blinded to the experimental groups.

SETTING

A single general surgery residency program.

PARTICIPANTS

Participants in the study included all categorical general surgery applicants selected for an interview in the 2017 match cycle (N = 62 standard group, N = 62 CSP group).

RESULTS

The CSP group took less time to schedule interviews (9 minutes vs. 80 minutes; p < 0.01), had fewer e-mail/phone communications (3 vs. 1; p < 0.01), and more total rescheduling events (26 vs. 4; p = 0.03) when compared to the standard group. Survey responses showed that 55% of applicants used CSPs at 5 or fewer other programs. The CSP group reported increased overall satisfaction (80% vs. 56% very satisfied; p = 0.02) and access to preferred interview dates (80% vs. 53% very satisfied; p = 0.02). Overall, 77% of applicants responded that CSPs should be widely adopted among general surgery residency programs.

CONCLUSIONS

CSPs expedited interview scheduling, decreased workload for program coordinators, and improved general surgery applicant satisfaction. However, despite the benefits of CSPs for programs and applicants, CSP use is not widespread among general surgery residency programs. Adoption of CSPs by all programs could greatly improve interview-scheduling processes for applicants and programs.

The Revised 2017 MSPE: Better, But Not "Outstanding"

AIM

The medical student performance evaluation (MSPE) is relied on as an objective summary evaluation by surgical program directors. In 2017, an MSPE task force released recommendations for best practice for their format and content. The purpose of this study was to analyze US medical schools' adherence to these guidelines.

METHODS

MSPEs from 113 of 147 Liaison committee on Medical Education (LCME)-accredited medical schools were analyzed for measurable attributes such as word counts, transparent clerkship grading, comparative performance data, and statements of professionalism. 2017 MSPEs were compared to a baseline group of 45 MSPEs from 2016 to measure change over time. Measurable attributes were compared using the Fisher exact and Mann Whitney-U tests. A p value < 0.05 was deemed statistically significant.

RESULTS

We analyzed 113 MSPEs from 2017. The median page count decreased by one from the prior year, with a narrower range of variation. 96% of schools reported a discreet grade in surgery. We observed substantial compliance with the recommendation for a statement of professionalism, noteworthy characteristics, and comparative clerkship data. More schools were observed to report school-wide rankings. There were significant variations in the graphical depiction of student achievement.

CONCLUSIONS

In response to the 2017 task force guidelines, MSPEs have become more standardized and transparent with regard to medical student evaluation. There is increased (but not ubiquitous) adherence with the recommendation for three noteworthy characteristics and statements of professionalism. Of particular importance to surgical program directors, 95.6% of 2017 MSPEs report a grade in the surgical clerkship and 85.8% include school-wide comparative clerkship performance data. Still, only 69.9% currently report school-wide summative performance data.

Use of Standard Guidelines for Department of Medicine Summary Letters

Phenomenon: Fourth-year medical students obtain Department of Medicine (DOM) letters ("Chair" letters) to support their residency applications. Writing and interpreting DOM letters are challenging. There is heterogeneity in the letters that makes it difficult to both write and read these letters.

APPROACH

The purpose of this study is to determine the value of new guidelines developed by a task force of clerkship directors and program directors in internal medicine and assess the implementation of these guidelines. The Clerkship Directors in Internal Medicine performed a cross-sectional survey of internal medicine clerkship directors at U.S. and Canadian medical schools in 2014. In addition, the primary author's institution reviewed 1,347 DOM letters between 2012 and 2014 to assess the implementation of these guidelines.

FINDINGS

The survey response rate was 78%. DOM letter writers reported the guidelines were better, easier to implement, and more compatible with the purpose of DOM letters than previously. Most letter readers reported that letters using the guidelines were more credible. Writers of DOM letters in lower academic ranks rated the letters with guidelines higher in several domains than those in higher academic ranks. Readers of DOM letters in higher academic ranks rated the letters with guidelines higher in several domains than those in lower academic ranks. In the DOM letters examined, the odds of meeting each guideline increased with each additional year. However, for 3 guidelines there was an initial decline in adherence from 2012 to 2013 before increasing again in 2014-the recommended length, clerkship description, and detailed narrative guidelines. Letters solely written by a chair were less likely to incorporate the guidelines. Insights: Clerkship directors often write the DOM letters and identify with the purpose of the guidelines. As writers, lower ranking academic faculty value the guidelines more than higher ranking academic faculty. As readers of DOM letters, higher academic ranking faculty value letters that incorporate the guidelines more than lower academic ranking faculty. DOM letters implemented more guideline criteria since the guidelines were released. If implementing the guidelines, chairs should solicit the help of their clerkship director or educational representative when writing DOM letters. Although many clerkship directors read letters for their residency programs, additional program directors' opinions are needed.