Resident selection: how we are doing and why?

Modeling Technical Skills Learning Curves of Incoming Surgical Trainees

OBJECTIVE

Not all trainees reach technical competency even after completing surgical training. While assessment of technical skill is not part of the residency interview process, identifying under-performers early on may help identify opportunities for individualized, targeted training. The objectives of this study were to (1) create predictive learning curve (LC) models for each of 3 basic laparoscopic tasks to identify performers versus underperformers and (2) evaluate the use of LCs to identify underperformers during selection into surgical training.

DESIGN

Predictive LC models were created for laparoscopic pattern cutting (PC), peg transfer (PT) and intra-corporeal knots (IC) over 40 repetitions by 65 novice trainees in 2014. Trainees were categorized as performers and underperformers. Receiver operator characteristic analysis determined the minimum number of repetitions required to predict individual LCs, which were then used to determine the proportion of underperformers.

SETTING

Technical performance was assessed onsite at the Canadian Residence Matching Service (CaRMS) interviews, after interview completion (January 2015).

PARTICIPANTS

Applicants to general surgery (GS) and gynecology (OBGYN) participated in a skills assessment during.

RESULTS

The PC, PT and IC tasks required a minimum of 8, 10, and 5 repetitions respectively, to predict overall performance. Predictive values for each task had excellent sensitivity and specificity: 1.00, 1.00 (PC); 1.00, 1.00 (PT); and 0.94, 1.00 (IC). Eighty applicants completed 8 PC repetitions; 16% were identified as underperformers.

CONCLUSIONS

Individual LCs for three different laparoscopic tasks can be predicted with excellent sensitivity and specificity based on 10 repetitions or less. This information can be used to identify trainees who may have difficulty with laparoscopic technical skills early on.

Recent Trends in Spine Topics on the Orthopaedic In-Training Examination

INTRODUCTION

The Orthopaedic In-Training Examination (OITE) is an important metric for orthopaedic residents and residency programs to gauge a resident's orthopaedic knowledge. Because the OITE is correlated with the likelihood of passing part I of the American Board of Orthopaedic Surgery, greater emphasis is being placed on the examination. However, a detailed look at the questions most likely to appear on the spine subsection of the OITE has not been done in the past decade.

METHODS

Digital copies of the OITEs during the years 2017 through 2021 were obtained online through the "ResStudy" program within the American Academy of Orthopaedic Surgeons Online Learning Platform. All spine-related questions were categorized into five different categories including type of spine question (knowledge-based, diagnosis, or evaluation/management), anatomical region, imaging modality provided, subject matter, and referenced journal or textbook. The total number and likelihood of each question type to appear on the OITE were defined as mean and percentage of the total number of spine questions, respectively.

RESULTS

A total of 139 spine questions were identified on the OITE during the years 2017 to 2021. The most common type of spine questions were evaluation/management (N = 65) and knowledge-based questions. We identified lumbar (N = 45), cervical (N = 42), thoracolumbar (N = 13), and thoracic (N = 12) as the most commonly tested anatomical regions. Spinal trauma (N = 26), disk disease/disk herniation (N = 16), postoperative complications (N = 15), and scoliosis/sagittal balance (N = 15) were the most commonly tested material. Spine (N = 54) was almost two times more likely to be referenced as the source for the tested material compared with other journals or textbooks.

CONCLUSIONS

Understanding the spine topics most likely to appear on the OITE may allow orthopaedic residents and residency programs to supplement educational objectives toward the highest yield spine topics and journals.

Do Orthopaedic In-Training Examination Scores Predict the Likelihood of Passing the American Board of Orthopaedic Surgery Part I Examination? An Update With 2014 to 2018 Data

INTRODUCTION

Previous research shows a correlation in performance between the Orthopaedic In-Training Examination (OITE) and the American Board of Orthopaedic Surgery (ABOS) Certifying Examination Part I; however, these studies are not current, with the most recent data from 2009. The purpose of this study was to update the relationship between the OITE and ABOS Part I scores with the five most recent nationwide cohorts of Part I scores and their corresponding OITE scores.

METHODS

The American Academy of Orthopaedic Surgeons provided the results for each resident taking the OITE from 2013 to 2017. The ABOS provided the results for each resident taking the Part I examination from 2014 to 2018. These two datasets were matched at the individual level and analyzed.

RESULTS

Between 2014 and 2018, 3,502 first-time test-takers were present for the ABOS Part I Examination. A 96.6% pass rate was noted (3,383 of 3,502). A statistically significant correlation between the OITE score and ABOS Part I score was observed at all levels of training: postgraduate year (PGY) 1 r = 0.380, PGY2 r = 0.463, PGY3 r = 0.498, PGY4 r = 0.504, and PGY5 r = 0.504 (P < 0.001 for all correlations).

CONCLUSION

The OITE scores continue to correlate with the ABOS scores and the pass rate for the ABOS with moderate strength correlation. However, although a correlation of the examinations exists when evaluated as a group, the predictive value of the OITE for passing the Part I examination for any specific individual is far from perfect. Individuals with PGY5 OITE percentile scores less than 10 pass the Part I examination, and individuals with PGY5 OITE percentile scores greater than 90 fail the Part I examination.

You never forget your first? Impact of interview timing on institutional rank order

OBJECTIVE

To analyze the impact of interview date on the applicant rank for Neurology residencies in the United States.

METHODS

A multi-institutional retrospective review of interview dates and applicant rank list data for the National Resident Matching Program (NRMP) was conducted for five Neurology programs, totaling 1932 interviewed applicants over a combined total of 31 interview years. For each candidate, the interview date and applicant rank were abstracted along with the total number of interviews for that season. Statistical analyses were completed on the cumulative institution data set as well for each individual institution to assess for a possible relationship between interview date and applicant rank.

RESULTS

The cumulative institutional analysis showed that the mean applicant rank decreased as the interview season progressed. Applicants who interviewed on the first day of the interview season were ranked 11.4% higher than those who interviewed on the last interview day. Additionally, applicants interviewed on the first interview day more likely to be ranked higher when compared to all other interview dates. Independent analysis of each program's data identified comparable, statistically significant, differences in mean applicant rank and interview position at three out of the five institutions.

CONCLUSIONS

This study evaluated the impact of interview order on the ranking of applicants by Neurology residency programs, noting a temporal relationship with applicant rank and interview date. The primacy bias appreciated in our data merits further evaluation in other medical specialties. Strategies to minimize the impact of this bias should be employed by residency programs who use medical matching services.

Characterizing the use of class rank in evaluating applicants for veterinary internship and residency positions

OBJECTIVE

To characterize how class rank and other criteria are used to evaluate applicants for veterinary internship and residency positions.

SAMPLE

Program directors for 572 internship and residency programs.

PROCEDURES

A survey was sent to program directors asking them to score the importance of 7 items (cover letter, letters of reference, curriculum vitae, veterinary class rank, grade point average, grades for classes specifically related to the internship or residency specialty area, and interview) they could use in evaluating applicants for an internship or residency and to rank those 7 items, along with an open item asking participants to list other criteria they used, from most to least important.

RESULTS

Responses were obtained for 195 internship and 222 residency programs. For both internship programs and residency programs, mean importance scores assigned to the 7 items resulted in the same ordering from most to least important, with letters of reference, interview, curriculum vitae, and cover letter most important. Rankings of the importance of the 7 items, along with an "other" item, were similar for internship and residency programs; the most important item was a candidate's letters of reference, followed by the interview, cover letter, and curriculum vitae.

CONCLUSIONS AND CLINICAL RELEVANCE

Results suggested that although most veterinary internship and residency programs consider class rank and overall grade point average when evaluating applicants, these 2 items were not the most important. For both internship and residency programs, the most important items were an applicant's letters of reference, followed by the interview, cover letter, and curriculum vitae. (J Am Vet Med Assoc 2021;258:776-785).

Resident Selection in the Wake of United States Medical Licensing Examination Step 1 Transition to Pass/Fail Scoring

INTRODUCTION

The numeric score for the United States Medical Licensing Examination Step 1 is one of the only universal, objective, scaled criteria for comparing the many students who apply to orthopaedic surgery residency. However, on February 12, 2020, it was announced that Step 1 would be transitioning to pass/fail scoring. The purpose of this study was to (1) determine the most important factors used for interview and resident selection after this change and (2) to assess how these factors have changed compared with a previous report on resident selection.

METHODS

A survey was distributed to the program directors (PDs) of all 179 orthopaedic surgery programs accredited by the Accreditation Council for Graduate Medical Education. Questions focused on current resident selection practices and the impact of the Step 1 score transition on expected future practices.

RESULTS

A total of 78 PDs (44%) responded to the survey. Over half of PDs (59%) responded that United States Medical Licensing Examination Step 2 clinical knowledge (CK) score is the factor that will increase most in importance after Step 1 transitions to pass/fail, and 90% will encourage applicants to include their Step 2 CK score on their applications. The factors rated most important in resident selection from zero to 10 were subinternship performance (9.05), various aspects of interview performance (7.49 to 9.01), rank in medical school (7.95), letters of recommendation (7.90), and Step 2 CK score (7.27). Compared with a 2002 report, performance on manual skills testing, subinternship performance, published research, letters of recommendations, and telephone call on applicants' behalf showed notable increases in importance.

DISCUSSION

As Step 2 CK is expected to become more important in the residency application process, current applicant stress on Step 1 scores may simply move to Step 2 CK scores. Performance on subinternships will remain a critical aspect of residency application, as it was viewed as the most important resident selection factor and has grown in importance compared with a previous report.

Personality Assessment and Emotional Intelligence in Orthopaedic Surgery Residency Selection and Training

Orthopaedic surgery has increasingly emerged as one of the most popular and competitive of all medical specialties. The selection of the best applicants has become a more difficult process because the number of qualified medical students has concomitantly grown. Although there are standardized guidelines in residency selection, there are several intangible factors that determine an applicant's fit into a program. Personality assessments, such as the Neuroticism-Extraversion-Openness Personality Inventory and the Myers-Briggs Type Indicator, are proficiencies that have garnered significant interest to help fill this void. Understanding and measuring empirically supported measures of personality traits and styles of medical students and residents may be valuable to medical educators and program directors in a variety of applications such as residency selection, mentoring, and education. Similar to personality identification, emotional intelligence assessments, such as the Mayer-Salovey-Caruso Emotional Intelligence Test, can prove to be pivotal tools in residency education and training. Emotional intelligence has shown to align with current Accreditation Council for Graduate Medical Education residency standardized core competencies that emphasize aptitude in noncognitive characteristics.

Use of Standardized Letters of Recommendation for Orthopaedic Surgery Residency Applications: A Single-Institution Retrospective Review

BACKGROUND

Standardized letters of recommendation (SLORs) were introduced to provide a more objective method of evaluating applicants for orthopaedic surgery residency positions. We sought to establish whether there exists an association between the SLOR summative rank statement (SRS), which is a question that asks the letter-writing authors where they would rank a student relative to other applicants, and success in matching into orthopaedic surgery residency.

METHODS

We reviewed 858 applications to an orthopaedic surgery residency program from 2017 to 2018. Data on 9 assessment categories, SRSs, and written comments in the SLORs were extracted. The match success of applicants was determined by an internet search algorithm. Logistic regression was used to evaluate the association between the SRSs and match success. The Spearman correlation was performed between the SRSs and other variables.

RESULTS

Only 60% of all LORs were SLORs. With 24% of the SLORs, a supplemental letter was used. Median percentile rank ranged from 90% to 100% for the 9 categories in the SLORs. Recommendations of "high rank" or higher were found in 88% of the SRSs. The mean of the SLOR SRSs was associated with match success.

CONCLUSIONS

The mean of the SLOR SRSs was associated with match success. However, the SLOR is not uniformly used. Future efforts should be devoted to improving question design and validity in order to better discriminate among applicants, increase adherence to the rating scale, and quantify the strength of the written comments in the SLOR.

Critical Assessment of the Contemporary Orthopaedic Surgery Residency Application Process

BACKGROUND

Matching into orthopaedic surgery residency in the United States has become an increasingly competitive process because of the large number of well-qualified applicants. Over the past several years, applicants have sought to maximize their chances of matching by submitting an increasing number of applications. The purpose of this study was to assess trends in application numbers, applicant qualifications, and application reviews, with the goal of obtaining data to help inform future improvements in the orthopaedic surgery residency application process.

METHODS

Applicant data were obtained from the Electronic Residency Application Service (ERAS, www.aamc.org/services/eras/stats) and the National Resident Matching Program (NRMP, www.nrmp.org/report-archives). These included residency application data from 2000 to 2017. In addition, we analyzed available NRMP Applicant Survey Reports between 2008 and 2017, Program Director Survey Reports between 2008 and 2016, and NRMP's Charting Outcomes in the Match between 2006 and 2016.

RESULTS

The number of U.S. senior medical student applicants per orthopaedic surgery residency position was stable from 2000 to 2017 (1.13 vs. 1.16 for 2000 and 2017, respectively). A significant increase in the United States Medical Licensing Examination (USMLE) Step-1 and Step-2 scores and self-reported research activity was present over the same time period. The number of applications submitted per applicant significantly increased, by 71.7%, from 48.4 in 2006 to 83.1 in 2017. Additionally, applications per program increased 46.4% from 457 in 2010 to 669 in 2016. In 2010, programs performed in-depth reviews for 54% of applications; however, in 2016, in-depth reviews had decreased to 45% of applications.

CONCLUSIONS

Orthopaedic residency applicant USMLE scores and research productivity have increased over time. Concurrently, the average number of applications submitted per applicant has increased, with the average applicant applying to nearly half of all orthopaedic residency programs. Consequently, programs have seen more than double the number of applications over this study period. The accompanying decline in the proportion of applications undergoing in-depth review, along with the applicant and program resources associated with these changes, warrants the development of strategies to enhance the efficiency of the application process for orthopaedic residency.

Optimizing the Orthopaedic Medical Student Rotation: Keys to Success for Students, Faculty, and Residency Programs

Senior medical students interested in pursuing careers in orthopaedic surgery participate in orthopaedic rotations around the country. These rotations are an important part of the application process because they allow students to demonstrate their work ethic and knowledge and learn more about the fit and culture of the residency program. Although knowledge and technical ability are important, several less tangible factors also contribute to success. These include maintaining situational awareness and a positive attitude, putting forth an appropriate effort, preparing effectively, and critically evaluating one's own performance. Attention to these details can help maximize the student's chance for a successful rotation. The hosting program and faculty can further facilitate a successful rotation by setting appropriate expectations, orienting the student to the program, carefully selecting appropriate services and faculty, and providing dedicated education to the student.

Beyond the United States Medical Licensing Examination Score: Assessing Competence for Entering Residency

Assessments of physician learners during the transition from undergraduate to graduate medical education generate information that may inform their learning and improvement needs, determine readiness to move along the medical education continuum, and predict success in their residency programs. To achieve a constructive transition for the learner, residency program, and patients, high-quality assessments should provide meaningful information regarding applicant characteristics, academic achievement, and competence that lead to a suitable match between the learner and the residency program's culture and focus.The authors discuss alternative assessment models that may correlate with resident physician clinical performance and patient care outcomes. Currently, passing the United States Medical Licensing Examination Step examinations provides one element of reliable assessment data that could inform judgments about a learner's likelihood for success in residency. Yet, learner capabilities in areas beyond those traditionally valued in future physicians, such as life experiences, community engagement, language skills, and leadership attributes, are not afforded the same level of influence when candidate selections are made.While promising new methods of screening and assessment-such as objective structured clinical examinations, holistic assessments, and competency-based assessments-have attracted increased attention in the medical education community, currently they may be expensive, be less psychometrically sound, lack a national comparison group, or be complicated to administer. Future research and experimentation are needed to establish measures that can best meet the needs of programs, faculty, staff, students, and, more importantly, patients.

Factors Predictive of Orthopaedic In-training Examination Performance and Research Productivity Among Orthopaedic Residents

INTRODUCTION

Selection of qualified candidates for orthopaedic residency is necessary for growth and innovation. The purpose of this study was to determine predictors of Orthopaedic In-training Exam (OITE) performance and research productivity.

METHODS

A survey was distributed to 13 residency programs collecting demographics, United States Medical Licensing Examination (USMLE) and OITE scores, and authored publications. Associations between preresidency qualifications and OITE scores and publications were determined.

RESULTS

A total of 274 of 294 surveys were returned (93.2%). We found a positive correlation between USMLE step 1 and 2 scores with recent OITE percentile (P < 0.001). Preresidency authorship (P < 0.001) and postgraduate training year (P < 0.001) were independent predictors of authorship during residency, whereas USMLE step 1 score was not (P = 0.094).

CONCLUSION

Candidates who perform well on the USMLE are likely to perform well on the OITE, whereas those with greater authored publications are likely to continue research during residency.

Matching into an Orthopedic Residency: Which Application Components Correlate with Final Rank List Order?

OBJECTIVE

To determine if any of the unique elements of the applications of medical students who were granted interviews to an orthopedic residency program correlate to the program's final rank order list of candidate residents.

DESIGN

Eight domains of the standard residency application were considered as independent variables for 36 applicants. Personal, identifying information was removed from the application material within each domain, thus blinding the application domains for 5 core faculty members from the program to review and rank independently. These 8 domain rank lists were then compared to the program's final rank list order to determine the correlation of each domain with the final rank list order.

SETTING

Academic medical center.

PARTICIPANTS

Applicants to a university-based orthopedic surgery residency program who were granted interviews in the 2016 academic year.

RESULTS

Two domains of the application correlated with the final rank list order: interview and personal statement. None of the other domains had a significant correlation with the final rank list order. Interobserver variability among the faculty members was high for the rankings of the different domains.

CONCLUSIONS

Interview and personal statement were the only domains within the application that had correlation with the final rank list for an orthopedic residency program. A better understanding of how these 2 components affect the rank list may provide opportunity for process improvements.

Predictors of orthodontic residency performance: An assessment of scholastic and demographic selection parameters

OBJECTIVE

To evaluate the association between resident selection criteria, including Graduate Record Examination (GRE) scores, and student performance in an orthodontic residency program.

MATERIALS AND METHODS

This retrospective study evaluated the academic records of 70 orthodontic residency graduates from the Indiana University School of Dentistry. The following demographic and scholastic data were extracted from the student academic records: applicant age, gender, ethnicity, race, country of origin, dental school graduation year, GRE score, and graduate orthodontic grade point average (GPA). In addition, student American Board of Orthodontics (ABO) written examination quintiles were obtained from the ABO. Scatterplots, analysis of variance, and correlation coefficients were used to analyze the data. Statistical significance was established at .05 for the study.

RESULTS

No associations were found with any component of the GRE, except with the quantitative GRE section, which displayed a weak association with ABO module 2 scores. Dental school GPA demonstrated weak correlations with all ABO modules and moderate correlations with overall and didactic orthodontic GPAs. When assessing demographic factors, significant differences (P < .05) were observed, with the following groups demonstrating higher performance on certain ABO modules: age (younger), race (whites), and country of origin (US citizens).

CONCLUSIONS

Findings suggest the GRE has no association with student performance in an orthodontic residency. However, dental school GPA and/or class rank appear to be the strongest scholastic predictors of residency performance.

Comparative Assessment of Grit, Conscientiousness, and Self-Control in Applicants Interviewing for Residency Positions and Current Orthopaedic Surgery Residents

OBJECTIVE

The purpose of this study was to quantify grit, conscientiousness, and self-control in orthopaedic residency applicants and current orthopaedic surgery residents. As part of a continual reassessment of the selection process, this study will help to improve this process by assessing the introduction of these non-cognitive assessments. This is the first study to both evaluate and compare the applicants' scores to those of current residents. This introduction will allow selection of not only the current top performers but those who have the wherewithal (read grit) to sustain their efforts throughout their residency.

DESIGN

A cross-sectional study composed of a confidential electronic survey consisting of a 17-item Grit scale, 10-item Self-control scale, and 9-item Conscientiousness scale was completed by medical school applicants and orthopaedic residents.

SETTING

Department of Orthopaedic Surgery, Hospital for Special Surgery.

PARTICIPANTS

Fifty-six (100%) medical student applicants (mean age = 27) were invited to participate in our study following a full day of interviews. Forty-five residents (mean age = 31) were asked and 32 (72%) completed the same surveys 4 months later.

RESULTS

There was a significant difference in grit for medical students (M = 4.19, SD = 0.34) and residents (M =3.86, SD = 0.48); t(86) = 3.76, p = 0.000. All grit subscales were also significantly different for medical students versus residents. Medical students (conscientiousness M = 4.60, SD = 0.41; self-control M = 3.51, SD = 0.30) and residents (conscientiousness M = 4.42, SD = 0.53; self-control M = 3.31, SD = 0.73) scored similarly in the conscientiousness t(86) = 1.75, p = 0.084 and self-control scales t(86) = 1.74, p = 0.086. Academic performance indicators such as the USMLE scores and residency ranking were also compared among medical student applicants.

CONCLUSIONS

The similar and above average levels of conscientiousness and self-control demonstrate the persevering nature of the individual who elects to pursue an orthopaedic residency program. Although the grit levels were different between medical school student and residents, they were above average for both groups, again demonstrating the type of individual willing to pursue an orthopaedic residency program. This study was the first to demonstrate varying degrees of grit for high-performing students versus residents in a competitive program, which seems to suggest that grit can vary over time. Future studies will investigate the validity of these non-cognitive variables in predicting achievement prospectively in a residency program.

A Cross-sectional Analysis of Minimum USMLE Step 1 and 2 Criteria Used by Orthopaedic Surgery Residency Programs in Screening Residency Applications

INTRODUCTION

The purpose of this study was to analyze how program directors (PDs) of orthopaedic surgery residency programs use United States Medical Licensing Examination (USMLE) Step 1 and 2 scores in screening residency applicants.

METHODS

A survey was sent to each allopathic orthopaedic surgery residency PD. PDs were asked if they currently use minimum Step 1 and/or 2 scores in screening residency applicants and if these criteria have changed in recent years.

RESULTS

Responses were received from 113 of 151 PDs (75%). One program did not have the requested information and five declined participation, leaving 107 responses analyzed. Eighty-nine programs used a minimum USMLE Step 1 score (83%). Eighty-three programs (78%) required a Step 1 score ≥210, 80 (75%) required a score ≥220, 57 (53%) required a score ≥230, and 22 (21%) required a score ≥240. Multiple PDs mentioned the high volume of applications as a reason for using a minimum score and for increasing the minimum score in recent years.

DISCUSSION

A large proportion of orthopaedic surgery residency PDs use a USMLE Step 1 minimum score when screening applications in an effort to reduce the number of applications to be reviewed.

A Comparison of Matched and Unmatched Orthopaedic Surgery Residency Applicants from 2006 to 2014: Data from the National Resident Matching Program

BACKGROUND

The Association of American Medical Colleges publishes residency match data and reports through the National Resident Matching Program (NRMP) every year. The purpose of this study was to analyze trends in orthopaedic surgery residency matching data and characteristics of successful applicants to counsel medical students with regard to their chances of matching.

METHODS

The annual reports of the NRMP were searched annually from 2006 to 2014 to determine the number of orthopaedic surgery residency positions available, the number of applicants, and the match rate among applicants. Comparisons were performed between matched applicants and unmatched applicants with regard to the number of contiguous ranks and distinct specialties, United States Medical Licensing Examination (USMLE) scores, number of research experiences and research products (abstracts, presentations, posters, publications), and proportion of Alpha Omega Alpha (AOA) Honor Medical Society members and students at a top-40, National Institutes of Health (NIH)-funded medical school.

RESULTS

The number of orthopaedic surgery positions available and number of applicants increased at a mean rate of 9 positions and 65 applicants per year (p = 0.11). The mean number of contiguous ranks for U.S. senior medical students was 11.5 for those who matched and 5.5 for those who did not match (p < 0.0001). The USMLE scores for applicants who matched were significantly greater than for those who did not match in each category: Step-1 scores for U.S. seniors (p < 0.001) and independent applicants (p = 0.039), and Step-2 scores for U.S. seniors (p < 0.01) and independent applicants (p = 0.026). The mean number of research products was significantly greater for matched U.S. seniors compared with unmatched U.S. seniors (p = 0.035). A significantly higher proportion of matched U.S. seniors compared with unmatched U.S. seniors were AOA members and students at a top-40, NIH-funded medical school (both p < 0.0001).

CONCLUSIONS

Successful applicants in the Match for orthopaedic surgery residency have higher USMLE Step-1 and 2 scores, number of research experiences and research products, and contiguous ranks. A higher proportion of successful applicants are AOA members and students at a top-40, NIH-funded medical school.

National Orthopedic Residency Attrition: Who Is At Risk?

BACKGROUND

National U.S. orthopedic resident attrition rates have been historically low, but no literature exists as to the characteristics of those who leave nor the circumstance of the departure. We aimed to determine factors that may place a resident at higher risk for attrition. Additionally, we planned to determine whether the 2003-work hour restriction affected attrition rate.

MATERIALS AND METHODS

All orthopedic surgery residency program directors in the United States were surveyed on demographic data for their current resident class, the number of residents who left the program, as well as demographic description for each of the residents who left their program from 1998 to 2013. Exclusion criteria included military programs and those younger than 3 years. All data were deidentified and compared to the Accreditation Council for Graduate Medical Education Data Resource book to protect against sample error in respondents.

RESULTS

Of 146 programs included, the overall response rate was 54% of residency directors, representing 51% of orthopedic residents. The respondent demographic make-up of 13.7% female, and average program size of 22.3 residents, compared similarly to the Accreditation Council for Graduate Medical Education national average of 13% female and 23-resident program size. Compared to all respondents, residents who left their program were more likely to be female (27%, p = 0.0018), single (51%, p = 0.0028), and without children (80%, p = 0.0018). There was no statistical difference based on minority status or 2003-instituted work hour restriction. Of those who left, 45% transferred to another specialty, 34% were dismissed, 14% voluntarily withdrew or cited personal reasons, and 6% transferred to another orthopedic program. The most common specialties to transfer into were radiology (30%), emergency medicine (25%), and anesthesia (18%).

CONCLUSIONS

Orthopedic residents who are female, single, or without children are statistically more likely to undergo attrition. Consideration could be given to targeted mentoring of these resident groups.

Academic Characteristics of Orthopaedic Surgery Residency Applicants from 2007 to 2014

BACKGROUND

Based on a relatively stable match rate, several authors have concluded that the competition for orthopaedic residency positions has not changed over the past 3 decades. However, the objective measures of applicant competitiveness have not been quantified in detail.

METHODS

National Resident Matching Program (NRMP) data from 2007 to 2014 for U.S. orthopaedic surgery applicants were compared with data for applicants to all specialties. Trends in the United Stated Medical Licensing Examination (USMLE) Step-1 and Step-2 scores, publications and research experiences, Alpha Omega Alpha (AOA) status, and the presence of an advanced degree are reported.

RESULTS

From 2007 to 2014, the match rate for orthopaedic surgery applicants remained stable near 80% (p = 0.14). For orthopaedic applicants who matched, the mean USMLE Step-1 scores increased from 234 points in 2007 to 245 points in 2014 (p = 0.005), and the mean scores increased from 220 points in 2007 to 229 points in 2014 for all applicants (p = 0.019). The mean USMLE Step-2 scores of orthopaedic applicants who matched increased from 235 points in 2007 to 251 points in 2014 (p = 0.005), and the mean scores of all applicants increased from 225 points in 2007 to 242 points in 2014 (p = 0.002). The mean number of research publications, presentations, and abstracts reported by orthopaedic applicants who matched more than doubled from 3.0 in 2007 to 6.7 in 2014 (p = 0.02) and increased less dramatically for all applicants from 2.2 in 2007 to 4.2 in 2014 (p = 0.004). The percentage of orthopaedic applicants elected to AOA or with advanced degrees did not significantly change (p > 0.2). Although orthopaedic applicants with AOA status experienced a very high match rate (97.1% in 2014), those with advanced degrees experienced match rates similar to or slightly lower than the applicant pool (73.7% in 2014).

CONCLUSIONS

The USMLE Step-1 and 2 scores of U.S. orthopaedic surgery residency applicants have increased significantly from 2007 to 2014. Additionally, the number of publications and presentations reported by orthopaedic applicants has more than doubled. These factors signal an increasing level of academic accomplishment in orthopaedic surgery applicants despite a consistent match rate.

Does Residency Selection Criteria Predict Performance in Orthopaedic Surgery Residency?

BACKGROUND

More than 1000 candidates applied for orthopaedic residency positions in 2014, and the competition is intense; approximately one-third of the candidates failed to secure a position in the match. However, the criteria used in the selection process often are subjective and studies have differed in terms of which criteria predict either objective measures or subjective ratings of resident performance by faculty.

QUESTIONS/PURPOSES

Do preresidency selection factors serve as predictors of success in residency? Specifically, we asked which preresidency selection factors are associated or correlated with (1) objective measures of resident knowledge and performance; and (2) subjective ratings by faculty.

METHODS

Charts of 60 orthopaedic residents from our institution were reviewed. Preresidency selection criteria examined included United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores, Medical College Admission Test (MCAT) scores, number of clinical clerkship honors, number of letters of recommendation, number of away rotations, Alpha Omega Alpha (AOA) honor medical society membership, fourth-year subinternship at our institution, and number of publications. Resident performance was assessed using objective measures including American Board of Orthopaedic Surgery (ABOS) Part I scores and Orthopaedics In-Training Exam (OITE) scores and subjective ratings by faculty including global evaluation scores and faculty rankings of residents. We tested associations between preresidency criteria and the subsequent objective and subjective metrics using linear correlation analysis and Mann-Whitney tests when appropriate.

RESULTS

Objective measures of resident performance namely, ABOS Part I scores, had a moderate linear correlation with the USMLE Step 2 scores (r = 0.55, p < 0.001) and number of clinical honors received in medical school (r = 0.45, p < 0.001). OITE scores had a weak linear correlation with the number of clinical honors (r = 0.35, p = 0.009) and USMLE Step 2 scores (r = 0.29, p = 0.02). With regards to subjective outcomes, AOA membership was associated with higher scores on the global evaluation (p = 0.005). AOA membership also correlated with higher global evaluation scores (r = 0.60, p = 0.005) with the strongest correlation existing between AOA membership and the "interpersonal and communication skills" subsection of the global evaluations.

CONCLUSIONS

We found that USMLE Step 2, number of honors in medical school clerkships, and AOA membership demonstrated the strongest correlations with resident performance. Our goal in analyzing these data was to provide residency programs at large a sense of which criteria may be "high yield" in ranking applicants by analyzing data from within our own pool of residents. Similar studies across a broader scope of programs are warranted to confirm applicability of our findings. The continually emerging complexities of the field of orthopaedic surgery lend increasing importance to future work on the appropriate selection and training of orthopaedic residents.

Standardized letters of recommendation and successful match into otolaryngology

OBJECTIVES/HYPOTHESIS

Historically, narrative letters of recommendation have been utilized in the selection of applicants for otolaryngology residency programs. In the last two application cycles, our specialty adopted a standardized letter of recommendation (SLOR). The intent was to decrease time burden for letter writers and to provide readers with an objective evaluation of applicants. The objective of this study was to determine attributes in the SLOR that correlate with matching into a residency program.

STUDY DESIGN

We performed a retrospective study using SLOR, United States Medical Licensing Examination (USMLE) step 1 scores, and matched outcomes of applicants who applied to our institution for the 2013 and 2014 match cycle.

METHODS

We included the following variables from the SLOR in the statistical analysis to determine which ones were associated with matching: patient care, medical knowledge, communication skills, procedural skills, research, initiative and drive, commitment to otolaryngology, commitment to academic medicine, match potential, and USMLE1 scores.

RESULTS

We identified 532 applicants and 963 SLOR. In successful applicants, scores for patient care, medical knowledge, communication skills, initiative and drive, and match potential were statistically higher (P < 0.05). Scores for professionalism, procedural skills, research, commitment to otolaryngology, commitment to academic medicine, and USMLE step 1 scores were not higher among successfully matched applicants.

CONCLUSION

Although SLOR can save time for letter writers and provide an objective description of applicants, the utility of individual domains within the SLOR is questionable. Additionally, it is concerning that applicants' professionalism and procedural skills are not correlated with matching in our specialty.

LEVEL OF EVIDENCE

NA. Laryngoscope, 126:1071-1076, 2016.

Examination of the Residency Interview Process for Academic Pathology Departments: How to Make the Most of a Resource-Heavy Process

Annual resident recruitment is a complex undertaking that requires many departmental resources of faculty time and effort and in many cases financial investment for meals and lodging. The applicants represent the future of the profession as well as the providers of patient care in the respective training programs. Although we understand the importance of this process, as we become more and more distracted by financial, administrative, and academic duties, the demands of recruitment have not decreased and continue annually. In an attempt to find the best practices for the improvement in our methods of recruitment, a review of the literature on the employment interviews with a specific eye to pathology residency relevant information was conducted. This article reviews some of the factors proven to be important to the applicants as well as an examination of the structure of the interview and the postinterview applicant evaluation process.

Use of the Interview in Resident Candidate Selection: A Review of the Literature

BACKGROUND

Although the resident candidate interview is costly and time-consuming for both applicants and programs, it is considered critically important for resident selection. Noncognitive attributes, including communication skills and professionalism, can be assessed by the personal interview.

OBJECTIVE

We conducted a review of the literature on the residency interview to identify the interview characteristics used for resident selection and to ascertain to what extent the interview yields information that predicts future performance.

METHODS

We searched PubMed and Scopus using the following search terms: residency, internship, interview, selection, and performance. We extracted information on characteristics of the interview process, including type of interview format, measures taken to minimize bias by interviewers, and testing of other clinical/surgical skills.

RESULTS

We identified 104 studies that pertained to the resident selection interview, with highly varied interview formats and assessment tools. A positive correlation was demonstrated between a medical school academic record and the interview, especially for unblinded interview formats. A total of 34 studies attempted to correlate interview score with performance in residency, with mixed results. We also identified a number of studies that included personality testing, clinical skills testing, or surgical skills testing.

CONCLUSIONS

Our review identified a wide variety of approaches to the selection interview and a range of factors that have been studied to assess its effectiveness. More research needs to be done not only to address and ascertain appropriate interview formats that predict positive performance in residency, but also to determine interview factors that can predict both residents' "success" and program attrition.

Aspectos generales y académicos de los* aspirantes y los* admitidos a la especialidad de pediatría de la Universidad Nacional de Colombia

<!--[if gte mso 9]><xml> <o:OfficeDocumentSettings> <o:RelyOnVML/> <o:AllowPNG/> </o:OfficeDocumentSettings> </xml><![endif]--><p class="MsoNormal" style="text-align: justify; text-justify: inter-ideograph; line-height: 200%;"><span style="font-size: 12.0pt; mso-bidi-font-size: 11.0pt; line-height: 200%; font-family: 'Verdana','sans-serif'; mso-ansi-language: ES-CO;" lang="ES-CO">Introducción: Existe escasa investigación local sobre el proceso de admisión a las especialidades medico-quirúrgicas. </span></p><p class="MsoNormal" style="text-align: justify; text-justify: inter-ideograph; line-height: 200%;"><span style="font-size: 12.0pt; mso-bidi-font-size: 11.0pt; line-height: 200%; font-family: 'Verdana','sans-serif'; mso-ansi-language: ES-CO;" lang="ES-CO">Objetivos: Caracterizar a los aspirantes y admitidos al programa de Pediatría de la Universidad Nacional de Colombia-UNal.</span></p><p class="MsoNormal" style="text-align: justify; text-justify: inter-ideograph; line-height: 200%;"><span style="font-size: 12.0pt; mso-bidi-font-size: 11.0pt; line-height: 200%; font-family: 'Verdana','sans-serif'; mso-ansi-language: ES-CO;" lang="ES-CO">Material y Métodos: Se estudiaron aspirantes y admitidos al programa, cohortes 2009 a 2011. Se estimó la tasa de admisión y los factores relacionados con ella.</span></p><p class="MsoNormal" style="text-align: justify; text-justify: inter-ideograph; line-height: 200%;"><span style="font-size: 12.0pt; mso-bidi-font-size: 11.0pt; line-height: 200%; font-family: 'Verdana','sans-serif'; mso-ansi-language: ES-CO;" lang="ES-CO">Resultados: Se registraron 997 aspirantes, 773 por primera vez<strong style="mso-bidi-font-weight: normal;">. </strong><span style="mso-spacerun: yes;"> </span>Predominó el sexo femenino (71%) con mediana de 28 años y provenientes de Bogotá. La graduación ocurrió en la mitad, en instituciones públicas y, en un quinto en la Unal. Se convocó a entrevista a 125 (12.5%) y 42 fueron aceptados (4.2% de los Inscritos y 34% de los convocados a entrevista).</span></p><p class="MsoNormal" style="text-align: justify; text-justify: inter-ideograph; line-height: 200%;"><span style="font-size: 12.0pt; mso-bidi-font-size: 11.0pt; line-height: 200%; font-family: 'Verdana','sans-serif'; mso-ansi-language: ES-CO;" lang="ES-CO">El promedio de calificación sobre 5, <span style="mso-spacerun: yes;"> </span>fue<span style="mso-spacerun: yes;">  </span>2.72 y 2.63 en la <em style="mso-bidi-font-style: normal;">Prueba Cognitiva</em> en la Prueba de <em style="mso-bidi-font-style: normal;">Inglés </em>respectivamente. En los preseleccionados pero no admitidos, la Calificación Final fue 3.19 y en los admitidos 3.54.</span></p><p class="MsoNormal" style="text-align: justify; text-justify: inter-ideograph; line-height: 200%;"><span style="font-size: 12.0pt; mso-bidi-font-size: 11.0pt; line-height: 200%; font-family: 'Verdana','sans-serif'; mso-ansi-language: ES-CO;" lang="ES-CO">Dentro de los admitidos predominaron la mujeres (60%) de 28 años, <span style="mso-spacerun: yes;"> </span>egresados de<span style="mso-spacerun: yes;">  </span>universidades públicas (88%). <span style="mso-spacerun: yes;"> </span>Robusta y significativamente, la <span style="mso-spacerun: yes;"> </span>admisión pudo ser predicha por la calificación en la <em style="mso-bidi-font-style: normal;">prueba cognitiva</em>, la calificación de la <em style="mso-bidi-font-style: normal;">Hoja de vida</em>, egresar de la Unal, <em style="mso-bidi-font-style: normal;">la calificación en la Entrevista</em>, el sexo masculino, tener menor edad y la calificación en la <em style="mso-bidi-font-style: normal;">prueba de inglés</em> (LR ji²(7)=307.93, Coeficiente-de-Determinación-CD:94.7%, p=0.0000). La Admisión es explicada, después de controlar por las demás variables, casi esclusivamente por la calificación en la <em style="mso-bidi-font-style: normal;">Prueba Cognitiva </em>(LR ji²(1)=300.53, CD:92.08%, p=0.0000).</span></p><p class="MsoNormal" style="text-align: justify; text-justify: inter-ideograph; line-height: 200%;"><span style="font-size: 12.0pt; mso-bidi-font-size: 11.0pt; line-height: 200%; font-family: 'Verdana','sans-serif'; mso-ansi-language: ES-CO;" lang="ES-CO">Conclusión</span></p><p class="MsoNormal" style="text-align: justify; text-justify: inter-ideograph; line-height: 200%;"><span style="font-size: 12.0pt; mso-bidi-font-size: 11.0pt; line-height: 200%; font-family: 'Verdana','sans-serif'; mso-ansi-language: ES-CO;" lang="ES-CO">Este trabajo muestra la consistencia y transparencia del proceso de admisión a la especialidad Pediatría de la Universidad Nacional de Colombia.</span></p><!--[if gte mso 9]><xml> <w:WordDocument> <w:View>Normal</w:View> <w:Zoom>0</w:Zoom> <w:TrackMoves/> <w:TrackFormatting/> <w:DoNotShowComments/> <w:HyphenationZone>21</w:HyphenationZone> <w:PunctuationKerning/> <w:ValidateAgainstSchemas/> <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> <w:IgnoreMixedContent>false</w:IgnoreMixedContent> <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> <w:DoNotPromoteQF/> <w:LidThemeOther>ES</w:LidThemeOther> <w:LidThemeAsian>X-NONE</w:LidThemeAsian> <w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript> <w:Compatibility> <w:BreakWrappedTables/> <w:SnapToGridInCell/> <w:WrapTextWithPunct/> <w:UseAsianBreakRules/> <w:DontGrowAutofit/> <w:SplitPgBreakAndParaMark/> <w:DontVertAlignCellWithSp/> <w:DontBreakConstrainedForcedTables/> <w:DontVertAlignInTxbx/> <w:Word11KerningPairs/> <w:CachedColBalance/> <w:UseFELayout/> </w:Compatibility> <w:DoNotOptimizeForBrowser/> <m:mathPr> <m:mathFont m:val="Cambria Math"/> <m:brkBin m:val="before"/> <m:brkBinSub m:val="&#45;-"/> <m:smallFrac m:val="off"/> <m:dispDef/> <m:lMargin m:val="0"/> <m:rMargin m:val="0"/> <m:defJc m:val="centerGroup"/> <m:wrapIndent m:val="1440"/> <m:intLim m:val="subSup"/> <m:naryLim m:val="undOvr"/> </m:mathPr></w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true" DefSemiHidden="true" DefQFormat="false" DefPriority="99" LatentStyleCount="267"> <w:LsdException Locked="false" Priority="0" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Normal"/> <w:LsdException Locked="false" Priority="9" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="heading 1"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/> <w:LsdException Locked="false" Priority="39" Name="toc 1"/> <w:LsdException Locked="false" Priority="39" Name="toc 2"/> <w:LsdException Locked="false" Priority="39" Name="toc 3"/> <w:LsdException Locked="false" Priority="39" Name="toc 4"/> <w:LsdException Locked="false" Priority="39" Name="toc 5"/> <w:LsdException Locked="false" Priority="39" Name="toc 6"/> <w:LsdException Locked="false" Priority="39" Name="toc 7"/> <w:LsdException Locked="false" Priority="39" Name="toc 8"/> <w:LsdException Locked="false" Priority="39" Name="toc 9"/> <w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/> <w:LsdException Locked="false" Priority="10" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Title"/> <w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/> <w:LsdException Locked="false" Priority="11" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/> <w:LsdException Locked="false" Priority="22" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Strong"/> <w:LsdException Locked="false" Priority="20" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/> <w:LsdException Locked="false" Priority="59" SemiHidden="false" UnhideWhenUsed="false" Name="Table Grid"/> <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/> <w:LsdException Locked="false" Priority="1" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false" UnhideWhenUsed="false" Name="Light Shading"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false" UnhideWhenUsed="false" Name="Light List"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false" UnhideWhenUsed="false" Name="Light Grid"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 1"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 2"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 1"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 2"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 1"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 2"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 3"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false" UnhideWhenUsed="false" Name="Dark List"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Shading"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful List"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Grid"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false" UnhideWhenUsed="false" Name="Light Shading Accent 1"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false" UnhideWhenUsed="false" Name="Light List Accent 1"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false" UnhideWhenUsed="false" Name="Light Grid Accent 1"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/> <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/> <w:LsdException Locked="false" Priority="34" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/> <w:LsdException Locked="false" Priority="29" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Quote"/> <w:LsdException Locked="false" Priority="30" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false" UnhideWhenUsed="false" Name="Dark List Accent 1"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful List Accent 1"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false" UnhideWhenUsed="false" Name="Light Shading Accent 2"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false" UnhideWhenUsed="false" Name="Light List Accent 2"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false" UnhideWhenUsed="false" Name="Light Grid Accent 2"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false" UnhideWhenUsed="false" Name="Dark List Accent 2"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful List Accent 2"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false" UnhideWhenUsed="false" Name="Light Shading Accent 3"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false" UnhideWhenUsed="false" Name="Light List Accent 3"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false" UnhideWhenUsed="false" Name="Light Grid Accent 3"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false" UnhideWhenUsed="false" Name="Dark List Accent 3"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful List Accent 3"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false" UnhideWhenUsed="false" Name="Light Shading Accent 4"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false" UnhideWhenUsed="false" Name="Light List Accent 4"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false" UnhideWhenUsed="false" Name="Light Grid Accent 4"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false" UnhideWhenUsed="false" Name="Dark List Accent 4"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful List Accent 4"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false" UnhideWhenUsed="false" Name="Light Shading Accent 5"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false" UnhideWhenUsed="false" Name="Light List Accent 5"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false" UnhideWhenUsed="false" Name="Light Grid Accent 5"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false" UnhideWhenUsed="false" Name="Dark List Accent 5"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful List Accent 5"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false" UnhideWhenUsed="false" Name="Light Shading Accent 6"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false" UnhideWhenUsed="false" Name="Light List Accent 6"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false" UnhideWhenUsed="false" Name="Light Grid Accent 6"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false" UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false" UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false" UnhideWhenUsed="false" Name="Dark List Accent 6"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful List Accent 6"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false" UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/> <w:LsdException Locked="false" Priority="19" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/> <w:LsdException Locked="false" Priority="21" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/> <w:LsdException Locked="false" Priority="31" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/> <w:LsdException Locked="false" Priority="32" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/> <w:LsdException Locked="false" Priority="33" SemiHidden="false" UnhideWhenUsed="false" QFormat="true" Name="Book Title"/> <w:LsdException Locked="false" Priority="37" Name="Bibliography"/> <w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/> </w:LatentStyles> </xml><![endif]--><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Tabla normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Cambria","serif"; mso-ascii-font-family:Cambria; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Cambria; mso-hansi-theme-font:minor-latin;} </style> <![endif]-->

Is orthopedics more competitive today than when my attending matched? An analysis of National Resident Matching Program data for orthopedic PGY1 applicants from 1984 to 2011

OBJECTIVE

This study evaluated supply and demand trends for orthopedic postgraduate year 1 (PGY1) positions from 1984 to 2011 for the purpose of estimating national intercandidate competition over time.

DESIGN

National Resident Matching Program (NRMP) data for orthopedic surgery from 1984 to 2011 were collected. Proxy variables including (total number of orthopedic applicants/number of orthopedic PGY1 positions), (number of US senior applicants to orthopedics/number of orthopedic PGY1 positions), (number of US seniors matching into orthopedics/number of US senior orthopedic applicants), (total number of matched orthopedic applicants/total number of orthopedic applicants), and (total number of US applicants who fail to match into orthopedics/total number of US senior applicants into orthopedics) as well as average United States Medical Licensing Examination Step 1 scores were used to gauge the level of competition between candidates and were compared over time.

SETTING

Academic medical center in the Midwestern United States.

PARTICIPANTS

Medical professors and medical students.

RESULTS

The NRMP data suggested that the number of positions per applicant decreased or remained stable since 1984 and that the percentage of applicants who did not match was no higher now than in the past. This finding was primarily because of the relative decrease in the ratio of applicants to available PGY1 positions, which stems from the number of positions increasing more rapidly than the number of applicants.

CONCLUSIONS

The NRMP data from 1984 to 2011 supported our hypothesis that intercandidate competition intensity for orthopedic PGY1 positions has not increased over time. The misconception that orthopedics is becoming more competitive likely arises from the increased number of applications submitted per candidate and the resulting relative importance placed on objective criteria such as United States Medical Licensing Examination Step 1 scores when programs select interview cohorts.

Applicant selection procedures in endodontic specialty programs in the United States: program director's perspective

INTRODUCTION

The purpose of this study was to determine the criteria evaluated by directors of endodontic specialty programs in the United States when selecting their residents and their satisfaction with the current process. Besides this, the study also aimed to determine the expected effect of the change in National Board Dental Examination (NBDE) score reporting to pass/fail on applicant evaluation.

METHODS

A 38-question web-based survey was distributed to the 54 endodontic specialty program directors (of 55 programs) in the United States. Questions regarded general program information, information obtained from applications, the interview process, the decision process, a retrospective view of the selection process, and director demographics.

RESULTS

Twenty-six (48.1%) responses were returned and analyzed. The most important application factors were interview ratings, dental school class rank, and general practice residency or advanced education in general dentistry experience. The most preferred sources for letters of recommendation were endodontic pre- and postdoctoral program directors and other academic endodontists. Desirable applicant characteristics included enthusiasm, listening skills, and verbal skills. Program directors indicated concern regarding the initial screening of applicants after the NBDE scoring change but not as much concern relating to final selection. Respondents generally supported a uniform acceptance date but did not support participation in the Postdoctoral Dental Matching Program (the Match).

CONCLUSIONS

Some important criteria when selecting prospective residents were identified. Program directors did not appear overwhelmingly concerned with the changes in NBDE score reporting. In addition, the establishment of a uniform acceptance date may resolve the biggest problem most respondents have with the current selection process.

Orthopaedic fellowship selection criteria: a survey of fellowship directors

BACKGROUND

The pursuit of a fellowship has become increasingly popular over the past several years, with &gt;90% of graduating orthopaedic residents applying for a fellowship position. Despite the ample literature available pertaining to the selection of orthopaedic residents, there is no similar research for the selection of fellows.

METHODS

Four hundred and fifteen of 475 orthopaedic fellowship program directors in the United States were surveyed on selection criteria used to choose fellows. The survey asked fellowship directors to rank the importance of various criteria on a Likert scale of 1 through 5 points (with 1 point denoting most important and 5 points denoting least important).

RESULTS

The overall response rate was 193 (46.5%) of 415 orthopaedic fellowship program directors. The most important criteria in selecting an applicant for an interview were a letter of recommendation from subspecialty faculty (1.38 points), quality of residency program (2.02 points), and a letter of recommendation from the residency program director (2.12 points). The most important criteria in completing the rank order list following the interview were the interview (1.17 points), a letter of recommendation from subspecialty faculty (1.46 points), a letter of recommendation from the residency program director (2.16 points), and expressed interest in program (2.16 points).

Associations between residency selection strategies and doctor performance: a meta-analysis

OBJECTIVES

The purpose of this study was to use meta-analysis to establish which of the information available to the resident selection committee is associated with resident or doctor performance.

METHODS

Multiple electronic databases were searched to 4 September 2012. Two reviewers independently selected studies that met the present inclusion criteria and extracted data in duplicate; disagreement was resolved by consensus. Risk for bias was assessed using a customised bias assessment tool. Measures of association were converted to a common effect size (Hedges' g). Meta-analysis was performed using the random-effects model for each selection strategy and all outcomes without pooling. Sensitivity analysis for each selection strategy-outcome pair was performed with pooling of effect size.

RESULTS

Eighty studies involving a total of 41 704 participants were included in the meta-analysis. Seventeen different selection strategies and 17 outcomes were assessed across these studies. The strongest positive associations referred to examination-based selection strategies, such as the US Medical Licensing Examination (USMLE) Step 1, and examination-based outcomes, such as scores on in-training examinations. Moderate positive associations were present for medical school marks and both examination-based and subjective outcomes. Minimal or no associations were seen for the selection tools represented by interviews, reference letters and deans' letters.

CONCLUSIONS

Standardised examination performance and medical school grades show the strongest associations with current measures of doctor performance. Deans' letters, reference letters and interviews all show a lower than expected strength of association given the relative value often assigned to them during resident doctor selection. Objective selection strategies are potentially the most useful to residency selection committees based on current evaluative methods. However, reports in the literature of validated long-term doctor performance outcomes are scant.

Aggregate Interview Method of ranking orthopedic applicants predicts future performance

This article evaluates and describes a process of ranking orthopedic applicants using what the authors term the Aggregate Interview Method. The authors hypothesized that higher-ranking applicants using this method at their institution would perform better than those ranked lower using multiple measures of resident performance. A retrospective review of 115 orthopedic residents was performed at the authors' institution. Residents were grouped into 3 categories by matching rank numbers: 1-5, 6-14, and 15 or higher. Each rank group was compared with resident performance as measured by faculty evaluations, the Orthopaedic In-Training Examination (OITE), and American Board of Orthopaedic Surgery (ABOS) test results. Residents ranked 1-5 scored significantly better on patient care, behavior, and overall competence by faculty evaluation (P<.05). Residents ranked 1-5 scored higher on the OITE compared with those ranked 6-14 during postgraduate years 2 and 3 (P⩽.5). Graduates who had been ranked 1-5 had a 100% pass rate on the ABOS part 1 examination on the first attempt. The most favorably ranked residents performed at or above the level of other residents in the program; they did not score inferiorly on any measure. These results support the authors' method of ranking residents. The rigorous Aggregate Interview Method for ranking applicants consistently identified orthopedic resident candidates who scored highly on the Accreditation Council for Graduate Medical Education resident core competencies as measured by faculty evaluations, performed above the national average on the OITE, and passed the ABOS part 1 examination at rates exceeding the national average.

The orthopaedic in-training examination: perspectives of program directors and residents from the United States and Canada

OBJECTIVE

Objective assessment of resident performance continues to task program directors (PDs) with a formidable challenge. This study evaluated attitudes toward the Orthopaedic In-Training Examination (OITE), compared its value between countries, assessed its value against other metrics of resident performance, and examined program and resident factors predictive of high achievement.

DESIGN

Survey.

SETTING

Orthopedic surgery residency programs across the United States and Canada.

PARTICIPANTS

One hundred sixty-six PDs and 945 residents.

RESULTS

Eighty-eight PDs and 331 residents completed the surveys (response rates, 54% and 35%, respectively). PDs and residents in the United States assigned greater importance to the OITE than did those in Canada and reported OITE scores from the United States were significantly higher. PDs in the United States reported greater consequences for residents with poor scores than did PDs from Canada, including remediation and reprimand. Observed structured clinical examinations, internal examinations, and in-training evaluation reports were assigned greater importance by PDs and residents in Canada, but low or no importance by those in the United States. In preparation for the OITE, residents strongly favored prior OITE and American Academy of Orthopaedic Surgeons self-assessment questions, the 'AAOS Comprehensive Orthopaedic Review' textbook, the Journal of the American Academy of Orthopaedic Surgeons, and an OITE-based multiple-choice question website. Regression analysis identified resident and program emphasis on OITE studying and higher level of training as positive predictors for higher OITE scores.

CONCLUSIONS

The OITE is more important to PDs and residents in the United States than it is in Canada, and the reported OITE scores reflect these attitudes. PDs in Canada also employ a greater diversity of evaluative tools, a practice in keeping with recent advances toward competency-based medical education. The findings of this report may help PDs be aware of alternative methods of formative resident evaluation and ultimately improve the training of future independent surgeons.

The relationship between the american board of anesthesiology part 1 certification examination and the United States medical licensing examination

BACKGROUND

The graduate medical education community uses results from the United States Medical Licensing Examination (USMLE) to inform decisions about individuals' readiness for postgraduate training.

OBJECTIVE

We sought to determine the relationship between performance on the USMLE and the American Board of Anesthesiology (ABA) Part 1 Certification Examination using a national sample of examinees, and we considered the relationship in the context of undergraduate medical education location and examination content.

METHODS

Approximately 7800 individuals met inclusion criteria. The relationships between USMLE scores and ABA Part 1 pass rates were examined, and predictions for the strength of the relationship between USMLE content areas and ABA performance were compared with observed relationships.

RESULTS

Pearson correlations between ABA Part 1 scores and USMLE Steps 1, 2 (clinical knowledge), and 3 scores for first-taker US/Canadian graduates were .59, .56, and .53, respectively. A clear relationship was demonstrated between USMLE scores and pass rates on ABA Part 1, and content experts were able to successfully predict the USMLE content categories that would least or most likely relate to ABA Part 1 scores.

CONCLUSIONS

The analysis provided evidence on a national scale that results from the USMLE and the ABA Part 1 were correlated and that success on the latter examination was associated with level of USMLE performance. Both testing programs have been successful in conceptualizing many of the knowledge areas of interest and in developing test content to reflect those areas.

The surgical residency interview: a candidate-centered, working approach

PURPOSE

The interview process is a pivotal, differentiating component of the residency match. Our bias is toward a working interview, producing better fulfillment of the needs of both parties, and a more informed match selection for the candidates and program.

METHODS

We describe a "candidate-centered" approach for integrating applicant interviews into our daily work schedule. Applicants are informed upon accepting the interview of the working interview model. Our program offers 33 interview days over a 12-week period. A maximum of 5 applicants are hosted per day. Applicants are assigned to 1 of our general, thoracic, vascular, or plastic surgery teams. The interview day begins with the applicant changing into scrubs, attending a morning conference, and taking part in a program overview by a Chief Resident. Applicants join their host team where 4-8 hours are spent observing the operative team, on rounds and sharing lunch. The faculty and senior residents are responsible for interviewing and evaluating applicants though the Electronic Residency Application Service.

RESULTS

A total of 13 surgeons are involved in the interview process resulting in broad-based evaluations. Each surgeon interviewed between 3 and 12 applicants. Faculty rate this interview approach highly because it allows them to maintain a rigorous operative schedule while interacting with applicants. Current residents are engaged in welcoming applicants to view the program. Faculty and residents believe cooperating in a real world manner aids their assessment of the applicant. Applicants routinely provide positive feedback, relaying this approach is informative, transparent, and should be the "standard." Applicants believe they are presented a realistic view of the program. Ultimately, this candidate-centered process may be attributable to our resident cohort who exhibit high satisfaction, excellent resident morale, and very low dropout rate.

CONCLUSIONS

We present a candidate-centered, working interview approach used in the selection of general surgery residents. While it may require more resources than the traditional approach, it harbors advantages for the applicant and the program.

Can knowledge tests and situational judgement tests predict selection centre performance?

OBJECTIVES

Written tests are an integral part of selection into general practice specialty training in the UK. Evidence supporting their validity and reliability as shortlisting tools has prompted their introduction into the selection processes of other medical specialties. This study explores whether candidate performance on two written tests predicts performance on subsequent workplace-based simulation exercises.

METHODS

A prospective analysis of candidate performance (n = 135) during the general practice selection process was undertaken. Candidates were shortlisted using their scores on two written tests, a clinical problem-solving test (CPST) and a situational judgement test (SJT). Successful candidates then undertook workplace-based simulation exercises at a selection centre (SC). Scores on the CPST and SJT were correlated with SC scores. Regression analysis was undertaken to explore the predictive validity of the CPST and SJT for SC performance.

RESULTS

The data show that the CPST and SJT are predictive of performance in workplace-based simulations (r = 0.598 for the CPST, r = 0.717 for the SJT). The SJT is a better predictor of SC performance than the CPST (R(2) = 0.51 versus R(2) = 0.35). However, the two tests together provide the greatest degree of predictive ability, accounting for 57% of the variance seen in mean scores across SC exercises.

CONCLUSIONS

The CPST and SJT play valuable roles in shortlisting and are predictive of performance in workplace-based SC exercises. This study provides evidence for their continued use in selection for general practice training and their expansion to other medical specialties.

Analysis of the orthopedic in-training examination (OITE) musculoskeletal trauma questions

OBJECTIVES

Residency program directors are responsible for providing assessment and feedback about resident performance and for developing a comprehensive resident curriculum in orthopedic surgery. One measure of resident knowledge is the Orthopedic In-Training Examination (OITE). Scores of the OITE examination have been found to correlate with the American Board of Orthopedic Surgery Part 1 Certifying Examination. The purpose of this study was to identify commonly tested orthopedic trauma topics, the taxonomic distribution of questions, and literature references in the OITE to aid curriculum development and individual test preparation.

METHODS

The musculoskeletal trauma-related questions on the OITE during a 5-year period (2004-2008) were reviewed, and the number of questions, topics, taxonomic classification, and educational references associated with each question were analyzed.

RESULTS

Nearly 30% of questions each year consist of musculoskeletal trauma-related topics. Femur, tibia, and hip fractures were the most commonly tested topics. The majority (65.6%) of musculoskeletal trauma questions tested recall of specific facts. Examiners referenced primary literature sources (74.9%) more than textbooks (25.1%). The Journal of Bone and Joint Surgery (American) and the Journal of Orthopaedic Trauma were cited most, accounting for 44.3% of all journal references. Forty-seven percent of the primary references were published within 5 years of the test administration.

CONCLUSIONS

One method for assessing orthopedic knowledge is the OITE examination. Longitudinal analysis of trauma-related questions shows a consistent pattern of both topics and primary literature citation. This information may be used to help guide structured review for future OITE examinations and develop an orthopedic trauma curriculum for a residency program.

The Utility of Letters of Recommendation in Predicting Resident Success: Can the ACGME Competencies Help?

BACKGROUND

The Accreditation Council for Graduate Medical Education (ACGME) core competencies are used to assess resident performance, and recently similar competencies have become an accepted framework for evaluating medical student achievements as well. However, the utility of incorporating the competencies into the resident application has not yet been assessed.

PURPOSE

The objective of this study was to examine letters of recommendation (LORs) to identify ACGME competency-based themes that might help distinguish the least successful from the most successful residents.

METHODS

Residents entering a university-based residency program from 1994 to 2004 were retrospectively evaluated by faculty and ranked in 4 groups according to perceived level of success. Applications from residents in the highest and lowest groups were abstracted. LORs were qualitatively reviewed and analyzed for 9 themes (6 ACGME core competencies and 3 additional performance measures). The mean number of times each theme was mentioned was calculated for each student. Groups were compared using the χ(2) test and the Student t test.

RESULTS

Seventy-five residents were eligible for analysis, and 29 residents were ranked in the highest and lowest groups. Baseline demographics and number of LORs did not differ between the two groups. Successful residents had statistically significantly more comments about excellence in the competency areas of patient care, medical knowledge, and interpersonal and communication skills.

CONCLUSION

LORs can provide useful clues to differentiate between students who are likely to become the least versus the most successful residency program graduates. Greater usage of the ACGME core competencies within LORs may be beneficial.

Achieving educational excellence: a strategic initiative to enhance orthopedic resident academic performance

Orthopedic resident training involves not only the hands-on learning of surgery but also should equally involve instructing the core knowledge of musculoskeletal medicine. Our program has developed a strategy that enhances resident educational performance; the educational curriculum entails conferences daily. Conferences include gross and surgical anatomy, orthopedic basic science, multidisciplinary trauma, radiology, pathology, journal club, and orthopedic subspecialty conferences. The primary purpose of the conference schedule is to provide the residents with a comprehensive education in orthopedic surgery. It is not geared toward taking the Orthopaedic In-Training Examination (OITE). The OITE is administered annually by the American Academy of Orthopaedic Surgeons (AAOS) and serves as an objective measure of knowledge acquisition. There has been a scientifically validated correlation between performance on the OITE and passage of the American Board for Orthopaedic Surgery Part I Examination. As a collective program, we have achieved at or above the 98th percentile nationally from 2004 to 2009. This academic success has not impacted the total surgical case volume negatively or interfered with Residency Review Committee (RRC) policies.

Success in orthopaedic training: resident selection and predictors of quality performance

Multiple studies have attempted to determine which attributes are predictive of success during residency as well as the optimal method of selecting residents who possess these attributes. Factors that are consistently ranked as being important in the selection of candidates into orthopaedic residency programs include performance during orthopaedic rotation, United States Medical Licensing Examination (USMLE) Step 1 score, Alpha Omega Alpha Honor Medical Society membership, medical school class rank, interview performance, and letters of recommendation. No consensus exists regarding the best predictors of resident success, but trends do exist. High USMLE Step 1 scores have been shown to correlate with high Orthopaedic In-Training Examination scores and improved surgical skill ratings during residency, whereas higher numbers of medical school clinical honors grades have been correlated to higher overall resident performance, higher residency interpersonal skills grading, higher resident knowledge grading, and higher surgical skills evaluations. Successful resident performance can be measured by evaluating psychomotor abilities, cognitive skills, and affective domain.

Assessment for selection for the health care professions and specialty training: consensus statement and recommendations from the Ottawa 2010 Conference

Assessment for selection in medicine and the health professions should follow the same quality assurance processes as in-course assessment. The literature on selection is limited and is not strongly theoretical or conceptual. For written testing, there is evidence of the predictive validity of Medical College Admission Test (MCAT) for medical school and licensing examination performance. There is also evidence for the predictive validity of grade point average, particularly in combination with MCAT for graduate entry but little evidence about the predictive validity of school leaver scores. Interviews have not been shown to be robust selection measures. Studies of multiple mini-interviews have indicated good predictive validity and reliability. Of other measures used in selection, only the growing interest in personality testing appears to warrant future work. Widening access to medical and health professional programmes is an increasing priority and relates to the social accountability mandate of medical and health professional schools. While traditional selection measures do discriminate against various population groups, there is little evidence on the effect of non-traditional measures in widening access. Preparation and outreach programmes show most promise. In summary, the areas of consensus for assessment for selection are small in number. Recommendations for future action focus on the adoption of principles of good assessment and curriculum alignment, use of multi-method programmatic approaches, development of interdisciplinary frameworks and utilisation of sophisticated measurement models. The social accountability mandate of medical and health professional schools demands that social inclusion, workforce issues and widening of access are embedded in the principles of good assessment for selection.

How we select our residents--a survey of selection criteria in general surgery residents

INTRODUCTION

The future of general surgery depends on the quality of the resident trainees, and successful resident selection is a factor that is important in the process of high-quality surgical education.

METHODS

A 36-question survey regarding resident selection and the interview process was sent to surgical program directors, department chairs, and associate program directors across the United States and Canada.

RESULTS

In all, 262 valid replies were received (65%), of which 83% were program directors. University hospital programs accounted for 49% of the completed surveys. The mean yearly applicant number per residency program was 571. Most programs indicated that they strictly adhere to their selection criteria (82%). The screening selection is made by the program director in 62%. Only 31% of programs show their selection criteria on their web page. United States Medical Licensing Examination (USMLE) Step 1 is the single most important factor in screening criteria (37%), followed by USLME Step 2 (24%). A total of 96% of all programs have female residents, 66% have non-Liaison Committee on Medical Education graduates, and 38% have Doctor of Osteopathy (DO) residents. Final selection is made by the program director in 49%. Although research experience is considered in selection criteria (80%), only 46% of programs offer research opportunities to their residents and only 13% require 1-year of research. On a Likert 5-point scale, the interview is by far the most important factor (4.69), followed by Step 1 score (4.21), and letters of recommendation (4.02).

CONCLUSIONS

Even though all general surgery programs have a wide range of screening/selection criteria, USLME Step 1 is the single most important factor for preliminary screening, and the interview is the most important factor in determining the final selection. The final selection is relatively subjective and based on a combination of interview, USLME scores, research experience, and personal judgment.

Have personal statements become impersonal? An evaluation of personal statements in anesthesiology residency applications

STUDY OBJECTIVE

To evaluate personal statements submitted to a major academic anesthesiology program to determine the prevalence of common features and overall subjective quality, and to survey anesthesiology program directors as to how they utilized these statements during the resident selection process.

DESIGN

Structured analysis of de-identified personal statements and Internet-based survey of program directors.

SETTING

Large academic anesthesiology training program.

SUBJECTS

670 applicant personal statements and academic anesthesiology program directors.

MEASUREMENTS

Prevalence of 13 specific essay features and 8 quality ratings were calculated for the essays and correlated with other aspects of the residency application, as abstracted from the Electronic Residency Application Service (ERAS) files. A 6-question survey regarding use of personal statements was collected from program directors.

MAIN RESULTS

70 of 131 program directors queried responded to our survey. Interest in physiology and pharmacology, enjoyment of a hands-on specialty, and desire to comfort anxious patients were each mentioned in more than half of the essays. Candidates invited for an interview had essays that received higher quality ratings than essays of those not invited (P = 0.02 to P < 0.0001). Higher quality ratings were also strongly associated with graduation from a U.S. or Canadian medical school, applicant file screening score, female gender, and younger age. Interrater reliability was good (kappa 0.75-0.99 for structural features, and 0.45-0.65 for quality features). More than 90% of program directors found proper use of English to be a somewhat or very important feature of the essay. Only 41% found the personal statement to be very or somewhat important in selecting candidates for interview invitations. However, over 90% stated that they used the statements during actual interviews with invited applicants.

CONCLUSION

The data showed a high prevalence of common features found within personal statements and a general ambivalence amongst those program directors for whom the statements were intended.

Using United States Medical Licensing Examination(®) (USMLE) examination results to predict later in-training examination performance among general surgery residents

OBJECTIVE

To examine the relationship between and predictive nature of United States Medical Licensing Examination(®) (USMLE) Step 1, Step 2, and American Board of Surgery In-Training Examination (ABSITE) postgraduate year (PGY) 1-5 scores from 2 general surgery programs from 1999-2009, with a goal of discerning how Step 1 and Step 2 scores should be used in resident selection and screening.

DESIGN

A descriptive, retrospective, correlational study was conducted using data from existing program records.

SETTING

Two accredited Midwestern community teaching hospital general surgery residency programs.

PARTICIPANTS

Data were collected from the records of N = 34 residents completing the programs in the years 1999-2009.

RESULTS

Although a statistically significant correlation exists between USMLE Step 1 and ABSITE PGY 3 scores, Step 2 scores were more highly correlated to ABSITE scores from all years of residency training. In both hierarchical and simple regression models, Step 2 scores were superior in predicting ABSITE PGY 1-5 scores, with the strongest predictability for PGY 3 and PGY 5 scores.

CONCLUSIONS

USMLE Step 1 scores showed limited utility in predicting later ABSITE scores whereas Step 2 scores were more predictive of all years of ABSITE scores. These findings should prompt additional research into the relationship between examination performances at different points along the general surgery education continuum. In the mean time, general surgery program directors and faculty might do well to examine the relationships between Step 1 and Step 2 scores and ABSITE scores in their own programs to evaluate the usefulness of considering either score when ranking potential residents for selection into a program.

Correlation of USMLE Step 1 scores with performance on dermatology in-training examinations

BACKGROUND

Although United States Medical Licensing Examination (USMLE) Step 1 was not designed to predict resident performance, scores are used to compare residency applicants. Multiple studies have displayed a significant correlation among Step 1 scores, in-training examination (ITE) scores, and board passage, although no such studies have been performed in dermatology.

OBJECTIVE

The purpose of this study is to determine if this correlation exists in dermatology, and how much of the variability in ITE scores is a result of differences in Step 1 scores. This study also seeks to determine if it is appropriate to individualize expectations for resident ITE performance.

METHODS

This project received institutional review board exemption. From 5 dermatology residency programs (86 residents), we collected Step 1 and ITE scores for each of the 3 years of dermatology residency, and recorded passage/failure on boards. Bivariate Pearson correlation analysis was used to assess correlation between USMLE and ITE scores. Ordinary least squares regression was computed to determine how much USMLE scores contribute to ITE variability.

RESULTS

USMLE and ITE score correlations were highly significant (P < .001). Correlation coefficients with USMLE were: 0.467, 0.541, and 0.527 for ITE in years 1, 2, and 3, respectively. Variability in ITE scores caused by differences in USMLE scores were: ITE first-year residency = 21.8%, ITE second-year residency = 29.3%, and ITE third-year residency = 27.8%.

LIMITATIONS

This study had a relatively small sample size, with data from only 5 programs.

CONCLUSIONS

There is a moderate correlation between USMLE and ITE scores, with USMLE scores explaining ∼26% of the variability in ITE scores.

Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination?: a multicenter study

BACKGROUND

The US Medical Licensing Examination (USMLE) and Orthopaedic In-Training Examination (OITE) are commonly used to select medical students or residents, respectively. Knowing how well these examinations predict performance on the American Board of Orthopaedic Surgery (ABOS) Part I certifying examination is important to provide evaluations for medical students and residents. Previous studies comparing the OITE scores with the ABOS Part 1 scores have been limited to one program.

QUESTIONS/PURPOSES

Therefore, we compared the scores on the USMLE Step 1 and OITE examinations with those on the ABOS Part I certifying examination using data from four ACGME approved residency programs.

METHODS

We reviewed 202 resident files from 1996 to 2008 from four programs in the same geographic region. Of those, 181 (90%) had complete records. De-identified data were used to compare USMLE Step 1 scores, OITE percentile rank scores, and ABOS Part I percentile rank scores. Pearson coefficients and receiver operator curves were calculated to assess the relationships between tests.

RESULTS

We found a correlation of 0.53 between the USMLE Step 1 and ABOS Part I, and an average correlation of 0.50 for postgraduate years (PGY) 2 through 5 OITE scores and ABOS Part I. There was a stepwise increase in correlation from PGY 2 through PGY 5 between the OITE scores and ABOS scores. Those who averaged in the 27th percentile or lower on the OITE had a 57% chance of failing the ABOS Part I examination.

CONCLUSION

USMLE Step 1 scores correlated with ABOS Part I certifying examination scores, and we therefore believe it may be used as one factor in resident selection. Use of the OITE scores in guiding education and feedback appears to be justified.

Can medical school performance predict residency performance? Resident selection and predictors of successful performance in obstetrics and gynecology

BACKGROUND

During the evaluation process, Residency Admissions Committees typically gather data on objective and subjective measures of a medical student's performance through the Electronic Residency Application Service, including medical school grades, standardized test scores, research achievements, nonacademic accomplishments, letters of recommendation, the dean's letter, and personal statements. Using these data to identify which medical students are likely to become successful residents in an academic residency program in obstetrics and gynecology is difficult and to date, not well studied.

OBJECTIVE

To determine whether objective information in medical students' applications can help predict resident success.

METHOD

We performed a retrospective cohort study of all residents who matched into the Johns Hopkins University residency program in obstetrics and gynecology between 1994 and 2004 and entered the program through the National Resident Matching Program as a postgraduate year-1 resident. Residents were independently evaluated by faculty and ranked in 4 groups according to perceived level of success. Applications from residents in the highest and lowest group were abstracted. Groups were compared using the Fisher exact test and the Student t test.

RESULTS

Seventy-five residents met inclusion criteria and 29 residents were ranked in the highest and lowest quartiles (15 in highest, 14 in lowest). Univariate analysis identified no variables as consistent predictors of resident success.

CONCLUSION

In a program designed to train academic obstetrician-gynecologists, objective data from medical students' applications did not correlate with successful resident performance in our obstetrics-gynecology residency program. We need to continue our search for evaluation criteria that can accurately and reliably select the medical students that are best fit for our specialty.