Factors that Predict an Intern's First ABSITE Score are Known by September

The Plan-Do-Study-Act (PDSA): An Iterative Approach to Optimize Residents Performance in the American Board of Surgery in-Training Exam (ABSITE)

INTRODUCTION

American Board of Surgery (ABS) In-Training Examination (ITE), or ABSITE, preparation requires an effective study approach. In 2014, the ABS announced the alignment of ABSITE to the SCORE® Curriculum. We hypothesized that implementing a Plan-Do-Study-Act (PDSA) approach would help surgery residents improve their performance on the ABSITE.

METHOD

Over 20 years, in a single institution, residents' ABSITE performance was evaluated over 3 timeframes: Time A (2004-2013), no specific curriculum; Time B (2014-2019), an annual comprehensive ABSITE-simulated SCORE®-based multiple-choice exam (MCQ) was administered; and Time C (2020-2023), like Time B with the addition of the PDSA approach for those with less than 60% correct on the ABSITE-simulated SCORE®-based exam. At the beginning of the academic year, in July, all residents are encouraged to (1) initiate a study plan for the upcoming ABSITE using SCORE® guided by the published ABSITE outlines content topics (Plan), (2) take an ABSITE-simulated SCORE®-based exam in October (Do), (3) assess the results/scores (Study), and (4) identify appropriate next steps (Act). Correlational analysis was performed to evaluate the association between ABSITE scores and ABSITE-simulated SCORE®-based exam scores in Time B and Time C. The primary outcome was the change in the proportions of ABSITE scores <30th percentile.

RESULTS

A total of 294 ABSITE scores of 94 residents (34 females and 60 males) were analyzed. We found stronger correlation between the correct percentage on ABSITE and ABSITE-simulated SCORE®-based exam scores in Time C (r = 0.73, p < 0.0001) compared to Time B (0.62, p < 0.0001). The percentage of residents with ABSITE scores lower than 30th percentile dropped significantly from 14.0% to 3.7% (p = 0.016).

CONCLUSION

Implementing the Plan-Do-Study-Act (PDSA) approach using the SCORE® curriculum significantly enhances residents' performance on the ABSITE exam. Surgery residents are encouraged to use this approach and to utilize the SCORE-contents outlined by the ABS in their study plan.

Scheduling surgical success: Early minimally invasive surgery rotations as a predictor of ABSITE performance

INTRODUCTION

The American Board of Surgery In-Training Examination (ABSITE®) serves as a benchmark for assessing resident knowledge and comprehension in surgery training programs. While previous studies have examined factors such as USMLE Step 1 and Step 2 performance in ABSITE® preparation, the impact of rotational schedule on ABSITE® performance remains underexplored.

METHODS

We retrospectively investigated the independent influence of rotational schedule on ABSITE® performance among interns at a single academic institution over an 11 year period.

RESULTS

Early exposure to minimally invasive surgery rotations and specialty rotations such as surgical oncology and vascular surgery during the winter or just before ABSITE® was associated with higher ABSITE® percentiles when controlled for USMLE Step 2 performance.

CONCLUSION

This study underscores the significance of rotational scheduling in optimizing ABSITE® performance and suggests potential strategies for enhancing resident preparation and success on this examination.

Bias in Surgical Residency Evaluations: A Scoping Review

OBJECTIVE

Given widespread disparities in the surgical workforce and the advent of competency-based training models that rely on objective evaluations of resident performance, this review aims to describe the landscape of bias in the evaluation methods of residents in surgical training programs in the United States.

DESIGN

A scoping review was conducted within PubMed, Embase, Web of Science, and ERIC in May 2022, without a date restriction. Studies were screened and reviewed in duplicate by 3 reviewers. Data were described descriptively.

SETTING/PARTICIPANTS

English-language studies conducted in the United States that assessed bias in the evaluation of surgical residents were included.

RESULTS

The search yielded 1641 studies, of which 53 met inclusion criteria. Of the included studies, 26 (49.1%) were retrospective cohort studies, 25 (47.2%) were cross-sectional studies, and 2 (3.8%) were prospective cohort studies. The majority included general surgery residents (n = 30, 56.6%) and nonstandardized examination modalities (n = 38, 71.7%), such as video-based skills evaluations (n = 5, 13.2%). The most common performance metric evaluated was operative skill (n = 22, 41.5%). Overall, the majority of studies demonstrated bias (n = 38, 73.6%) and most investigated gender bias (n = 46, 86.8%). Most studies reported disadvantages for female trainees regarding standardized examinations (80.0%), self-evaluations (73.7%), and program-level evaluations (71.4%). Four studies (7.6%) assessed racial bias, of which all reported disadvantages for trainees underrepresented in surgery.

CONCLUSIONS

Evaluation methods for surgery residents may be prone to bias, particularly with regard to female trainees. Research is warranted regarding other implicit and explicit biases, such as racial bias, as well as for nongeneral surgery subspecialties.

The Association Between USMLE Step 2 Clinical Knowledge Scores and Residency Performance: A Systematic Review and Meta-Analysis

PURPOSE

With the change in Step 1 score reporting, Step 2 Clinical Knowledge (CK) may become a pivotal factor in resident selection. This systematic review and meta-analysis seeks to synthesize existing observational studies that assess the relationship between Step 2 CK scores and measures of resident performance.

METHOD

The authors searched MEDLINE, Web of Science, and Scopus databases using terms related to Step 2 CK in 2021. Two researchers identified studies investigating the association between Step 2 CK and measures of resident performance and included studies if they contained a bivariate analysis examining Step 2 CK scores' association with an outcome of interest: in-training examination (ITE) scores, board certification examination scores, select Accreditation Council for Graduate Medical Education core competency assessments, overall resident performance evaluations, or other subjective measures of performance. For outcomes that were investigated by 3 or more studies, pooled effect sizes were estimated with random-effects models.

RESULTS

Among 1,355 potential studies, 68 met inclusion criteria and 43 were able to be pooled. There was a moderate positive correlation between Step 2 CK and ITE scores (0.52, 95% CI 0.45-0.59, P < .01). There was a moderate positive correlation between Step 2 CK and ITE scores for both nonsurgical (0.59, 95% CI 0.51-0.66, P < .01) and surgical specialties (0.41, 95% CI 0.33-0.48, P < .01). There was a very weak positive correlation between Step 2 CK scores and subjective measures of resident performance (0.19, 95% CI 0.13-0.25, P < .01).

CONCLUSIONS

This study found Step 2 CK scores have a statistically significant moderate positive association with future examination scores and a statistically significant weak positive correlation with subjective measures of resident performance. These findings are increasingly relevant as Step 2 CK scores will likely become more important in resident selection.

Association Between USMLE Step 1 Scores and In-Training Examination Performance: A Meta-Analysis

PURPOSE

On February 12, 2020, the sponsors of the United States Medical Licensing Examination announced that Step 1 will transition to pass/fail scoring in 2022. Step 1 performance has historically carried substantial weight in the evaluation of residency applicants and as a predictor of subsequent subject-specific medical knowledge. Using a systematic review and meta-analysis, the authors sought to determine the association between Step 1 scores and in-training examination (ITE) performance, which is often used to assess knowledge acquisition during residency.

METHOD

The authors systematically searched Medline, EMBASE, and Web of Science for observational studies published from 1992 through May 10, 2020. Observational studies reporting associations between Step 1 and ITE scores, regardless of medical or surgical specialty, were eligible for inclusion. Pairs of researchers screened all studies, evaluated quality assessment using a modified Newcastle-Ottawa Scale, and extracted data in a standardized fashion. The primary endpoint was the correlation of Step 1 and ITE scores.

RESULTS

Of 1,432 observational studies identified, 49 were systematically reviewed and 37 were included in the meta-analysis. Overall study quality was low to moderate. The pooled estimate of the correlation coefficient was 0.42 (95% confidence interval [CI]: 0.36, 0.48; P < .001), suggesting a weak-to-moderate positive correlation between Step 1 and ITE scores. The random-effects meta-regression found the association between Step 1 and ITE scores was weaker for surgical (versus medical) specialties (beta -0.25 [95% CI: -0.41, -0.09; P = .003]) and fellowship (versus residency) training programs (beta -0.25 [95% CI: -0.47, -0.03; P = .030]).

CONCLUSIONS

The authors identified a weak-to-moderate positive correlation between Step 1 and ITE scores based on a meta-analysis of low-to-moderate quality observational data. With Step 1 scoring transitioning to pass/fail, the undergraduate and graduate medical education communities should continue to develop better tools for evaluating medical students.

Prospective Factors that Predict American Board of Surgery In-Training Examination Performance: A Systematic Review

INTRODUCTION

American Board of Surgery In-Training Examination (ABSITE) performance has become an important factor when monitoring resident progress. Understanding which prospective factors predict performance can help identify residents at risk.

METHODS

A literature search was conducted searching PubMed, EMBASE, and JAMA Network from June 2011 to June 2021, in accordance with the PRISMA guidelines. Searches were performed for the terms "ABSITE" and "American Board of Surgery In-Training Examination." Prospective factors such as prior examination performance, clinical evaluations, and demographics were evaluated.

RESULTS

A final 35 studies were included. The prospective factor most consistently found to predict ABSITE performance is performance on prior knowledge-based examinations such as the USMLE step exams. The ACGME Medical Knowledge 1 milestone evaluation also appears to correlate to ABSITE performance, although clinical evaluations, in general, do not. Demographics have no significant correlation to ABSITE performance.

DISCUSSION

Using performance on prior knowledge-based examinations programs may be able to identify residents at risk for failing ABSITE. It may be possible to initiate early intervention before rather than only remediation after poor performance.

Trainee Utilization of the SCORE Curriculum is Associated With Improved ABSITE Performance: A Multi-institutional Study

BACKGROUND

The Surgical Council on Resident Education's (SCORE) structured educational curriculum for general surgery residents uses a 2-year repeating cycle of modules and quizzes called "This Week in SCORE" (TWIS) to organize and sequence the curricular content on the SCORE portal. The purpose of this study was to evaluate the impact of using the SCORE content and TWIS curriculum on American Board of Surgery In-Training Examination (ABSITE) performance. To date, no multi-institutional studies have examined this effect.

METHODS

Eight residency programs participated, including university and community-based programs. SCORE usage overall, TWIS usage, and ABSITE percentile scores (adjusted for post-graduate year [PGY]) were analyzed for the academic years 2017 to 2020. SCORE usage was defined as number of SCORE logins annually per resident with "low usage" ≤10 times and "high usage" >10 times. TWIS usage was defined as "low usage" (no TWIS quizzes done) or "high usage" (≥1 TWIS quiz per year).

RESULTS

Four hundred and twenty-eight trainees were evaluated. Trainees with high SCORE and TWIS usage consistently achieved a higher ABSITE percentile score. This difference was maintained in subgroup analysis by PGY with the greatest impact in PGY-1 and PGY-2 levels.

CONCLUSIONS

Utilization of multiple aspects of SCORE content appears to have a significant positive impact on ABSITE performance across all levels of postgraduate training. This multi-institutional study of a large number of users is the first to demonstrate that increased usage of SCORE content appears to be a predictor of ABSITE performance success.

Anatomy Scholars Program for Medical Students Entering a Surgical Residency

A near-peer teaching experience for upper-level medical students could help prepare them for surgical residency by providing specific education opportunities and exposure to the field of surgery. Five medical students were selected to be near-peer teachers (NPTs) in gross anatomy, and then they reflected on their experiences. The NPTs spent the majority of effort in a teaching role, and reported improved NTS, anatomy knowledge, and dissection skills. MS1s and faculty also reported on the value of the ASP. Further development and evaluation of the ASP may be an excellent opportunity for future surgeons.

Spaced Education With ABSITE Quest Resulting in Improved American Board of Surgery In-Training Examination Performance

OBJECTIVE

The American Board of Surgery In-Training Examination (ABSITE) is an annual exam taken by general surgery residents as a cognitive assessment of the knowledge gained throughout each year of training. Several question banks are available for ABSITE preparation. However, ABSITE Quest (AQ) utilizes a method called spaced education which has been demonstrated to help with retaining information longer and improve exam performance. This study hypothesizes that using this method will help residents improve their ABSITE performance.

DESIGN

Retrospective survey data was collected from residents who participated in AQ, including postgraduate year (PGY) level, as well as 2019 and 2020 ABSITE percentiles. AQ user data was used to match respondent's total number of questions completed and daily engagement level to the survey data. Paired, single-tailed student's t test was used to evaluate the significance of ABSITE percentile change between 2019 and 2020 among AQ users.

SETTING

ChristianaCare, Newark, DE, United States. Nonclinical.

PARTICIPANTS

All ABSITE Quest users were surveyed (n = 390), of which 104 responded. 21 responses were from PGY1 residents and were excluded, resulting in a total of 83 responses.

RESULTS

The mean percentile difference of AQ users from 2019 to 2020 was +15.8 (p < 0.00001). When categorizing by the total number of questions completed, high users demonstrated a mean percentile difference of +15.3 (p = 0.00002), average users had a difference of +19.1 (p = 0.00029), and low users showed a percentile difference of +1.2 (p = 0.45244). When categorizing by daily engagement level, high users demonstrated a mean percentile difference of +17.9 (p < 0.00001), low users had a mean percentile difference of +15.3 (p = 0.00124), and minimal users showed a mean percentile change of -5.7.

CONCLUSIONS

The use of the spaced education method with ABSITE Quest, especially in users with a greater number of questions completed and high levels of daily engagement, correlated with a significant improvement on ABSITE performance.

The Surgical Knowledge "Growth Curve": Predicting ABSITE Scores and Identifying "At-Risk" Residents

OBJECTIVE

Resident performance on the American Board of Surgery In-Training Examination (ABSITE) is used for evaluation of surgical knowledge and guides resident selection for institutional remediation programs. Remediation thresholds have historically been based on ABSITE percentile scores; however, this does not account for predictors that can impact a resident's exam performance. We sought to identify predictors of yearly ABSITE performance to help identify residents "at-risk" for performing below their expected growth trajectory.

DESIGN

The knowledge of the residents, as measured by standardized ABSITE scores, was modeled as a function of the corresponding postgraduate year via a linear mixed effects regression model. Additional model covariates included written USMLE-1-3 examination scores, gender, number of practice questions completed, and percentage correct of practice questions. For each resident, the predicted ABSITE standard score along with a 95% bootstrap prediction interval was obtained. Both resident-specific and population-level predictions for ABSITE standard scores were also estimated.

SETTING

The study was conducted at a single, large academic medical center (Massachusetts General Hospital, Boston, MA).

PARTICIPANTS

Six years of general surgery resident score reports at a single institution between 2014 and 2019 were deidentified and analyzed.

RESULTS

A total of 376 score reports from 130 residents were analyzed. Covariates that had a significant effect on the model included USMLE-1 score (PGY1: p = 0.013; PGY2: p = 0.007; PGY3: p = 0.011), USMLE-2 score (PGY1: p < 0.001; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p < 0.001; PGY5: p = 0.032), male gender (PGY1: p = 0.003; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p = 0.008), and number of practice questions completed (p=0.003). Five residents were identified as having "fallen off" their predicted knowledge curve, including a single resident on 2 occasions. Population prediction curves were obtained at 7 different covariate percentile levels (5%, 10%, 25%, 50%, 75%, 90%, and 95%) that could be used to plot predicted resident knowledge progress.

CONCLUSION

Performance on USMLE-1 and -2 examinations, male gender, and number of practice questions completed were positive predictors of ABSITE performance. Creating residency-wide knowledge growth curves as well as individualized predictive ABSITE performance models allows for more efficient identification of residents potentially at risk for poor ABSITE performance and structured monitoring of surgical knowledge progression.

Impact of Surgery Program Characteristics on Fate of Non-designated Preliminary Surgery Interns

OBJECTIVE

Non-designated preliminary (NDP) general surgery residents face the daunting challenge of obtaining a categorical residency position while undertaking the rigors of a general surgery residency. This additional application cycle represents a stressful time for these trainees and limited data exists to help guide applicants and program directors regarding the factors predictive of application success. While previous studies have focused solely on applicant related factors, no study to date has evaluated the effect of the residency program structure, institutional resources, or administrative support on these outcomes.

DESIGN/SETTING

A multicenter retrospective review of 10 general surgery residency programs over a 5-year period from 2014 to 2019 was performed. Applicant related information was compiled from NDP general surgery residents and the results of their attempted second application into a categorical position. Applicant factors including age, gender, standardized test scores (USMLE/ABSITE), and professional training were examined. Program and administrative structure including residency class size, number of NDP PGY-2 positions, number of assistant program directors and program director (PD) background were also examined. Primary success was defined as a NDP resident successfully obtaining a categorical position within general surgery or a surgical subspecialty. Secondary success was obtaining a categorical residency position in any field of medical practice other than surgery or a surgical subspecialty in the United States.

RESULTS

A total of 260 NDP trainees were evaluated with an average age of 29.1. Almost seventy percent of applicants were male, 40% graduated from a non-U.S. medical school and 24.2% required a visa to work in the United States. Thirty 4 percent of NDPs successfully obtained a categorical surgery position and an additional 35% obtained a categorical residency position in a nonsurgical field for an overall match success rate of 68.9%. Factors associated with primary success included ABSITE score (p < 0.001), US medical school graduation (p = 0.02), visa status (p = 0.03), presence of preliminary PGY-2 positions (p = 0.02), and PD professional development time (p = 0.004). Overall success was associated USMLE Step 1 scores (p = 0.02), number of approved chiefs (p = 0.03), presence of dedicated faculty researchers (p = 0.001), and PD professional development time (p < 0.001).

CONCLUSIONS

Applicant, program-related, and administrative factors all have a significant impact on the success of NDP general surgery residents in obtaining a categorical surgical position. Trainees should consider all of these factors when applying to NDP residencies and in approaching their second application cycle to maximize their likelihood of a successful match.

The Effects of Pass/Fail USMLE Step 1 Scoring on the Otolaryngology Residency Application Process

OBJECTIVES

To investigate how the decision to report United States Medical Licensing Examination (USMLE) Step 1 score as pass/fail will influence future otolaryngology residency application and match processes.

STUDY DESIGN

Survey study.

METHODS

An anonymous and voluntary survey approved by the Otolaryngology Program Directors Organization was administered to academic faculty members from April 24, 2020 through May 19, 2020.

RESULTS

Two hundred fifty-seven surveys were received from department chairs (17.5%), program directors (24.1%), associate program directors (12.5%), and department faculty (45.9%). USMLE Step 1 has been the most heavily weighted metric for offering interviews (44.0%), and it has correlated with residents' medical knowledge (77.0%) and in-service performance (79.8%) but not with surgical skills (57.6%) or patient care (47.1%). In total, 68.1% disagreed with the decision to make USMLE Step 1 pass/fail. This change is anticipated to lead to an increase in significance of USMLE Step 2 CK (89.1%), core clerkship grades (80.9%), elective rotation at the respective institutions (65.7%), Alpha Omega Alpha and other awards (64.6%), and letters of recommendation (63.8%). The new scoring is also anticipated to especially benefit students from top-ranked schools (70.8%), increase medical students' anxiety/uncertainty regarding obtaining interview invites (59.1%), and negatively affect international (51.4%), doctor of osteopathic medicine (45.9%), and underrepresented students (36.9%). Indication that USMLE Step 2 CK will significantly increase in weight varied according to department position (P = .049), geographic region (P = .047), years of practice (P < .001), and residency program size (P = .002).

CONCLUSION

Most academic otolaryngologists disagreed with changing USMLE Step 1 scoring to pass/fail and believe that it will increase other objective/subjective metrics' weight and put certain student populations at a disadvantage.

LEVEL OF EVIDENCE

N/A. Laryngoscope, 131:E738-E743, 2021.

Colon and Rectal Surgery Robotic Training Programs: An Evaluation of Gender Disparities

BACKGROUND

Female surgeons are subjected to implicit bias throughout their careers. The evaluation of gender bias in training is warranted with increasing numbers of female trainees in colon and rectal surgery.

OBJECTIVE

This study aimed to evaluate gender bias in colon and rectal surgery training program operative experience.

DESIGN

This is a retrospective cohort study.

SETTING

The Association of Program Directors for Colon and Rectal Surgery robotic case log database contains operative details (procedure, attending surgeon, case percentage, and operative segments) completed by trainees as console surgeon for 2 academic years (2016-2017, 2017-2018).

MAIN OUTCOME MEASURE

The primary outcomes measured are the percentage of trainee console participation and the completion of total mesorectal excision. Resident and attending surgeon gender was recorded retrospectively. The cohort was separated into 4 groups based on resident and attending surgeon gender combination. Case volume, average console participation per case, and completion of total mesorectal excisions were compared for each group by using interaction regression analysis.

RESULTS

Fifty-two training programs participated, including 120 trainees and 190 attending surgeons. Forty-five (37.5%) trainees and 36 (18.9%) attending surgeons were women. The average number of cases per trainee was 23.27 per year for women and 28.15 per year for men (p = 0.19). Average console participation was 53.5% for women and 61.7% for men (p < 0.001). Male attending surgeons provided female trainees less console participation than male counterparts (52.1% vs 59.7%, p < 0.001). Female attending surgeons provided the same amount of console participation to female and male trainees (63.3% vs 61.8%, p = 0.62). Male trainees performed significantly more complete total mesorectal excision console cases than female trainees (57.16% vs 42.38%, p < 0.0001).

LIMITATIONS

The data are subject to self-reporting bias.

CONCLUSIONS

There is gender disparity in robotic operative experience in colon and rectal surgery training programs with less opportunity for console participation and less opportunity to complete total mesorectal excisions for female trainees. This trend should be highlighted and further evaluated to resolve this disparity. See Video Abstract at http://links.lww.com/DCR/B224. PROGRAMAS DE CAPACITACIÓN ROBÓTICA SOBRE CIRUGÍA DE COLON Y RECTO: UNA EVALUACIÓN DE LAS DISPARIDADES DE GÉNERO: Cirujanos mujeres están sujetas a sesgos implícitos a lo largo de sus carreras. La evaluación del sesgo de género en el entrenamiento se amerita por un número cada vez mayor de aprendices femeniles en cirugía de colon y recto.Evaluar el sesgo de género en la experiencia operativa en programas de entrenamiento de cirugía de colon y recto.Estudio de cohorte retrospectivo.La base de datos de registro de casos robóticos de la Asociación de Directores de Programas para Cirugía de Colon y Rectal contiene detalles operativos (procedimiento, cirujano asistente, porcentaje de casos y segmentos operativos) completados por los alumnos como cirujanos de consola durante dos años académicos (2016-17, 2017-18).Porcentaje de participación de la consola de entrenamiento y finalización de la escisión mesorrectal total. Se registraron retrospectivamente el sexo de los médicos residentes y asistentes. La cohorte se separó en cuatro grupos según la combinación de género residente y asistente. El volumen de casos, la participación promedio de la consola por caso y la finalización de las extirpaciones mesorrectales totales se compararon para cada grupo mediante el análisis de regresión de interacción.Participaron 52 programas de capacitación, incluidos 120 aprendices y 190 cirujanos asistentes. Cuarenta y cinco (37.5%) aprendices y 36 (18.9%) cirujanos asistentes eran mujeres. El número promedio de casos por aprendiz fue de 23.27 / año para mujeres y 28.15 / año para hombres (p = 0.19). La participación promedio de la consola fue del 53.5% para las mujeres y del 61.7% para los hombres (p <0.001). Los cirujanos asistentes masculinos proporcionaron a las mujeres aprendices menos participación en la consola en comparación con sus compañeros masculinos (52.1% vs 59.7%, p <0.001). Los cirujanos asistentes femeninos proporcionaron la misma cantidad de participación en la consola a los aprendices femeninos y masculinos (63.3% vs 61.8%, p = 0.62). Los aprendices masculinos realizaron casos de consola TME significativamente más completos que las aprendices femeninas (57.16% vs 42.38%, p <0.0001).Los datos están sujetos a sesgos de autoinforme.Existe una disparidad de género en la experiencia quirúrgica robótica en los programas de entrenamiento de cirugía de colon y recto con menos oportunidades para la participación de la consola y menos oportunidades para completar las extirpaciones mesorrectales totales para las mujeres en formación. Esta tendencia debe destacarse y evaluarse para resolver esta disparidad. Consulte Video Resumen en http://links.lww.com/DCR/B224. (Traducción-Dr. Adrián Ortega).

Mid-Year Medical Knowledge Milestones and ABSITE Scores in First-Year Surgery Residents

OBJECTIVE

Accreditation Council for Graduate Medical Education (ACGME) Surgery milestone ratings in the "Knowledge of Diseases and Conditions" (MK1) sub competency have been shown to correlate with American Board of Surgery In Training Examination (ABSITE) scores, and hypothesized to predict them. To better assess the predictive value of the MK1 milestone and avoid the potential bias caused by previous years' ABSITE scores, we designed a study including only first-year (PGY-1) residents and analyzed the correlation between their mid-year MK1 ratings and their scores in the ABSITE they took approximately a month later.

METHODS

De-identified United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores, mid-year MK1 milestone ratings and the subsequent ABSITE standard scores for the five academic years from 2014-2015 to 2018-2019 were collected and tabulated for 247 PGY-1 preliminary- and categorical-track residents from ten ACGME-accredited surgery residency programs.

RESULTS

The mid-year rating of PGY-1 residents' MK1 was predictive of their subsequent first ABSITE score for the entire cohort and for the categorical residents' subset. Notably, controlling for all other independent predictors, each half-point increase in MK1 rating was associated with a 25-point increase in ABSITE score. Preliminary residents performed significantly worse on the ABSITE, and their scores did not correlate significantly with their MK1 ratings.

CONCLUSIONS

The mid-year rating of PGY-1 residents' MK1 was predictive of their subsequent first ABSITE score for the entire cohort and for the categorical but not the preliminary residents. This finding suggests that evaluators correctly rated MK1 higher in the categorical residents who did perform better on the subsequent ABSITE.

USMLE Scores and Clinical Rotation Role in Predicting ABSITE Performance Among Surgery Interns

BACKGROUND

The American Board of Surgery In-training Examination (ABSITE) is an important marker of medical knowledge. It is unclear what factors predict or improve these scores. We evaluated demographics, United States Medical Licensing Examination (USMLE) step 1 and 2 scores, and surgical rotations during the intern year to determine if there were any correlations with the ABSITE performance.

METHODS

This was a multicenter retrospective review during a 6-y period, investigating the correlation and association of demographics, USMLE scores, and types of rotations on the ABSITE percentile score of interns. Demographics included age, gender, race/ethnicity, U.S. versus international/foreign medical graduate for stratified analyses. Descriptive analysis was performed with ANOVA, correlation was evaluated with 95% confidence interval, and significance was defined as P < 0.05.

RESULTS

Complete records obtained on 89 interns from six different general surgery programs over 6 y revealed that there was a significant correlation between USMLE 1 and 2 with the ABSITE. USMLE 2 correlation was the strongest (r = 0.44, 95% confidence interval = [0.25-0.60], P < 0.05). There was a significant difference in ABSITE performance (mean score difference of 17.3 percentile, P = 0.01) of interns who had an intensive care unit rotation before examination. Other surgical rotations were not associated with an ABSITE difference. Demographic factors such as age, gender, race/ethnicity, or medical graduate background status were not associated with ABSITE scores.

CONCLUSIONS

USMLE step 2 scores had a higher correlation with intern ABSITE performance. An intensive care unit rotation before taking the ABSITE was associated with a significant difference in their percentile scores. Demographic factors were not correlated with ABSITE performance.