Practice Makes Perfect: Introducing a Question Bank for ABSITE Preparation Improves Program Performance

A Team-Based American Board of Surgery In-Training Examination (ABSITE) Competition Improves Exam Performance

OBJECTIVE

The American Board of Surgery In-Training Examination (ABSITE) is an important factor in general surgery resident career development. The use of practice question banks for ABSITE preparation improves exam scores and completing more practice questions results in higher scores. The objective of this study is to describe the design of a novel, team-based ABSITE competition implemented into our residency program and analyze its effect on exam performance.

DESIGN

For the 2023 to 2024 academic year (AY), residents were randomly sorted into teams. The number of TrueLearn (TL) practice questions completed by each team during the 6-month period preceding the 2024 ABSITE was monitored at regular intervals. Paired sample t-test was used to compare total questions completed and ABSITE percent scores from the prior AY. Simple linear regression was performed to determine if total completed questions predicted ABSITE percent scores, and if the change in total completed questions predicted a change in ABSITE percent scores for unique residents.

SETTING

University-affiliated general surgery residency program.

PARTICIPANTS

34 clinical and research general surgery residents.

RESULTS

Mean total TL questions completed per resident increased by 175.2, with a difference trending toward statistical significance (1037.9 ± 583.6 vs 1213.1 ± 596.0, p = 0.08). Mean ABSITE percent scores significantly increased by 4.6% (68.2 ± 8.7 vs 72.8 ± 5.7, p < 0.001). Total completed questions positively correlated with ABSITE percent scores (R2 = 0.21, F (1,66) = 17.04, p < 0.001). Among individual residents, completing more questions than the prior AY predicted improvement in ABSITE percent scores (R2 =.18, F (1,32) = 7.02, p = 0.01). Our model predicts that by completing 300 additional practice questions (e.g. 10 more questions/day for 1 month) a resident's ABSITE percent score will increase by 4.8 percentage points.

CONCLUSION

Implementation of a team-based ABSITE competition is a straightforward and effective intervention that increases general surgery resident question bank utilization and significantly improves ABSITE percent scores.

Examining the Most Impactful Strategies for In-service Preparation: A Systemic Review

INTRODUCTION

Resident performance on the Plastic Surgery In-Service Examination (PSITE) is used as a predictor of success on the American Board of Plastic Surgery Written Examination, as well as resident progression and fellowship applications. However, information specifically addressing strategies on optimal PSITE preparation is lacking in the plastic surgery literature when compared to general surgery. For this reason, we aim to understand if the topic is well-studied and denote effective study strategies and curricular interventions in both fields that can help residents and programs optimize PSITE performance.

METHODS

A literature search including studies from 2012 to 2022 was conducted following PRISMA guidelines in PubMed and EMBASE to identify articles on strategies to improve in-service exam scores for general surgery and plastic surgery. Only studies that reported measurable outcomes in raw score, percentile score, or percent correct were included.

RESULTS

Qualitative analysis of 30 articles revealed 2 categories of interventions: individual study habits and institutional curricular interventions. In general surgery literature, 27 articles examined interventions positively impacting resident ABSITE scores, with 21 studies classified as institutional curricular interventions and 6 articles addressing individual study habits. Themes associated with improved ABSITE performance included mandatory remediation programs, dedicated study time, and problem-based learning interventions. In contrast, only 3 articles in plastic surgery literature discussed interventions associated with improved PSITE scores, all falling under curricular interventions.

CONCLUSION

Unfortunately, the plastic surgery literature lacks concrete evidence on how residents can improve performance. Future research in plastic surgery should replicate successful strategies from general surgery and further investigate optimal preparation strategies for the PSITE. Such endeavors can contribute to improving resident performance and advancing plastic surgery education and patient care.

American Board of Surgery In-Training Exam Performance Predicted by Question Bank Use While Unassociated With Other Learning Strategies

INTRODUCTION

There is no consensus regarding optimal curricula to teach cognitive elements of general surgery. The American Board of Surgery In-Training Exam (ABSITE) aims to measure trainees' progress in attaining this knowledge. Resources like question banks (QBs), Surgical Council on Resident Education (SCORE) curriculum, and didactic conferences have mixed findings related to ABSITE performance and are often evaluated in isolation. This study characterized relationships between multiple learning methods and ABSITE performance to elucidate the relative educational value of learning strategies.

METHODS

Use and score of QB, SCORE use, didactic conference attendance, and ABSITE percentile score were collected at an academic general surgery residency program from 2017 to 2022. QB data were available in the years 2017-2018 and 2021-2022 during institutional subscription to the same platform. Given differences in risk of qualifying exam failure, groups of ≤30th and >30th percentile were analyzed. Linear quantile mixed regressions and generalized linear mixed models determined factors associated with ABSITE performance.

RESULTS

Linear quantile mixed regressions revealed a relationship between ABSITE performance and QB questions completed (1.5 percentile per 100 questions, P < 0.001) and QB score (1.2 percentile per 1% score, P < 0.001), but not with SCORE use and didactic attendance. Performers >30th percentile had a significantly higher QB score.

CONCLUSIONS

Use and score of QB had a significant relationship with ABSITE performance, while SCORE use and didactic attendance did not. Performers >30th percentile completed a median 1094 QB questions annually with a score of 65%. Results emphasize success of QB use as an active learning strategy, while passive learning methods warrant further evaluation.

Flipping the Classroom in Residency Didactics Leads to Higher American Board of Surgery in-Training Examination Scores

OBJECTIVE

Didactic education in General Surgery (GS) residency typically follows a nationally standardized curriculum; however, instructional format varies by institution. In recent years, GS residents at our institution expressed discontentment with weekly didactics and were not meeting their goals on the American Board of Surgery In-Training Examination (ABSITE). We sought to develop improvements in our didactic curriculum to increase resident satisfaction and ABSITE scores of GS junior residents (Jrs).

DESIGN

In a quality improvement project, we changed the weekly didactic curriculum format from hour-long lectures in the 2018 to 2019 academic year (AY) to a partially-flipped classroom in the 2019 to 2020 AY, involving a 30-minute faculty-led presentation followed by 30 minutes of resident-led practice questions. The outcomes measured were ABSITE scores taken in 2019 and 2020 and resident opinions via an anonymous survey.

SETTING

This study was conducted at the University of Minnesota (Minneapolis, MN).

PARTICIPANTS

The cohort for this study included all GS Jrs in our GS residency program, including postgraduate year (PGY) 1 nondesignated preliminary, PGY1 to 3 categorical GS residents, and residents in their lab time. Senior residents attended a separate didactics session.

RESULTS

After curriculum changes, the ABSITE percentile scores for GS Jrs rose from 52% ± 5% to 66% ± 4% (p = 0.03). No categorical GS Jr scored <30% in 2020, compared to 20% (6/30) of categorical General Surgery residents in 2019. All residents preferred the new format overall and reported greater engagement in and preparation for didactics.

CONCLUSIONS

After changing didactic education from hour-long lectures in the 2018 to 2019 AY to a flipped classroom model in the 2019 to 2020 AY including 30 minutes of faculty-led lecture followed by 30 minutes of resident-led practice questions, ABSITE scores and resident satisfaction at the University of Minnesota General Surgery Program improved.

National ABSITE Preparation Practices: A Survey of General Surgery Residency Program Directors

OBJECTIVE

The American Board of Surgery In-Training Examination (ABSITE) was designed as a low-stakes, medical knowledge examination for US general surgery residency programs. However, in practice, this exam has been utilized for higher stakes purposes, such as resident promotion or remediation, and fellowship selection. Several studies have demonstrated the efficacy of ABSITE preparation resources, but best practices for ABSITE preparation and national preparatory habits are currently unknown. The aim of this work was to determine current residency programs' strategies for ABSITE preparation.

DESIGN

We distributed an electronic survey to program directors or program coordinators of US general surgery programs asking them to anonymously report program ABSITE educational practices and ABSITE scores. We analyzed the proportion of responses using descriptive statistics and compared the effect of various strategies using the Mann-Whitney testing for nonparametric data. An average ABSITE percentile score was calculated for each residency based on program self-reported scores.

SETTING

Association of Program Directors (APDS) Listserv PARTICIPANTS: General surgery residency programs participating at the time of distribution (278).

RESULTS

Response rate was 24% (66/278); 41 programs (62.1%) identified as university-affiliated, and 25 (37.9%) were community-based. Median intern class size was 8 (range: 3-14), including preliminary interns. Average ABSITE percentile score was 52.8% (range 36.9%-67.6%). There were no significant differences in ABSITE scores based on affiliation or program size. Educational resources utilized for ABSITE preparation included SCORE (89.3%), Q-banks (50%), and surgical textbooks (25.8%). The majority (56.1%) of programs reported using a year-long curriculum for ABSITE preparation, and 66.6% used a time-limited curriculum completed in the months immediately prior to ABSITE. Most programs reported that ABSITE scores were a low priority (63.6%) or not a priority (13.6%). The existence of a year-long curriculum for ABSITE was positively correlated with score as compared to programs without a year-long curricula (53.9% vs 48.5%, p <0.01). Programs using a time-limited curriculum demonstrated lower scores as compared to programs without time-limited curricula (51.3% v 56.1%, p < 0.01).

CONCLUSION

General surgery programs use a variety of strategies to prepare residents for the ABSITE. Despite reporting that they utilize ABSITE scores for a variety of high stakes purposes including evaluation for promotion and as a predictor of the preparedness for the ABS QE, many programs reported that they consider ABSITE scores as a low priority. A year-long focused curriculum was the only strategy correlated with increased scores, which may reflect the value of encouraging consistent studying and spaced repetition. Additional work is needed to guide programs in optimal utilization of ABSITE scores for remediation and resident evaluation, as well as understanding how ABSITE preparatory strategies correlate with clinical performance.

Novel interactive text-messaging curriculum for endocrinology board review

Background

The ABIM certification exam is one of the measures to ensure that physicians have the clinical skills for good care delivery. The 5-year average pass rate for ABIM Endocrinology exam is 82%. The pass rate significantly decreased to a nadir of 74% in 2021 and 2022, lowest of all medicine subspecialties.

Objectives

To assess the feasibility of text messaging curriculum for fellows and its utility in improving their test performance.

Methods

In 2021, endocrinology fellows from 51 programs across the country were invited to participate in our curriculum. They completed a pre-test, joined a texting group via Remind application and received 1 multiple choice question daily (total n = 78). After 15 weeks, they completed a post-test and survey. Paired results from pre- and post-test were compared.

Results

A total of 89 fellows from 27 programs responded. Of these, 82 fellows, predominantly females (n = 60; 73 %), filled out the pre-test. On an average, 42 fellows (SD = 12) responded to the questions daily and 57 % of them answered the questions within 24 h. Thirty fellows completed the post-test. The median number of correct responses on the pre-test was 5 (IQR 3-6), compared to 8 (IQR 6-9) in the post-test. There was a significant improvement (p-value < 0.0001) in fellows' performance in the post-test when compared with the pre-test following our intervention.

Conclusions

Text-messaging based curriculum for exam preparation is feasible and can improve test performance. Fellows find receiving a daily high yield multiple choice question via text-message as a useful tool for exam preparation.

We Tabulated and Organized American Board of Neurological Surgeons Primary Exam Keywords (2015-2023) so You Don't Have to

Background Passing the American Board of Neurological Surgeons (ABNS) Primary Exam is required for residents in training. Both the program directors and residents are given keywords of the exam afterward in the hope to help program directors determine their relative strengths and weakness. We have organized and tabulated these keywords for neurosurgery residents' benefit. Methodology We collected and analyzed ABNS Primary Exam keywords (2015-2023) in each of the exam's main categories for trends and recurrences. We examined the overall passing rates among first-time credit test takers. The frequency of each subcategory was calculated as a percentage within its corresponding category. Recurrent keywords were grouped together with their corresponding years and categorized as once, twice, or thrice and greater occurrences; the last category was considered to be high-yield keywords. Results The number of questions in Neurosciences and Neurology has decreased over the years while Neurosurgery and Critical Care questions have increased. Similarly, there are fewer keyword repeats in Neurosciences and Neurology. The most repeated keywords are in Neuroimaging. The most common keywords are presented and listed along with the years of occurrences. Overall, the passing rate among first-time credit test takers is over 90%. Conclusions Neurosurgery residents can consider the common keywords as a guide in preparation for the ABNS Primary Exam.

Impact of an Online Question Bank on Resident In-Training Exam Performance

OBJECTIVE

In-training exams (ITEs) are administered annually to Obstetrics and Gynecology (OBGYN) residents and have been demonstrated to correlate with success on licensing examinations. Our study objective was to determine the impact of a question bank and mock exam on the performance of Council on Resident Education in Obstetrics and Gynecology (CREOG) ITEs. Secondarily, we investigated the correlation between the extent of question bank usage and performance on the exam.

METHODS

Pre-post intervention study of resident performance on CREOG ITE before and after implementation of the question bank and mock ITE at Indiana University in 2018. Performance was measured as year-to-year improvement in percent correct on ITE exams. Scores were excluded if a resident did not have a prequestion bank score report or if they did not sit for all eligible ITE exams.

RESULTS

There were 51 OBGYN residents at Indiana University during the study period, with 38 available for analysis (75%). Before implementation, average year-to-year improvement for PGY1-2, PGY2-3 and PGY3-4 classes were 0.60%, 1.0% and -1.6%, respectively. After implementation, all resident classes had significant improvements in ITE scores of 6.6% (P < .01), 9.0% (P < .01), and 7.2% (P < .01), respectively. There was a moderate program-wide correlation between the number of questions completed and the percent improvement on the ITE of R = 0.36 (P = .046).

CONCLUSIONS

Our study demonstrated that access to a question bank and mock ITE significantly improved CREOG ITE performance of OBGYN residents at Indiana University.

Impact of Question Bank Use for In-Training Examination Preparation by OBGYN Residents - A Multicenter Study

OBJECTIVE

To examine the impact of access to and utilization of a commercially available question bank (TrueLearn) for in-training examination (ITE) preparation in Obstetrics and Gynecology (OBGYN).

DESIGN

This was a retrospective cohort study examining the impact of TrueLearn usage on ITE examination performance outcomes. Produced by the educational arm of the American College of Obstetricians and Gynecologists, the Council on Resident Education in Obstetrics and Gynecology (CREOG) exam is a multiple-choice test given to all residents annually. Residency programs participating in this study provided residency program mean CREOG scores from the year prior (2015), and the first (2016) and second (2017) years of TrueLearn usage. Programs also contributed resident-specific CREOG scores for each resident for 2016 and 2017. This data was combined with each resident's TrueLearn usage data that was provided by TrueLearn with residency program consent. The CREOG scores consisted of the CREOG score standardized to all program years, the CREOG score standardized to the same program year (PGY) and the total percent (%) correct. TrueLearn usage data included number of practice questions completed, number of practice tests taken, average number of days between successive tests, and percent correct of answered practice questions.

SETTING

OBGYN Residency Training Programs.

PARTICIPANTS

OBGYN residency programs that purchased and utilized TrueLearn for the 2016 CREOG examination were eligible for participation (n = 14). Ten residency programs participated, which consisted of 212 residents in 2016 and 218 residents in 2017.

RESULTS

TrueLearn was used by 78.8% (167/212) of the residents in 2016 and 84.9% (185/218) of the residents in 2017. No significant difference was seen in the average CREOG scores available on a per- program level before versus after the first year of implementation either using the CREOG score standardized to all PGYs (mean difference 1.0; p = 0.58) or standardized to the same PGY (mean difference 3.1; p = 0.25). Using resident-level data, there was no significant difference in mean CREOG score standardized to all PGYs between users and non-users of TrueLearn in 2016 (mean, 199.4 vs 196.7; p = 0.41) or 2017 (mean, 198.2 vs 203.4; p = 0.19). The percent of practice questions answered correctly on TrueLearn was positively correlated with the CREOG score standardized to all PGYs (r = 0.47 for 2016 and r = 0.60 for 2017), as well as with the CREOG total percent correct (r = 0.47 for 2016 and r = 0.61 for 2017). Based on a simple linear regression, for every 500 practice questions completed, the CREOG score significantly increased for PGY-2 residents by an average (±SE) of 7.3 ± 2.8 points (p = 0.013); the average increase was 0.7 ± 2.5 (p = 0.79) for PGY-3 residents and 5.8 ± 3.3 points (p = 0.09) for PGY-4 residents.

CONCLUSIONS

Adoption of an online question bank did not result in higher mean CREOG scores at participating institutions. However, performance on the TrueLearn questions correlated with ITE performance, supporting predictive validity and the use of this question bank as a formative assessment for resident education and exam preparation.

How to Prepare for the American Board of Surgery In-Training Examination (ABSITE): A Systematic Review

INTRODUCTION

Performance on ABSITE is an important factor when monitoring resident progress. It predicts future performance and has lasting effects. Understanding the highest-yield preparation strategies can help residents in their study efforts and optimize performance.

METHODS

A literature search was conducted searching PubMed, EMBASE and JAMA Network in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Searches were performed for the terms "ABSITE" and "American Board of Surgery In-Training Examination". Only studies discussing individual study habits from May 2011 to May 2021 were included.

RESULTS

19 studies were included in qualitative synthesis. Year-round clinical study failed to show significant correlation to ABSITE performance although year-round ABSITE review was more consistently correlated. During a dedicated study period, increased time and increased total practice questions completed are associated with improved performance. The correlation of individual resources such as ABSITE review books, textbooks, audio podcasts and ABSITE preparatory courses to improved ABSITE performance was not proven.

CONCLUSIONS

Residents should optimize study strategies based on methods that have consistently shown to improve performance. Recommendations for best preparation strategies are provided.

Formative Feedback with In-Class Question Bank Utilization Improves Resident Satisfaction with General Surgery Didactics

BACKGROUND

Formative feedback provides low-stakes opportunities for educational improvement. To enrich our basic science didactics, formative feedback measures were incorporated into our didactics using mobile devices.

MATERIALS AND METHODS

Lecture changes included institutional paid access to a commercial question bank, a 5-item in-class pre-didactic quiz curated from the question bank and taken on the resident's mobile device, and group discussion of quiz topics. An anonymous survey was sent to participating residents.

RESULTS

Overall response rate was 71% among residents. All reported that the new lecture format was a valuable addition to the basic science curriculum (100% Agree/Strongly Agree), and formative assessments provided valuable feedback about the progress of their learning (Strongly Agree = 42%, Agree =58%). All residents reported that in-class use of their mobile device for quizzes was convenient, with majority (84%) preferring it over paper printouts. Residents were more motivated to study before lecture (Strongly Agree = 42%, Agree =42%), with majority also reporting the new format helped identify weaknesses in their knowledgebase (Strongly Agree = 58%, Agree =33%). While majority of residents agreed that quizzes motivated them to study more after lecture, a large portion disagreed (42%). Majority of senior residents reported that the process of composing quizzes prior to lecture enriched their own learning (57%) and helped them find gaps in their knowledge (71%).

CONCLUSION

Incorporating a commercial question bank within didactics gives general surgery residents formative feedback and encourages learning outside the classroom, leading to improved satisfaction with basic science didactics.

SCORE-Based Simulated ABSITE Exam Performance as a Predictor of Performance on the ABSITE

INTRODUCTION

The American Board of Surgery In-Training Examination (ABSITE) is a crucial, objective assessment of surgical knowledge during training. In 2014, the American Board of Surgery (ABS) announced the alignment of the ABSITE to the SCORE® (Surgical Council on Resident Education) Curriculum Outline for General Surgery Residency. We hypothesized that implementing a pre-ABSITE SCORE-based exam would help identify underperforming residents and provide early guidance to improve performance on the ABSITE.

METHODS

In October 2014, our university-based surgical residency program began administering a yearly comprehensive pre-ABSITE SCORE-based exam consisting of 225 to 250 multiple-choice questions selected from the SCORE question bank to all our general surgery residents, preliminary and categorical. The 4-hour exam addresses both clinical management (80%) and applied sciences (20%). Residents receive reports with their scores (percentage correct). Residents performing at less than 60% meet with the Program Director for discussion and formulation of a study plan. Correlational analysis was performed between resident ABSITE scores, pre-ABSITE SCORE-based exam scores, gender, resident status (preliminary vs. categorical), postgraduate year (PGY), and the United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores.

RESULTS

A total of 244 exam scores (122 pre-ABSITE SCORE-based exams and 122 matched ABSITE) were completed by 51 residents at different PGY levels (32 PGY1, 32 PGY2, 20 PGY3, 19 PGY4, and 18 PGY5). Fifty-seven percent were males, 62% were categorical residents, and 38% were preliminary residents. October pre-ABSITE SCORE-based exam scores were compared to the subsequent January ABSITE scores. Categorical residents completed 101 (83%) of the January exams, while preliminary residents completed 21 (17%) of these paired exams. We found strong correlations between the correct percentage on ABSITE and pre-ABSITE SCORE-based scores (r = 0.637, p < 0.001), between the correct percentage on ABSITE and PGY (r = 0.688, p < 0.001), and between ABSITE and resident status (r = 0.462, p < 0.001). Additionally, there was a weak to negligible correlation between the correct percentage on ABSITE and resident gender (r = 0.274, p = 0.001), USMLE-2 (r = 0.12, p = 0.16), and USMLE-1 (r = 0.04, p = 0.653). Multiple regression analysis, with all predictors, was performed to predict the percentage score on the ABSITE and produced R² 0.58, with an adjusted R² of 0.57, with a large size effect, p < 0.001. After controlling for the other variables, three factors reached statistical significance (p < 0.05): pre-ABSITE SCORE-based exam scores, PGY, and resident gender.

CONCLUSION

We found a strong correlation between performance on the pre-ABSITE SCORE-based exam and performance on the ABSITE exam. Surgery residents are encouraged to start studying earlier and to utilize SCORE contents as outlined by the ABS in their study plan.

Early Detection and Remediation of Problem Learners

There are myriad types of problem learners in surgical residency and most have difficulty in more than 1 competency. Programs that use a standard curriculum of study and assessment are most successful in identifying struggling learners early. Many problem learners lack appropriate systems for study; a multidisciplinary educational team that is separate from the team that evaluates the success of remediation is critical. Struggling residents who require formal remediation benefit from performance improvement plans that clearly outline the issues of concern, describe the steps required for remediation, define success of remediation, and outline consequences for failure to remediate appropriately.

Spaced Education With ABSITE Quest Resulting in Improved American Board of Surgery In-Training Examination Performance

OBJECTIVE

The American Board of Surgery In-Training Examination (ABSITE) is an annual exam taken by general surgery residents as a cognitive assessment of the knowledge gained throughout each year of training. Several question banks are available for ABSITE preparation. However, ABSITE Quest (AQ) utilizes a method called spaced education which has been demonstrated to help with retaining information longer and improve exam performance. This study hypothesizes that using this method will help residents improve their ABSITE performance.

DESIGN

Retrospective survey data was collected from residents who participated in AQ, including postgraduate year (PGY) level, as well as 2019 and 2020 ABSITE percentiles. AQ user data was used to match respondent's total number of questions completed and daily engagement level to the survey data. Paired, single-tailed student's t test was used to evaluate the significance of ABSITE percentile change between 2019 and 2020 among AQ users.

SETTING

ChristianaCare, Newark, DE, United States. Nonclinical.

PARTICIPANTS

All ABSITE Quest users were surveyed (n = 390), of which 104 responded. 21 responses were from PGY1 residents and were excluded, resulting in a total of 83 responses.

RESULTS

The mean percentile difference of AQ users from 2019 to 2020 was +15.8 (p < 0.00001). When categorizing by the total number of questions completed, high users demonstrated a mean percentile difference of +15.3 (p = 0.00002), average users had a difference of +19.1 (p = 0.00029), and low users showed a percentile difference of +1.2 (p = 0.45244). When categorizing by daily engagement level, high users demonstrated a mean percentile difference of +17.9 (p < 0.00001), low users had a mean percentile difference of +15.3 (p = 0.00124), and minimal users showed a mean percentile change of -5.7.

CONCLUSIONS

The use of the spaced education method with ABSITE Quest, especially in users with a greater number of questions completed and high levels of daily engagement, correlated with a significant improvement on ABSITE performance.

The Surgical Knowledge "Growth Curve": Predicting ABSITE Scores and Identifying "At-Risk" Residents

OBJECTIVE

Resident performance on the American Board of Surgery In-Training Examination (ABSITE) is used for evaluation of surgical knowledge and guides resident selection for institutional remediation programs. Remediation thresholds have historically been based on ABSITE percentile scores; however, this does not account for predictors that can impact a resident's exam performance. We sought to identify predictors of yearly ABSITE performance to help identify residents "at-risk" for performing below their expected growth trajectory.

DESIGN

The knowledge of the residents, as measured by standardized ABSITE scores, was modeled as a function of the corresponding postgraduate year via a linear mixed effects regression model. Additional model covariates included written USMLE-1-3 examination scores, gender, number of practice questions completed, and percentage correct of practice questions. For each resident, the predicted ABSITE standard score along with a 95% bootstrap prediction interval was obtained. Both resident-specific and population-level predictions for ABSITE standard scores were also estimated.

SETTING

The study was conducted at a single, large academic medical center (Massachusetts General Hospital, Boston, MA).

PARTICIPANTS

Six years of general surgery resident score reports at a single institution between 2014 and 2019 were deidentified and analyzed.

RESULTS

A total of 376 score reports from 130 residents were analyzed. Covariates that had a significant effect on the model included USMLE-1 score (PGY1: p = 0.013; PGY2: p = 0.007; PGY3: p = 0.011), USMLE-2 score (PGY1: p < 0.001; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p < 0.001; PGY5: p = 0.032), male gender (PGY1: p = 0.003; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p = 0.008), and number of practice questions completed (p=0.003). Five residents were identified as having "fallen off" their predicted knowledge curve, including a single resident on 2 occasions. Population prediction curves were obtained at 7 different covariate percentile levels (5%, 10%, 25%, 50%, 75%, 90%, and 95%) that could be used to plot predicted resident knowledge progress.

CONCLUSION

Performance on USMLE-1 and -2 examinations, male gender, and number of practice questions completed were positive predictors of ABSITE performance. Creating residency-wide knowledge growth curves as well as individualized predictive ABSITE performance models allows for more efficient identification of residents potentially at risk for poor ABSITE performance and structured monitoring of surgical knowledge progression.

Learning to Read: Successful Program-Based Remediation Using the Surgical Council on Resident Education (SCORE) Curriculum

BACKGROUND

The Surgical Council on Resident Education (SCORE) curriculum is aligned with the American Board of Surgery (ABS) objectives. Our program adopted the SCORE curriculum in 2015 after poor ABS In-Training Examination (ABSITE) performance and lowest quartile ABS Certifying Exam (CE) and Qualifying Exam (QE) first-time pass rates. We examined the association of SCORE use with ABSITE performance and ABS board exam first-time pass rate.

STUDY DESIGN

At a single institution, a retrospective review of surgery residents' SCORE metrics and ABSITE percentile was conducted for academic years 2015 to 2019. Metrics analyzed on the SCORE web portal were mean total minutes and total visits per resident for all residents using SCORE that year. First-time pass rates of the ABS QE and CE were examined from 2013 to 2019. Chi-square and linear regression analysis were performed, and a 95% level of confidence was assumed (alpha = 0.05).

RESULTS

Yearly data from categorical general surgery residents showed a significant increase in total minutes, total visits, and ABSITE percentile. Combined first time pass rates for the ABS QE and CE significant increased from 70.8% in 2013 to 2015 to 93.9% in 2016 to 2019 (p = 0.018).

CONCLUSIONS

Increased longitudinal use of the SCORE curriculum was associated with programmatic improvements in ABSITE performance and ABS board exam first-time pass rate.

Exploring Pediatric Resident Attitudes and Preferences for Board Exam Preparation

Objective The American Board of Pediatrics Certifying Exam (ABP CE) is a high stakes exam and is important for employment and fellowship opportunities in pediatrics. Although research has suggested interventions which may improve scores, little research has focused on resident perception of preparation for the ABP CE. In this study, we aimed to better define pediatric residents' attitudes and preferences regarding preparation for the ABP CE. Methods Pediatric residents from one residency program were invited to participate in focus groups to discuss their attitudes and preferences on board exam preparation for the ABP CE. Focus groups were audio recorded and transcribed. Conventional content analysis was used to analyze qualitative data. From the transcripts, authors developed codes through an iterative process, which were then organized into categories. These categories were then grouped into themes. Results Nineteen residents participated in three focus groups. Focus group transcription analysis resulted in 49 codes, which were sorted into 26 categories. The categories were then grouped into four key themes: 1) the ABP CE is not immediately important early in residency; 2) more personalized guidance is preferred; 3) consistent board preparation focus from the residency program is valued; 4) learning format is important. Conclusions Residents believe preparation for the ABP CE increases in importance as they progress through residency, and they desire more personalized, consistent, and structured focus from their training program related to ABP CE preparation. Attention to these perceptions can help guide pediatric residency program leadership in developing effective board exam preparation strategies and curricula for their residents.