1
|
Bajestani N, Singh A, Gerald M, Ueno C. Examining the Most Impactful Strategies for In-service Preparation: A Systemic Review. JOURNAL OF SURGICAL EDUCATION 2024; 81:1203-1214. [PMID: 38971679 DOI: 10.1016/j.jsurg.2024.06.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/01/2024] [Revised: 04/25/2024] [Accepted: 06/02/2024] [Indexed: 07/08/2024]
Abstract
INTRODUCTION Resident performance on the Plastic Surgery In-Service Examination (PSITE) is used as a predictor of success on the American Board of Plastic Surgery Written Examination, as well as resident progression and fellowship applications. However, information specifically addressing strategies on optimal PSITE preparation is lacking in the plastic surgery literature when compared to general surgery. For this reason, we aim to understand if the topic is well-studied and denote effective study strategies and curricular interventions in both fields that can help residents and programs optimize PSITE performance. METHODS A literature search including studies from 2012 to 2022 was conducted following PRISMA guidelines in PubMed and EMBASE to identify articles on strategies to improve in-service exam scores for general surgery and plastic surgery. Only studies that reported measurable outcomes in raw score, percentile score, or percent correct were included. RESULTS Qualitative analysis of 30 articles revealed 2 categories of interventions: individual study habits and institutional curricular interventions. In general surgery literature, 27 articles examined interventions positively impacting resident ABSITE scores, with 21 studies classified as institutional curricular interventions and 6 articles addressing individual study habits. Themes associated with improved ABSITE performance included mandatory remediation programs, dedicated study time, and problem-based learning interventions. In contrast, only 3 articles in plastic surgery literature discussed interventions associated with improved PSITE scores, all falling under curricular interventions. CONCLUSION Unfortunately, the plastic surgery literature lacks concrete evidence on how residents can improve performance. Future research in plastic surgery should replicate successful strategies from general surgery and further investigate optimal preparation strategies for the PSITE. Such endeavors can contribute to improving resident performance and advancing plastic surgery education and patient care.
Collapse
Affiliation(s)
- Nojan Bajestani
- Ohio State College of Medicine - 1645 Neil Ave, Columbus, OH 43210.
| | - Anusha Singh
- Ohio State College of Medicine - 1645 Neil Ave, Columbus, OH 43210
| | - Mykal Gerald
- Ohio State Department of Plastic Surgery - 915 Olentangy River Rd 2nd Floor Suite 2140
| | - Cristiane Ueno
- Ohio State College of Medicine - 1645 Neil Ave, Columbus, OH 43210; Ohio State Department of Plastic Surgery - 915 Olentangy River Rd 2nd Floor Suite 2140
| |
Collapse
|
2
|
Montreuil J, Lacasse M, Audétat MC, Boileau É, Laferrière MC, Lafleur A, Lee S, Nendaz M, Steinert Y. Interventions for undergraduate and postgraduate medical learners with academic difficulties: A BEME systematic review update: BEME Guide No. 85. MEDICAL TEACHER 2024:1-19. [PMID: 38589011 DOI: 10.1080/0142159x.2024.2331041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Accepted: 03/12/2024] [Indexed: 04/10/2024]
Abstract
BACKGROUND Clinical teachers often struggle to record trainee underperformance due to lacking evidence-based remediation options. OBJECTIVES To provide updated evidence-based recommendations for addressing academic difficulties among undergraduate and postgraduate medical learners. METHODS A systematic review searched databases including MEDLINE, CINAHL, EMBASE, ERIC, Education Source, and PsycINFO (2016-2021), replicating the original Best Evidence Medical Education 56 review strategy. Original research/innovation reports describing intervention(s) for medical learners with academic difficulties were included. Data extraction used Michie's Behaviour Change Techniques (BCT) Taxonomy and program evaluation models from Stufflebeam and Kirkpatrick. Quality appraised used the Mixed Methods Appraisal Tool (MMAT). Authors synthesized extracted evidence by adapting GRADE approach to formulate recommendations. RESULTS Eighteen articles met the inclusion criteria, primarily addressing knowledge (66.7%), skills (66.7%), attitudinal problems (50%) and learner's personal challenges (27.8%). Feedback and monitoring was the most frequently employed BCT. Study quality varied (MMAT 0-100%). We identified nineteen interventions (UG: n = 9, PG: n = 12), introducing twelve new thematic content. Newly thematic content addressed contemporary learning challenges such as academic procrastination, and use of technology-enhanced learning resources. Combined with previous interventions, the review offers a total dataset of 121 interventions. CONCLUSION This review offers additional evidence-based interventions for learners with academic difficulties, supporting teaching, learning, faculty development, and research efforts.
Collapse
Affiliation(s)
- Julie Montreuil
- Department of Family Medicine and Emergency Medicine, Université Laval, Quebec, Canada
| | - Miriam Lacasse
- Department of Family Medicine and Emergency Medicine, Université Laval, Quebec, Canada
| | - Marie-Claude Audétat
- University Institute for primary care (IuMFE), University of Geneva, Geneva, Switzerland
- Unit of Development and Research (UDREM), University of Geneva, Geneva, Switzerland
| | - Élisabeth Boileau
- Department of Family and Emergency Medicine, Université de Sherbrooke, Sherbrooke, Canada
| | | | | | - Shirley Lee
- Department of Emergency Medicine, University of Ottawa, Ottawa, Canada
| | - Mathieu Nendaz
- Unit of Development and Research (UDREM), University of Geneva, Geneva, Switzerland
- Department of Medicine, University Hospitals, Geneva, Switzerland
| | - Yvonne Steinert
- Institute of Health Sciences Education, Faculty of Medicine, McGill University, Montreal, Canada
| |
Collapse
|
3
|
Fair L, Gough B, Hyman N, Bello B, Steinhagen R, Cleary R, Ziegler M, Maun D, Fleshner P, Ogola G, Wells K, Lichliter W, Fleshman J, Fichera A. Predictive factors of first-time failure on the American Board of Colorectal Surgery certifying and qualifying examinations. Proc AMIA Symp 2023; 36:483-489. [PMID: 37334084 PMCID: PMC10269382 DOI: 10.1080/08998280.2023.2204776] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 04/06/2023] [Accepted: 04/10/2023] [Indexed: 06/20/2023] Open
Abstract
Objective To discover if first-attempt failure of the American Board of Colon and Rectal Surgery (ABCRS) board examination is associated with surgical training or personal demographic characteristics. Methods Current colon and rectal surgery program directors in the United States were contacted via email. Deidentified records of trainees from 2011 to 2019 were requested. Analysis was performed to identify associations between individual risk factors and failure on the ABCRS board examination on the first attempt. Results Seven programs contributed data, totaling 67 trainees. The overall first-time pass rate was 88% (n = 59). Several variables demonstrated potential for association, including Colon and Rectal Surgery In-Training Examination (CARSITE) percentile (74.5 vs 68.0, P = 0.09), number of major cases in colorectal residency (245.0 vs 219.2, P = 0.16), >5 publications during colorectal residency (75.0% vs 25.0%, P = 0.19), and first-time passage of the American Board of Surgery certifying examination (92.5% vs 7.5%, P = 0.18). Conclusion The ABCRS board examination is a high-stakes test, and training program factors may be predictive of failure. Although several factors showed potential for association, none reached statistical significance. Our hope is that by increasing our data set, we will identify statistically significant associations that can potentially benefit future trainees in colon and rectal surgery.
Collapse
Affiliation(s)
- Lucas Fair
- Baylor University Medical Center at Dallas, Dallas, Texas
- Baylor Scott and White Research Institute, Dallas, Texas
| | - Benjamin Gough
- Baylor University Medical Center at Dallas, Dallas, Texas
| | - Neil Hyman
- University of Chicago Pritzker School of Medicine, Chicago, Illinois
| | - Brian Bello
- MedStar Washington Hospital Center, Washington, District of Columbia
| | | | | | | | - Dipen Maun
- Franciscan Health Inc, Mishawaka, Indiana
| | | | - Gerald Ogola
- Baylor Scott and White Research Institute, Dallas, Texas
| | - Katerina Wells
- Baylor University Medical Center at Dallas, Dallas, Texas
| | | | - James Fleshman
- Baylor University Medical Center at Dallas, Dallas, Texas
| | | |
Collapse
|
4
|
Tarras S, White MT, Toloff K, Cooley D, Edelman D. Just Do It: Participation in Structured Online Curricula Reliably Improves Low ABSITE Scores. JOURNAL OF SURGICAL EDUCATION 2022; 79:e166-e172. [PMID: 35902350 DOI: 10.1016/j.jsurg.2022.07.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Revised: 06/03/2022] [Accepted: 07/05/2022] [Indexed: 06/15/2023]
Abstract
OBJECTIVE We hypothesized residents enrolled in an Accelerated Clinical Education in Surgery (ACES) program would improve their scores to above the 30th percentile. We analyzed which components of ACES correlated with improvement. DESIGN AND SETTING A retrospective review of three academic cycles (2018-2021) at an academic general surgery residency. PARTICIPANTS Residents scoring ≤30th percentile on the ABSITE were enrolled in ACES. Baseline demographics including STEP scores were collected. ACES included: (1) SCORE and DeckerMed assignments (2) Weekly faculty review sessions and (3) Monthly meeting with assigned mentor. Data were analyzed by Student's t-test, one-way ANOVA and Fisher's exact test. RESULTS Twenty-six surgical residents enrolled in ACES. Compared to residents not in ACES, there was no significant difference females (15 vs. 15; p = 0.19) and STEP 2 scores (241 vs. 246; p = 0.06). Residents in ACES had significantly lower STEP 1 (225 vs. 237; p < 0.001) and STEP 3 (212 vs. 223; p < 0.001) scores. Demographics of ACES residents who subsequently scored >30th percentile were similar to those who didn't, except for STEP 3 scores (216 vs. 204; p = 0.008). For residents in ACES, the completion of assignments between July and January was significantly higher for those who subsequently achieved an ABSITE score >30th percentile: TWIS, 77% vs. 53% (p = 0.022), Decker WC, 80% vs. 49% (p = 0.009) and Decker MR, 53% vs. 29% (p = 0.016). Completion of an online practice exam prior to ABSITE also correlated with score >30th percentile (57% vs. 13%, p = 0.007). There was also no correlation between the number of faculty review sessions and ABSITE (11.5 vs.11.9, p = 0.931). CONCLUSIONS Participation in a structured online program of reading and quizzes was durably effective in improving ABSITE scores >30th percentile. Completion of online assignments, rather than scores on practice tests or review sessions, appeared to be the most important factor for success.
Collapse
Affiliation(s)
- Samantha Tarras
- The Michael and Marian Ilitch Department of Surgery, Wayne State University, Detroit, Michigan.
| | - Michael T White
- The Michael and Marian Ilitch Department of Surgery, Wayne State University, Detroit, Michigan
| | - Katelyn Toloff
- The Michael and Marian Ilitch Department of Surgery, Wayne State University, Detroit, Michigan
| | - Dana Cooley
- The Michael and Marian Ilitch Department of Surgery, Wayne State University, Detroit, Michigan
| | - David Edelman
- The Michael and Marian Ilitch Department of Surgery, Wayne State University, Detroit, Michigan
| |
Collapse
|
5
|
Mills LM, Boscardin C, Joyce EA, Ten Cate O, O'Sullivan PS. Emotion in remediation: A scoping review of the medical education literature. MEDICAL EDUCATION 2021; 55:1350-1362. [PMID: 34355413 DOI: 10.1111/medu.14605] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2021] [Revised: 07/28/2021] [Accepted: 07/31/2021] [Indexed: 05/15/2023]
Abstract
OBJECTIVES Remediation can be crucial and high stakes for medical learners, and experts agree it is often not optimally conducted. Research from other fields indicates that explicit incorporation of emotion improves education because of emotion's documented impacts on learning. Because this could present an important opportunity for improving remediation, we aimed to investigate how the literature on remediation interventions in medical education discusses emotion. METHODS The authors used Arksey and O'Malley's framework to conduct a scoping literature review of records describing remediation interventions in medical education, using PubMed, CINAHL Complete, ERIC, Web of Science and APA PsycInfo databases, including all English-language publications through 1 May 2020 meeting search criteria. They included publications discussing remediation interventions either empirically or theoretically, pertaining to physicians or physician trainees of any level. Two independent reviewers used a standardised data extraction form to report descriptive information; they reviewed included records for the presence of mentions of emotion, described the mentions and analysed results thematically. RESULTS Of 1644 records, 199 met inclusion criteria and were reviewed in full. Of those, 112 (56%) mentioned emotion in some way; others focused solely on cognitive aspects of remediation. The mentions of emotion fell into three themes based on when the emotion was cited as present: during regular coursework or practice, upon referral for remediation and during remediation. One-quarter of records (50) indicated potential intentional incorporation of emotion into remediation programme design, but they were non-specific as to how emotions related to the learning process itself. CONCLUSION Even though emotion is omnipresent in remediation, medical educators frequently do not factor emotion into the design of remediation approaches and rarely explicitly utilise emotion to improve the learning process. Applications from other fields may help medical educators leverage emotion to improve learning in remediation, including strategies to frame and design remediation.
Collapse
Affiliation(s)
- Lynnea M Mills
- Department of Medicine, University of California, San Francisco, San Francisco, CA, USA
| | - Christy Boscardin
- Department of Anaesthesia and Perioperative Care and Medicine, University of California, San Francisco, San Francisco, CA, USA
| | - Elizabeth A Joyce
- Department of Microbiology and Immunology, University of California, San Francisco, San Francisco, CA, USA
| | - Olle Ten Cate
- Center for Research and Development of Education, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Patricia S O'Sullivan
- Departments of Medicine and Surgery, University of California, San Francisco, San Francisco, CA, USA
| |
Collapse
|
6
|
McGaha P, Klingensmith ME, Cross A, Stewart K, Grimes A, Horton M, Kowdley G, Patel K, Galante J, Wise PE, Delman KA, Joshi ART. Trainee Utilization of the SCORE Curriculum is Associated With Improved ABSITE Performance: A Multi-institutional Study. JOURNAL OF SURGICAL EDUCATION 2021; 78:e196-e200. [PMID: 34384730 DOI: 10.1016/j.jsurg.2021.06.022] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Accepted: 06/27/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND The Surgical Council on Resident Education's (SCORE) structured educational curriculum for general surgery residents uses a 2-year repeating cycle of modules and quizzes called "This Week in SCORE" (TWIS) to organize and sequence the curricular content on the SCORE portal. The purpose of this study was to evaluate the impact of using the SCORE content and TWIS curriculum on American Board of Surgery In-Training Examination (ABSITE) performance. To date, no multi-institutional studies have examined this effect. METHODS Eight residency programs participated, including university and community-based programs. SCORE usage overall, TWIS usage, and ABSITE percentile scores (adjusted for post-graduate year [PGY]) were analyzed for the academic years 2017 to 2020. SCORE usage was defined as number of SCORE logins annually per resident with "low usage" ≤10 times and "high usage" >10 times. TWIS usage was defined as "low usage" (no TWIS quizzes done) or "high usage" (≥1 TWIS quiz per year). RESULTS Four hundred and twenty-eight trainees were evaluated. Trainees with high SCORE and TWIS usage consistently achieved a higher ABSITE percentile score. This difference was maintained in subgroup analysis by PGY with the greatest impact in PGY-1 and PGY-2 levels. CONCLUSIONS Utilization of multiple aspects of SCORE content appears to have a significant positive impact on ABSITE performance across all levels of postgraduate training. This multi-institutional study of a large number of users is the first to demonstrate that increased usage of SCORE content appears to be a predictor of ABSITE performance success.
Collapse
Affiliation(s)
- Paul McGaha
- University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma
| | | | - Alisa Cross
- University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma
| | - Kenneth Stewart
- University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma
| | - Arthur Grimes
- University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma
| | | | | | - Kevin Patel
- Kaiser Permanente Los Angeles Medical Center, Los Angeles, California
| | | | - Paul E Wise
- Washington University St. Louis, St. Louis, Missouri
| | | | - Amit R T Joshi
- Einstein Healthcare Network, Jefferson Medical College, Philadelphia, Pennsylvania.
| |
Collapse
|
7
|
To H, Cargill A, Tobin S, Nestel D. Remediation for surgical trainees: recommendations from a narrative review. ANZ J Surg 2021; 91:1117-1124. [PMID: 33538072 DOI: 10.1111/ans.16637] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2020] [Revised: 01/13/2021] [Accepted: 01/15/2021] [Indexed: 11/27/2022]
Abstract
BACKGROUND Remediation involves formalized support for surgical trainees with significant underperformance to return to expected standards. There is a need to understand current evidence of remediation for surgical trainees to inform practice and justify investment of resources. METHODS Following the principles of a systematic review, we conducted a narrative analysis to make recommendations for remediation of underperforming surgical trainees. RESULTS From a review of 55 articles on remediation of trainees in medical and surgical sub-specialities, we have identified system and process level recommendations. Remediation is reported as long-term, complex and resource-intensive. Establishing a defined and standardized remediation framework enables co-ordination of multi-modal interventions. System level recommendations aim to consolidate protocols via developing better assessment, intervention and re-evaluation modalities whilst also strengthening support to supervisors conducting the remediation. Process level recommendations should be tailored for the specific needs of each trainee, aiming to be proactive with interventions within a programmatic framework. Regular reassessment is required, and long-term follow-up shows that remediation efforts are often successful. CONCLUSION While remediation within a programmatic framework is complex, it is often a successful approach to return surgical trainees to their expected standard. Future directions involve applying learning theories, encouraging research methods and to develop integrated collaborate protocols and support to synergize efforts.
Collapse
Affiliation(s)
- Henry To
- The University of Melbourne, Melbourne, Victoria, Australia
| | - Ashleigh Cargill
- Department of Surgery, St Vincent's Hospital, Melbourne, Victoria, Australia
| | - Stephen Tobin
- School of Medicine, Western Sydney University, Sydney, New South Wales, Australia
| | - Debra Nestel
- Department of Surgery (Austin), The University of Melbourne, Melbourne, Victoria, Australia.,Monash Institute for Health and Clinical Education, Monash University, Melbourne, Victoria, Australia
| |
Collapse
|
8
|
Cassidy DJ, Chakraborty S, Panda N, McKinley SK, Mansur A, Hamdi I, Mullen J, Petrusa E, Phitayakorn R, Gee D. The Surgical Knowledge "Growth Curve": Predicting ABSITE Scores and Identifying "At-Risk" Residents. JOURNAL OF SURGICAL EDUCATION 2021; 78:50-59. [PMID: 32694087 DOI: 10.1016/j.jsurg.2020.06.038] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Revised: 06/09/2020] [Accepted: 06/28/2020] [Indexed: 06/11/2023]
Abstract
OBJECTIVE Resident performance on the American Board of Surgery In-Training Examination (ABSITE) is used for evaluation of surgical knowledge and guides resident selection for institutional remediation programs. Remediation thresholds have historically been based on ABSITE percentile scores; however, this does not account for predictors that can impact a resident's exam performance. We sought to identify predictors of yearly ABSITE performance to help identify residents "at-risk" for performing below their expected growth trajectory. DESIGN The knowledge of the residents, as measured by standardized ABSITE scores, was modeled as a function of the corresponding postgraduate year via a linear mixed effects regression model. Additional model covariates included written USMLE-1-3 examination scores, gender, number of practice questions completed, and percentage correct of practice questions. For each resident, the predicted ABSITE standard score along with a 95% bootstrap prediction interval was obtained. Both resident-specific and population-level predictions for ABSITE standard scores were also estimated. SETTING The study was conducted at a single, large academic medical center (Massachusetts General Hospital, Boston, MA). PARTICIPANTS Six years of general surgery resident score reports at a single institution between 2014 and 2019 were deidentified and analyzed. RESULTS A total of 376 score reports from 130 residents were analyzed. Covariates that had a significant effect on the model included USMLE-1 score (PGY1: p = 0.013; PGY2: p = 0.007; PGY3: p = 0.011), USMLE-2 score (PGY1: p < 0.001; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p < 0.001; PGY5: p = 0.032), male gender (PGY1: p = 0.003; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p = 0.008), and number of practice questions completed (p=0.003). Five residents were identified as having "fallen off" their predicted knowledge curve, including a single resident on 2 occasions. Population prediction curves were obtained at 7 different covariate percentile levels (5%, 10%, 25%, 50%, 75%, 90%, and 95%) that could be used to plot predicted resident knowledge progress. CONCLUSION Performance on USMLE-1 and -2 examinations, male gender, and number of practice questions completed were positive predictors of ABSITE performance. Creating residency-wide knowledge growth curves as well as individualized predictive ABSITE performance models allows for more efficient identification of residents potentially at risk for poor ABSITE performance and structured monitoring of surgical knowledge progression.
Collapse
Affiliation(s)
- Douglas J Cassidy
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts.
| | - Saptarshi Chakraborty
- Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, Manhattan, New York
| | - Nikhil Panda
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Sophia K McKinley
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Arian Mansur
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Isra Hamdi
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - John Mullen
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Emil Petrusa
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Roy Phitayakorn
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Denise Gee
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| |
Collapse
|