Vascular In-person for Students In the match Trial: An investigation of postinterview site visits to address the limitations of virtual interviews

PURPOSE

The graduate medical education community implemented virtual residency interviews in response to travel restrictions during the COVID-19 pandemic, and this approach has persisted. Although many residency applicants wish to visit in-person prospective training sites, such opportunities could bias programs toward those who are able to meet this financial burden, exacerbating equity concerns. One proposed solution is to offer applicants the opportunity to visit only after a program's rank list is "locked," avoiding favoritism to applicants who visit, but allowing applicants to experience some of the camaraderie, geography, and local effects of an in-person visit. As debate about the optimal format of residency interviews continues, it is important to investigate whether in-person program visits, completed after program rank list certification, provide meaningful benefits to applicants in the residency match process.

METHODS

All vascular programs entering the 2023 integrated vascular surgery residency match were invited to participate. Programs agreed to certify their National Resident Matching Program rank lists by February 1, 2023. Applicants then had the opportunity to visit the programs at which they interviewed. The particulars of the visit were determined by the individual programs. Applicants completed their standard rank list and locked on the standard date: March 1, 2023. Applicants then completed a survey regarding the impact of the visits on their rank order list decision-making. Program directors (PDs) completed a survey regarding their experiences as well. Data were collected using REDCap.

RESULTS

Twenty-one of the 74 (28%) programs participated. Nineteen PDs completed the postinterview site visit survey (response rate 90%). Applicants interviewing at the participating programs (n = 112) were informed of the study, offered the opportunity to attend postinterview site visits, and received the survey. Forty-seven applicants responded (response rate 42%). Eighty-six percent of applicants stated that the visit impacted their rank list. Most important factors were esprit de corps of the program (86%), the faculty/trainees/staff (81%), and the physical setting (62%). Seventy-one percent of those participating spent ≤$800 on their visit. Eighty-one percent were satisfied with the process. Twenty-one percent of PDs would have changed their rank list if they could have based on the applicants' in-person visit. Sixty-three percent of the visit sessions cost the programs ≤$500, and 63% were satisfied with the process.

CONCLUSIONS

This study is the first to document the impact of in-person site visits by applicants on a graduate medical education match process in one specialty. Our results suggest that this process provides meaningful data to applicants that helped them with their decision-making evidenced by most altering their rank lists, while avoiding some of the critical equity issues that accompany traditional in-person interviews. This may provide a model for future interview processes for residency programs.

Co-Existing Vascular Surgery Integrated Residencies are Associated with Increased General Surgery Resident Proficiency and Autonomy in Vascular Cases

BACKGROUND

Integrated vascular surgery residency positions have doubled more than the last decade. Studies have investigated the impact of co-existing subspecialty surgical training programs on case volume of general surgery residents (GSRs). However, no studies have explored the impact of subspecialty training on GSR operative competency. The aim of this study is to understand the impact of integrated residencies on operative performance and autonomy of GSR performing vascular procedures.

METHODS

Autonomy and performance ratings of GSR participating in vascular surgery cases were collected from all institutions participating in the Society for Improving Medical Professional Learning application database from 2015 to 2023. Faculty and self-assessments of autonomy and performance on vascular cases performed by GSRs at programs with co-existing vascular integrated residency (VIR), vascular surgery fellowship (VSF), or no subspecialty vascular training (VX) were compared using Fisher's exact tests with Bonferroni corrections across training levels and case complexity.

RESULTS

Eleven thousand one hundred seventy five assessments (26% at institutions with VIR, 46% VSF, and 28% VX) were submitted by 920 GSRs and 343 faculty. Senior GSRs at programs with VSF achieved lower autonomy than those with VIR (P = 0.049) or VX (P = 0.042) based on faculty assessment. GSRs achieved a level of "practice ready" at significantly higher rates when training at programs with VIR, and at the lowest rates with VSF (P < 0.001). However, self-perception of autonomy and performance was highest among GSRs at programs with VX compared with VIR and VSF (P < 0.001).

CONCLUSIONS

The presence of VIR was associated with higher achievement of "practice ready" competency and higher levels of operative autonomy among senior GSRs performing vascular procedures. Shared-learning among peers and faculty expertise in teaching resident-level trainees may contribute to this finding.

Gender differences in autonomy and performance assessments in a national cohort of vascular surgery trainees

OBJECTIVE

Gender disparities in surgical training and assessment are described in the general surgery literature. Assessment disparities have not been explored in vascular surgery. We sought to investigate gender disparities in operative assessment in a national cohort of vascular surgery integrated residents (VIRs) and fellows (VSFs).

METHODS

Operative performance and autonomy ratings from the Society for Improving Medical Professional Learning (SIMPL) application database were collected for all vascular surgery participating institutions from 2018 to 2023. Logistic generalized linear mixed models were conducted to examine the association of faculty and trainee gender on faculty and self-assessment of autonomy and performance. Data were adjusted for post-graduate year and case complexity. Random effects were included to account for clustering effects due to participant, program, and procedure.

RESULTS

One hundred three trainees (n = 63 VIRs; n = 40 VSFs; 63.1% men) and 99 faculty (73.7% men) from 17 institutions (n = 12 VIR and n = 13 VSF programs) contributed 4951 total assessments (44.4% by faculty, 55.6% by trainees) across 235 unique procedures. Faculty and trainee gender were not associated with faculty ratings of performance (faculty gender: odds ratio [OR], 0.78; 95% confidence interval [CI], 0.27-2.29; trainee gender: OR, 1.80; 95% CI, 0.76-0.43) or autonomy (faculty gender: OR, 0.99; 95% CI, 0.41-2.39; trainee gender: OR, 1.23; 95% CI, 0.62-2.45) of trainees. All trainees self-assessed at lower performance and autonomy ratings as compared with faculty assessments. However, women trainees rated themselves significantly lower than men for both autonomy (OR, 0.57; 95% CI, 0.43-0.74) and performance (OR, 0.40; 95% CI, 0.30-0.54).

CONCLUSIONS

Although gender was not associated with differences in faculty assessment of performance or autonomy among vascular surgery trainees, women trainees perceive themselves as performing with lower competency and less autonomy than their male colleagues. These findings suggest utility for exploring gender differences in real-time feedback delivered to and received by trainees and targeted interventions to align trainee self-perception with actual operative performance and autonomy to optimize surgical skill acquisition.

Health Disparities Curricula in General Surgery Residency Programs: A Critical Scoping Review

INTRODUCTION

In 2021, the structural determinants of health (SDOH) were added to the Accreditation Council of Graduate Medical Education common program requirements for all accredited residency programs, including general surgery. In this study, we sought to explore the current scope of, and concepts used in, health disparities curricula for general surgery residents, specifically investigating how general surgery residents learn about health disparities and the SDOH.

METHODS

We searched PubMed, EMBASE, Education Research Complete (EBSCOhost), and Web of Science Core Collection using keywords related to health disparities and the SDOH. Inclusion criteria consisted of all studies published after 2005 that discussed health disparities curricula for Accreditation Council of Graduate Medical Education-accredited general surgery residency programs. Five thousand three hundred seventeen articles were screened using a two-phase process. Data extraction and analysis was performed using critical review methods.

RESULTS

Seventeen articles were identified. Within these articles, seven unique health disparities curricula were found. All seven of the identified curricula employed cultural frameworks as methods to mitigate health disparities. Three curricula, all published after 2011, included education on the SDOH. A wide variety of educational methods were utilized; in-person didactics was the most common.

CONCLUSIONS

In the current literature, culture continues to play a large role in health disparities training for general surgery residents. Though further efforts are needed to understand the methods used in programs that have not published scholarly work, it is imperative to ensure that residents are provided with the sociopolitical perspective needed to understand the SDOH and serve all patients, including those affected by health disparities.

Accreditation Council for Graduate Medical Education Milestone Training Ratings and Surgeons' Early Outcomes

Importance

National data on the development of competence during training have been reported using the Accreditation Council for Graduate Medical Education (ACGME) Milestones system. It is now possible to consider longitudinal analyses that link Milestone ratings during training to patient outcomes data of recent graduates.

Objective

To evaluate the association of in-training ACGME Milestone ratings in a surgical specialty with subsequent complication rates following a commonly performed operation, endovascular aortic aneurysm repair (EVAR).

Design, Setting, and Participants

This study of patient outcomes followed EVAR in the Vascular Quality Initiative (VQI) registry (4213 admissions from 208 hospitals treated by 327 surgeons). All surgeons included in this study graduated from ACGME-accredited training programs from 2015 through 2019 and had Milestone ratings 6 months prior to graduation. Data were analyzed from December 1, 2021, through September 15, 2023. Because Milestone ratings can vary with program, they were corrected for program effect using a deviation score from the program mean.

Exposure

Milestone ratings assigned to individual trainees 6 months prior to graduation, based on judgments of surgical competence.

Main Outcomes and Measures

Surgical complications following EVAR for patients treated by recent graduates during the index hospitalization, obtained using the nationwide Society for Vascular Surgery Patient Safety Organization's VQI registry, which includes 929 participating centers in 49 US states.

Results

The study included outcomes for 4213 patients (mean [SD] age, 73.25 [8.74] years; 3379 male participants [80.2%]). Postoperative complications included 9.5% major (400 of 4213 cases) and 30.2% minor (1274 of 4213 cases) complications. After adjusting for patient risk factors and site of training, a significant association was identified between individual Milestone ratings of surgical trainees and major complications in early surgical practice in programs with lower mean Milestone ratings (odds ratio, 0.50; 95% CI; 0.27-0.95).

Conclusions and Relevance

In this study, Milestone assessments of surgical trainees were associated with subsequent clinical outcomes in their early career. Although these findings represent one surgical specialty, they suggest Milestone ratings can be used in any specialty to identify trainees at risk for future adverse patient outcomes when applying the same theory and methodology. Milestones data should inform data-driven educational interventions and trainee remediation to optimize future patient outcomes.

Impact of COVID-19 on patients undergoing scheduled procedures for chronic venous disease

OBJECTIVE

The COVID-19 pandemic has drastically altered the medical landscape. Various strategies have been employed to preserve hospital beds, personal protective equipment, and other resources to accommodate the surges of COVID-19 positive patients, hospital overcapacities, and staffing shortages. This has had a dramatic effect on vascular surgical practice. The objective of this study is to analyze the impact of the COVID-19 pandemic on surgical delays and adverse outcomes for patients with chronic venous disease scheduled to undergo elective operations.

METHODS

The Vascular Surgery COVID-19 Collaborative (VASCC) was founded in March 2020 to evaluate the outcomes of patients with vascular disease whose operations were delayed. Modules were developed by vascular surgeon working groups and tested before implementation. A data analysis of outcomes of patients with chronic venous disease whose surgeries were postponed during the COVID-19 pandemic from March 2020 through February 2021 was performed for this study.

RESULTS

A total of 150 patients from 12 institutions in the United States were included in the study. Indications for venous intervention were: 85.3% varicose veins, 10.7% varicose veins with venous ulceration, and 4.0% lipodermatosclerosis. One hundred two surgeries had successfully been completed at the time of data entry. The average length of the delay was 91 days, with a median of 78 days. Delays for venous ulceration procedures ranged from 38 to 208 days. No patients required an emergent intervention due to their venous disease, and no patients experienced major adverse events following their delayed surgeries.

CONCLUSIONS

Interventions may be safely delayed for patients with venous disease requiring elective surgical intervention during the COVID-19 pandemic. This finding supports the American College of Surgeons' recommendations for the management of elective vascular surgical procedures. Office-based labs may be safe locations for continued treatment when resources are limited. Although the interventions can be safely postponed, the negative impact on quality of life warrants further investigation.

Impact of Integrated Vascular Surgery Residency Training Pathway and Professional Development Time on Career Choice and Research Productivity

OBJECTIVE

The limited availability of academic surgery positions has led to increased competition for these jobs. Integrated vascular surgery residency (IVSR) allows for earlier specialization, with some programs providing professional development time (PDT). We hypothesized that IVSR and PDT lead to academic employment and increased research productivity.

DESIGN

This is a retrospective study of vascular surgery fellowship (VSF) and IVSR graduates.

SETTING

Training, number of publications, H-index, NIH funding, and employment history were collected using institutional websites, Doximity, Scopus, PubMed, and NIH Research Portfolio Reporting.

PARTICIPANTS

After a review of the research protocol, the Association of Program Directors in Vascular Surgery (APDVS) provided a list of vascular surgery fellowship (VSF) and IVSR graduates.

METHODS

After review of the research protocol, the Association of Program Directors in Vascular Surgery (APDVS) provided a list of vascular surgery fellowship (VSF) and IVSR graduates. Training, number of publications, H-index, NIH funding, and employment history were collected using institutional websites, Doximity, Scopus, PubMed, and NIH Research Portfolio Reporting.

RESULTS

From 2013-2017, comparison of IVSR (n=131) to VSF (n=603) graduates showed that IVSR graduates were more likely to be women (38.17% vs 28.19%; p = 0.024), be MD graduates (99.24% vs 93.37%; p = 0.008), attended programs in the northeast (41.98% vs 27.5%; p < 0.001), have advanced degrees (13.74% vs 6.97%; p = 0.01) and graduate from larger programs (median 15 vs 14 faculty; p = 0.013). There was no significant difference in number of publications per trainee by the end of training (median 4 vs 3; P=0.61) or annual trend in average number of publications. After training, there was no significant difference in the type of practice, academic affiliation, practice region, publication number, H-index, NIH funding, level of academic appointment, or leadership positions. From 2013-2019, a comparison of IVSR graduates with (n=32) and without PDT (n=190) demonstrated that those with PDT were more likely to be women (53.13% vs 34.74%; p = 0.038), have advanced degrees (28.12% vs 8.95%; p = 0.002), be at larger programs (median 14 vs 9 faculty; p < 0.001), train at a top 10 NIH funded program (65.62% vs 21.58%; p < 0.001) and publish more by the end of IVSR (median 9 vs 3; p < 0.001). Graduates with PDT were more likely to have academic employment and affiliation, a higher yearly publication rate, and greater H-index.

CONCLUSION

IVSR and VSF graduates have comparable academic employment and research productivity. However, PDT during IVSR correlates with an eventual academic career and greater research productivity. This study supports the importance of PDT in developing academic vascular surgeons. It remains necessary to continue both IVSR and VSF training paradigms as healthcare needs of the population are met through both academic and non-academic surgeons.

The Association of ACGME Milestones With Performance on American Board of Surgery Assessments: A National Investigation of Surgical Trainees

OBJECTIVE

To determine the relationship between, and predictive utility of, milestone ratings and subsequent American Board of Surgery (ABS) vascular surgery in-training examination (VSITE), vascular qualifying examination (VQE), and vascular certifying examination (VCE) performance in a national cohort of vascular surgery trainees.

BACKGROUND

Specialty board certification is an important indicator of physician competence. However, predicting future board certification examination performance during training continues to be challenging.

METHODS

This is a national longitudinal cohort study examining relational and predictive associations between Accreditation Council for Graduate Medical Education (ACGME) Milestone ratings and performance on VSITE, VQE, and VCE for all vascular surgery trainees from 2015 to 2021. Predictive associations between milestone ratings and VSITE were conducted using cross-classified random-effects regression. Cross-classified random-effects logistic regression was used to identify predictive associations between milestone ratings and VQE and VCE.

RESULTS

Milestone ratings were obtained for all residents and fellows(n=1,118) from 164 programs during the study period (from July 2015 to June 2021), including 145,959 total trainee assessments. Medical knowledge (MK) and patient care (PC) milestone ratings were strongly predictive of VSITE performance across all postgraduate years (PGYs) of training, with MK ratings demonstrating a slightly stronger predictive association overall (MK coefficient 17.26 to 35.76, β = 0.15 to 0.23). All core competency ratings were predictive of VSITE performance in PGYs 4 and 5. PGY 5 MK was highly predictive of VQE performance [OR 4.73, (95% CI, 3.87-5.78), P <0.001]. PC subcompetencies were also highly predictive of VQE performance in the final year of training [OR 4.14, (95% CI, 3.17-5.41), P <0.001]. All other competencies were also significantly predictive of first-attempt VQE pass with ORs of 1.53 and higher. PGY 4 ICS ratings [OR 4.0, (95% CI, 3.06-5.21), P <0.001] emerged as the strongest predictor of VCE first-attempt pass. Again, all subcompetency ratings remained significant predictors of first-attempt pass on CE with ORs of 1.48 and higher.

CONCLUSIONS

ACGME Milestone ratings are highly predictive of future VSITE performance, and first-attempt pass achievement on VQE and VCE in a national cohort of surgical trainees.

Individual and Program-Related Predictors of Academic Vascular Surgery Practice

BACKGROUND

Several studies have explored factors affecting academic employment in surgical subspecialties; however, vascular surgery has not yet been investigated. We examined which elements of surgical training predict future academic productivity and studied characteristics of NIH-funded vascular surgery attendings.

METHODS

With approval from the Association of Program Directors in Vascular Surgery (APDVS), the database of recent vascular surgery fellowship (VSF) and integrated vascular surgery residency (IVSR) graduates was obtained, and public resources (Doximity, Scopus, PubMed, NIH, etc.) were queried for research output during and after training, completion of dedicated research years, individual and program NIH funding, current practice setting, and academic rank. Adjusted multivariate regression analyses were conducted for postgraduate academic productivity.

RESULTS

From 2013 to 2017, there were 734 graduates. Six hundred three completed VSF and 131 IVSR; 220 (29%) were female. Academic employment was predicted by MD degree, advanced degree, training at a top NIH-funded program, number publications by end of training, and H-index. Dedicated research time before or during vascular training, advanced degree, or graduating from a top NIH-funded program were predictors of publishing >1 paper/year. Number of publications by end of training and years in practice were predictive of H-index ≥5. VSF versus IVSR pathway did not have an impact on future academic employment, annual publication rate as an attending, or H-index. Characterization of NIH-funded attendings showed that they often completed dedicated research time (72%) and trained at a top NIH-funded program (79%). Mean publications by graduation among this group was 15.82 ± 11.3, and they averaged 4.31 ± 4.2 publications/year as attendings.

CONCLUSIONS

Research output during training, advanced degrees, and training at a top NIH-funded program predict an academic vascular surgery career. VSF and IVSR constitute equally valid paths to productive academic careers.

Early findings and strategies for successful implementation of SIMPL workplace-based assessments within vascular surgery residency and fellowship programs

OBJECTIVE

As medical education systems increasingly move toward competency-based training, it is important to understand the tools available to assess competency and how these tools are utilized. The Society for Improving Medical Professional Learning (SIMPL) offers a smart phone-based assessment system that supports workplace-based assessment of residents' and fellows' operative autonomy, performance, and case complexity. The purpose of this study was to characterize implementation of the SIMPL app within vascular surgery integrated residency (0+5) and fellowship (5+2) training programs.

METHODS

SIMPL operative ratings recorded between 2018 and 2022 were collected from all participating vascular surgery training institutions (n = 9 institutions with 5+2 and 0+5 programs; n = 4 institutions with 5+2 program only). The characteristics of programs, trainees, faculty, and SIMPL operative assessments were evaluated using descriptive statistics.

RESULTS

Operative assessments were completed for 2457 cases by 85 attendings and 86 trainees, totaling 4615 unique operative assessment ratings. Attendings included dictated feedback in 52% of assessments. Senior-level residents received more assessments than junior-level residents (postgraduate year [PGY]1-3, n = 439; PGY4-5, n = 551). Performance ratings demonstrated increases from junior to senior trainees for both resident and fellow cohorts with "performance-ready" or "exceptional performance" ratings increasing by nearly two-fold for PGY1 to PGY5 residents (28.1% vs 40.6%), and from first- to second-year fellows (PGY6, 46.7%; PGY7, 60.3%). Similar gains in autonomy were demonstrated as trainees progressed through training. Senior residents were more frequently granted autonomy with "supervision only" than junior residents (PGY1, 8.7%; PGY5, 21.6%). "Supervision only" autonomy ratings were granted to 21.8% of graduating fellows. Assessment data included a greater proportion of complex cases for senior compared with junior fellows (PGY6, 20.9% vs PGY7, 26.5%). Program Directors felt that faculty and trainee buy-in were the main barriers to implementation of the SIMPL assessment app.

CONCLUSIONS

This is the first description of the SIMPL app as an operative assessment tool within vascular surgery that has been successfully implemented in both residency and fellowship programs. The assessment data demonstrates expected progressive gains in trainees' autonomy and performance, as well as increasing case complexity, across PGY years. Given the selection of SIMPL as the assessment platform for required American Board of Surgery and Vascular Surgery Board Entrustable Professional Activities assessments, understanding facilitators and barriers to implementation of workplace-based assessments using this app is imperative, particularly as we move toward competency-based medical education.

Surgical Trainee Perspectives on the Opioid Crisis: The Influence of Explicit and Hidden Curricula

OBJECTIVE

In order to effectively create and implement an educational program to improve opioid prescribing practices, it is important to first consider the unique perspectives of residents on the frontlines of the opioid epidemic. We sought to better understand resident perspectives on opioid prescribing, current practices in pain management, and opioid education as a needs assessment for designing future educational interventions.

DESIGN

This is a qualitative study using focus groups of surgical residents at 4 different institutions.

SETTING

We conducted focus groups using a semistructured interview guide in person or over video conferencing. The residency programs selected for participation represent a broad geographic range and varying residency sizes.

PARTICIPANTS

We used purposeful sampling to recruit general surgery residents from the University of Utah, University of Wisconsin, Dartmouth-Hitchcock Medical Center, and the University of Alabama at Birmingham. All general surgery residents at these locations were eligible for inclusion. Participants were assigned to focus groups by residency site and their status as junior (PGY-2, PGY-3) or senior resident (PGY-4, PGY-5).

RESULTS

We completed 8 focus groups with a total of 35 residents included. We identified 4 main themes. First, residents relied on clinical and nonclinical factors when making decisions about opioid prescribing. However, hidden curricula based on unique institutional cultures and attending preferences heavily influenced residents' prescribing practices. Second, residents acknowledged that stigma and biases towards certain patient groups influenced opioid prescribing practices. Third, residents encountered barriers within their health systems to evidence-based opioid prescribing. Fourth, residents did not routinely receive formal education on pain management or opioid prescribing. Residents recommended several interventions to improve the current state of opioid prescribing, including standardized prescribing guidelines, improved patient education, and formal training during the first year of residency.

CONCLUSIONS

Our study highlighted several areas of opioid prescribing that can be improved upon through educational interventions. These findings can be used to develop programs aimed at improving residents' opioid prescribing practices, both during and after training, and ultimately the safe care of surgical patients.

ETHICS STATEMENT

This project was approved by the University of Utah Institutional Review Board, ID # 00118491. All participants provided written informed consent.

The USMLE® STEP 1 Pass or Fail Era of the Vascular Surgery Residency Application Process: Implications for Structural Bias and Recommendations

MOTIVES BEHIND THE SHIFT

USMLE® STEP 1 score reporting has been changed to a binary pass/fail format since January 26, 2022. The motives behind this change were (1) the questionable validity of using USMLE® STEP 1 as a screening tool during the candidate selection process and (2) the negative impact of using standardized examination scores as an initial gatekeeping threshold for the underrepresented in medicine (URiM) candidates applying to GME programs, given their generally lower mean standardized exams scores compared to non-URiM students. The USMLE® administrators justified this change as a tactic to enhance the overall educational experience for all students and to increase the representation of URiM groups. Moreover, they advised the program directors (PDs) to give more attention to other important qualities and components such as the applicant's personality traits, leadership roles and other extracurricular accomplishments, as part of a holistic evaluation strategy. IMPACT OF USMLE® STEP 1 PASS/FAIL: At this early stage, it is unclear how this change will impact Vascular Surgery Integrated residency (VSIR) programs. Several questions are outstanding, most importantly, how VSIR PDs will evaluate applicants absent the variable which heretofore was the primary screening tool. Our previously published survey showed that VSIR PDs will move their attention to other measures such as USMLE® STEP 2CK and letters of recommendation during the VSIR selection process. Furthermore, more emphasis on subjective measures such as the applicant's medical school rank and extracurricular student activities is expected. Given the expected higher weight of USMLE® STEP 2CK in the selection process than ever, many anticipate that medical students will dedicate more of their limited time to its preparation at the expense of both clinical and non-clinical activities. Potentially leaving less time to explore specialty pathways and to determine whether VS is the appropriate career for them.

A FRAMEWORK FOR CANDIDATE EVALUATION

The critical juncture in the VSIR candidate evaluation paradigm presents an opportunity to thoughtfully transform the process via current (Standardized Letter of Recommendation, USMLE® STEP 2CK, and clinical research) and future (Emotional Intelligence, Structure Interview and Personality Assessment) measures which constitute a framework to follow in the USMLE® STEP 1 pass/fail era.

The Use of Learning Analytics to Enable Detection of Underperforming Trainees: An Analysis of National Vascular Surgery Trainee ACGME Milestones Assessment Data

OBJECTIVE

This study aims to investigate at-risk scores of semiannual Accreditation Council for Graduate Medical Education (ACGME) Milestone ratings for vascular surgical trainees' final achievement of competency targets.

SUMMARY BACKGROUND DATA

ACGME Milestones assessments have been collected since 2015 for Vascular Surgery. It is unclear whether milestone ratings throughout training predict achievement of recommended performance targets upon graduation.

METHODS

National ACGME Milestones data were utilized for analyses. All trainees completing 2-year vascular surgery fellowships in June 2018 and 5-year integrated vascular surgery residencies in June 2019 were included. A generalized estimating equations model was used to obtain at-risk scores for each of the 31 subcompetencies by semiannual review periods, to estimate the probability of trainees achieving the recommended graduation target based on their previous ratings.

RESULTS

A total of 122 vascular surgery fellows (VSFs) (95.3%) and 52 integrated vascular surgery residents (IVSRs) (100%) were included. VSFs and IVSRs did not achieve level 4.0 competency targets at a rate of 1.6% to 25.4% across subcompetencies, which was not significantly different between the 2 groups for any of the subcompetencies ( P = 0.161-0.999). Trainees were found to be at greater risk of not achieving competency targets when lower milestone ratings were assigned, and at later time-points in training. At a milestone rating of 2.5, with 1 year remaining before graduation, the at-risk score for not achieving the target level 4.0 milestone ranged from 2.9% to 77.9% for VSFs and 33.3% to 75.0% for IVSRs.

CONCLUSION

The ACGME Milestones provide early diagnostic and predictive information for vascular surgery trainees' achievement of competence at completion of training.

Impact of frailty on risk of long-term functional decline following vascular surgery

OBJECTIVE

Frailty is a clinical syndrome associated with slow recovery after vascular surgery. However, the degree and length of functional impairment frail patients experience after surgery is unclear. The objective of this study was to prospectively measure changes in functional status among frail and non-frail patients undergoing a spectrum of different vascular surgery procedures.

METHODS

Patients consented to undergo elective minor and major vascular surgery procedures at an academic medical center between May 2018 and March 2019 were prospectively identified. Prior to surgery, all patients underwent provider assessment of frailty using the validated Clinical Frailty Scale (CFS), as well as baseline assessment of functional status using the Katz Activities of Daily Living (ADL) index and the Lawton Instrumental Activities of Daily Living (iADL) index. These same instruments were used to evaluate each patient's functional status at 2-weeks, 1-month, 1-year, and 2-year time points following surgery. Changes in iADL and ADL scores among frail (CFS ≥5) and non-frail patients were compared using paired Wilcoxon signed-rank tests and logistic regression models.

RESULTS

A total of 126 patients were assessed before and after minor (55%) and major (45%) vascular procedures, of which 43 patients (34%) were determined to be frail prior to surgery. Frail patients were older and more likely than non-frail patients to have medical comorbidities including chronic kidney disease, chronic obstructive pulmonary disease, or diabetes (all P < .05). When compared with the non-frail cohort, frail patients had significantly lower ADL and iADL scores before surgery and experienced a greater decline in ability to independently complete ADL and iADL activities after surgery that was sustained at 2 years (P < .05 and P < .001, respectively). After risk-adjustment, frailty was associated with an increased likelihood of decline in ADLs (odds ratio, 5.4; 95% confidence interval, 1.9-15.4; P < .05) and iADLs (odds ratio, 6.3; 95% confidence interval, 2.6-15.1; P < .001) at 2 years following surgery.

CONCLUSIONS

Frail patients experience a significant decline in ability to perform ADL and iADLs that persists 2 years following vascular surgery. These data highlight the degree of functional decline occurring immediately following surgery, as well as risk for long-term, sustained impairment that should be shared with frail patients before undergoing a procedure.

Vascular surgery integrated resident selection criteria in the pass or fail era

OBJECTIVES

Vascular surgery integrated residency (VSIR) programs are highly competitive; however, criteria for resident selection remain opaque and non-standardized. The already unclear selection criteria will be further impacted by the impending transition of the United States Medical Licensing Examination (USMLE) Step 1 from numeric scores to a binary pass/fail outcome. The purpose of this study was to investigate the historical and anticipated selection criteria of VSIR applicants.

METHODS

This was a cross-sectional, nationwide, 59-item survey that was sent to all VSIR program directors (PDs). Data was analyzed using the Fisher exact test if categorical and the Mann-Whitney U test and the Kruskal-Wallis test if ordinal.

RESULTS

Forty of 69 PDs (58%) responded to the survey. University-based programs constituted 85% of responders. Most VSIR PDs (65%) reported reviewing between 101 to 150 applications for 1 to 2 positions annually. Forty-two percent of the responding PDs reported sole responsibility for inviting applicants to interview, whereas 50% had a team of faculty responsible for reviewing applications. On a five-point Likert scale, letters of recommendation (LOR) from vascular surgeons or colleagues (a person the PD knows) were the most important objective criteria. Work within a team structure was rated highest among subjective criteria. The majority of respondents (72%) currently use the Step 1 score as a primary method to screen applicants. Regional differences in use of Step 1 score as a primary screening method were: Midwest (100%), Northeast (76%), South (43%), and West (40%) (P = .01). PDs responded that that they will use USMLE Step 2 score (42%) and LOR (10%) to replace USMLE Step 1 score. The current top ranked selection criteria are letters from a vascular surgeon, USMLE Step 1 score and overall LOR. The proposed top ranked selection criteria after transition of USMLE Step 1 to pass/fail include LOR overall followed by Step 2 score.

CONCLUSIONS

This is the first study to evaluate the selection criteria used by PDs for VSIR. The landscape of VSIR selection criteria is shifting and increasing transparency is essential to applicants' understanding of the selection process. The transition of USMLE Step 1 to a pass/fail report will shift the attention to Step 2 scores and elevate the importance of other relatively more subjective criteria. Defining VSIR program selection criteria is an important first step toward establishing holistic review processes that are transparent and equitable.

High-intensity statin therapy reduces risk of amputation and reintervention among patients undergoing lower extremity bypass for chronic limb-threatening ischemia

OBJECTIVE

Statins are considered standard-of-care medical therapy for patients undergoing lower extremity bypass (LEB) procedures for chronic limb-threatening ischemia (CLTI). It is unclear, however, whether up-titrating and maintaining patients on higher-intensity statin medications following LEB improves limb salvage outcomes. This study was designed to evaluate whether high-intensity statin therapy impacts the risk of amputation and reintervention following LEB for patients with CLTI.

METHODS

The IBM MarketScan database was used to identify adult patients (18-99 years old) who underwent a LEB for CLTI between 2008 and 2017. Patients lacking insurance covering drug reimbursement or those who already had undergone amputation before time of bypass were excluded. Using pharmacy claims and national drug codes to define statin intensity, patients were stratified into three groups: high-intensity, low-intensity, and limited statin therapy. The association between intensity of statin therapy and need for reintervention and/or major amputation after LEB was analyzed using Kaplan-Meier curves and risk-adjusted Cox proportional hazard models.

RESULTS

A total of 25,907 patients who underwent LEB for CLTI were identified, of which 6696 (26%) were maintained on high-dose statins, 9297 (36%) were on low-dose statins, and 9914 (38%) had inconsistent pharmacy claims for statin therapy after surgery. Patients on high-intensity statins were, on average, younger and more likely to be male with comorbid disease (diabetes, hypertension, hyperlipidemia, obesity, renal insufficiency, ischemic heart disease, cerebrovascular disease, and tobacco abuse) than patients on low-intensity statins or limited statin therapy (P < .001 for all comparisons). Following LEB, 6649 patients (25.6%) required a reintervention, and 2550 patients (9.8%) went on to have a major amputation during follow-up. Patients maintained on high-intensity statins after LEB had a significantly lower likelihood of requiring a reintervention (hazard ratio [HR], 0.48; 95% confidence interval [CI], 0.45-0.51; P < .001) or amputation (HR, 0.27; 95% CI, 0.24-0.30; P < .001) as compared with patients on limited statin therapy. Further, there was a dose-dependent effect for these outcomes relative to patients on low-intensity statins in risk-adjusted models, and it was independent of whether an autologous vein graft was used for the LEB. Finally, among patients who underwent a reintervention, high-dose statin therapy also significantly reduced the HR for subsequent amputation (HR, 0.21; 95% CI, 0.18-0.25; P < .001).

CONCLUSIONS

Patients with CLTI on high-intensity therapy following LEB had a significantly lower risk of requiring subsequent reintervention and amputation when compared with patients on low-intensity statins or with limited statin use. These data suggest that patients with CLTI should be up-titrated and/or maintained on high-intensity statins following revascularization whenever possible.

Predicting Performance at Graduation From Early ACGME Milestone Ratings: Longitudinal Learning Analytics in Professionalism and Communication in Vascular Surgery

OBJECTIVE

Program directors in surgical disciplines need more tools from the ACGME to help them use Milestone ratings to improve trainees' performance. This is especially true in competencies that are notoriously difficult to measure, such as professionalism (PROF) and interpersonal and communication skills (ICS). It is now widely understood that skills in these two areas have direct impact on patient care outcomes. This study investigated the potential for generating early predictors of final Milestone ratings within the PROF and ICS competency categories.

DESIGN

This retrospective cohort study utilized Milestone ratings from all ACGME-accredited vascular surgery training programs, covering residents and fellows who completed training in June 2019. The outcome measure studied was the rate of achieving the recommended graduation target of Milestone Level 4 (possible range: 1-5), while the predictors were the Milestone ratings attained at earlier stages of training. Predictive probability values (PPVs) were calculated for each of the 3 PROF and two ICS sub-competencies to estimate the probability of trainees not reaching the recommended graduation target based on their previous Milestone ratings.

SETTING

All ACGME-accredited vascular surgery training programs within the United States.

PARTICIPANTS

All trainees completing a 2 year vascular surgery fellowship (VSF) in June 2019 (n = 119) or a 5 year integrated vascular surgery residency (IVSR) in June 2019 (n = 52) were included in the analyses.

RESULTS

The overall rate of failing to achieve the recommended graduation target across all PROF and ICS sub-competencies ranged from 7.7% to 21.8% of all trainees. For trainees with a Milestone rating at ≤ 2.5 with 1 year remaining in their training program, the predictive probability of not achieving the recommended graduation target ranged from 37.0% to 71.5% across sub-competencies, with the highest risks observed under PROF for "Administrative Tasks" (71.5%) and under ICS for "Communication with the Healthcare Team" (56.7%).

CONCLUSIONS

As many as 1 in 4 vascular surgery trainees did not achieve the ACGME vascular surgery Milestones targets for graduation in at least one of the PROF and ICS sub-competencies. Biannual ACGME Milestone assessment ratings of PROF and ICS during early training can be used to predict achievement of competency targets at time of graduation. Early clues to problems in PROF and ICS enable programs to address potential deficits early in training to ensure competency in these essential non-technical skills prior to entering unsupervised practice.

Designing the "match of the future": challenges and proposed solutions in the interview and match phase of the UME-GME transition

This review focuses on the interview and match process with the purpose of broadly reviewing challenges in the current surgical residency selection process, detailing potential solutions, and identifying future avenues of investigation.

Educational Safety for the Surgical Learner

Psychological safety is known to improve team performance and organizational learning. The related concept of "educational safety" has recently emerged to describe an environment in which learners can unreservedly focus on learning and professional growth, without worrying about the potential repercussions of interpersonal risk-taking. Educational safety is crucial for optimal learning in clinical environments, and yet is difficult to establish due to constant performance assessment, fear of failure, and pervasive hierarchies. In this perspective, we propose a framework for conceptualizing educational safety in surgical learning environments, and explore current threats to educational safety. We also discuss strategies for combating these threats, as well as the importance of further research to evaluate the impact of educational safety on surgical learning outcomes.