Is Initial Board Certification Associated With Better Early Career Surgical Outcomes?

Accreditation Council for Graduate Medical Education Milestone Training Ratings and Surgeons' Early Outcomes

Importance

National data on the development of competence during training have been reported using the Accreditation Council for Graduate Medical Education (ACGME) Milestones system. It is now possible to consider longitudinal analyses that link Milestone ratings during training to patient outcomes data of recent graduates.

Objective

To evaluate the association of in-training ACGME Milestone ratings in a surgical specialty with subsequent complication rates following a commonly performed operation, endovascular aortic aneurysm repair (EVAR).

Design, Setting, and Participants

This study of patient outcomes followed EVAR in the Vascular Quality Initiative (VQI) registry (4213 admissions from 208 hospitals treated by 327 surgeons). All surgeons included in this study graduated from ACGME-accredited training programs from 2015 through 2019 and had Milestone ratings 6 months prior to graduation. Data were analyzed from December 1, 2021, through September 15, 2023. Because Milestone ratings can vary with program, they were corrected for program effect using a deviation score from the program mean.

Exposure

Milestone ratings assigned to individual trainees 6 months prior to graduation, based on judgments of surgical competence.

Main Outcomes and Measures

Surgical complications following EVAR for patients treated by recent graduates during the index hospitalization, obtained using the nationwide Society for Vascular Surgery Patient Safety Organization's VQI registry, which includes 929 participating centers in 49 US states.

Results

The study included outcomes for 4213 patients (mean [SD] age, 73.25 [8.74] years; 3379 male participants [80.2%]). Postoperative complications included 9.5% major (400 of 4213 cases) and 30.2% minor (1274 of 4213 cases) complications. After adjusting for patient risk factors and site of training, a significant association was identified between individual Milestone ratings of surgical trainees and major complications in early surgical practice in programs with lower mean Milestone ratings (odds ratio, 0.50; 95% CI; 0.27-0.95).

Conclusions and Relevance

In this study, Milestone assessments of surgical trainees were associated with subsequent clinical outcomes in their early career. Although these findings represent one surgical specialty, they suggest Milestone ratings can be used in any specialty to identify trainees at risk for future adverse patient outcomes when applying the same theory and methodology. Milestones data should inform data-driven educational interventions and trainee remediation to optimize future patient outcomes.

Association between the American Board of Emergency Medicine Oral Certifying Examination and Future State Medical Board Disciplinary Actions

Objectives

The American Board of Emergency Medicine (ABEM) requires a written examination (the Qualifying Examination) followed by the Oral Certifying Examination (OCE) to obtain ABEM certification. Maintaining ABEM certification is associated with fewer state medical board (SMB) disciplinary actions. We sought to determine the association between poor initial performance on the OCE and subsequent severe SMB disciplinary action.

Methods

We included physicians who completed US categorical emergency medicine residencies in 2016 and earlier. We classified OCE performance as good (passed on first attempt) and poor (never passed or required > 1 attempt to pass). We obtained data on physician SMB disciplinary actions from the National Practitioner Data Bank that were limited to actions that denied licensure or altered the status of a medical license (eg, suspension). We determined the association between poor OCE performance and subsequent severe SMB disciplinary action.

Results

Of 34,871, 93.5% passed the OCE on the first attempt, 6.1% required multiple attempts, and 0.3% never passed. Of the physicians (93.5%) with good OCE performance, 1.0% received a severe SMB action. Among physicians with poor OCE performance, 2.3% received a severe action; and of those who never passed, 1.7% received a severe action (Table 1). Poor OCE performance was associated with an increased odds of severe SMB disciplinary action (OR 2.21, 95% CI: 1.57-3.12).

Conclusion

Physicians with poor OCE performance exhibited higher odds of experiencing a subsequent severe SMB disciplinary action. The OCE may have utility as a predictor of future professionalism or clinical performance.

John A, Kavic SM. A scoping review of successful strategies for passing the American Board of Surgery certifying examination

Background

This scoping review identifies existing literature that investigates what factors contribute to success on the American Board of Surgery (ABS) Certifying Exam (CE) to provide practical, evidence-based recommendations.

Methods

A Pubmed search was completed utilizing the preferred reporting items for systematic reviews and meta-analysis extension for scoping review (PRISMA-ScR) method.

Results

Of 4368 articles identified, 45 articles met criteria for review. Manuscripts were placed into one of five categories: predictors from medical school, program interventions, modifiable candidate factors, the effect of mock oral exams, and those factors shown not to provide benefit for CE preparation.

Conclusions

A variety of factors have either been shown to provide benefit for or be predictive of CE performance. Acknowledgement of these factors can provide benefit to both surgery residents as well as surgery programs. Despite these findings, research into these factors is generally of low quality, prompting the need for ongoing, high-quality investigations.

The Association of ACGME Milestones With Performance on American Board of Surgery Assessments: A National Investigation of Surgical Trainees

OBJECTIVE

To determine the relationship between, and predictive utility of, milestone ratings and subsequent American Board of Surgery (ABS) vascular surgery in-training examination (VSITE), vascular qualifying examination (VQE), and vascular certifying examination (VCE) performance in a national cohort of vascular surgery trainees.

BACKGROUND

Specialty board certification is an important indicator of physician competence. However, predicting future board certification examination performance during training continues to be challenging.

METHODS

This is a national longitudinal cohort study examining relational and predictive associations between Accreditation Council for Graduate Medical Education (ACGME) Milestone ratings and performance on VSITE, VQE, and VCE for all vascular surgery trainees from 2015 to 2021. Predictive associations between milestone ratings and VSITE were conducted using cross-classified random-effects regression. Cross-classified random-effects logistic regression was used to identify predictive associations between milestone ratings and VQE and VCE.

RESULTS

Milestone ratings were obtained for all residents and fellows(n=1,118) from 164 programs during the study period (from July 2015 to June 2021), including 145,959 total trainee assessments. Medical knowledge (MK) and patient care (PC) milestone ratings were strongly predictive of VSITE performance across all postgraduate years (PGYs) of training, with MK ratings demonstrating a slightly stronger predictive association overall (MK coefficient 17.26 to 35.76, β = 0.15 to 0.23). All core competency ratings were predictive of VSITE performance in PGYs 4 and 5. PGY 5 MK was highly predictive of VQE performance [OR 4.73, (95% CI, 3.87-5.78), P <0.001]. PC subcompetencies were also highly predictive of VQE performance in the final year of training [OR 4.14, (95% CI, 3.17-5.41), P <0.001]. All other competencies were also significantly predictive of first-attempt VQE pass with ORs of 1.53 and higher. PGY 4 ICS ratings [OR 4.0, (95% CI, 3.06-5.21), P <0.001] emerged as the strongest predictor of VCE first-attempt pass. Again, all subcompetency ratings remained significant predictors of first-attempt pass on CE with ORs of 1.48 and higher.

CONCLUSIONS

ACGME Milestone ratings are highly predictive of future VSITE performance, and first-attempt pass achievement on VQE and VCE in a national cohort of surgical trainees.

Novel interactive text-messaging curriculum for endocrinology board review

Background

The ABIM certification exam is one of the measures to ensure that physicians have the clinical skills for good care delivery. The 5-year average pass rate for ABIM Endocrinology exam is 82%. The pass rate significantly decreased to a nadir of 74% in 2021 and 2022, lowest of all medicine subspecialties.

Objectives

To assess the feasibility of text messaging curriculum for fellows and its utility in improving their test performance.

Methods

In 2021, endocrinology fellows from 51 programs across the country were invited to participate in our curriculum. They completed a pre-test, joined a texting group via Remind application and received 1 multiple choice question daily (total n = 78). After 15 weeks, they completed a post-test and survey. Paired results from pre- and post-test were compared.

Results

A total of 89 fellows from 27 programs responded. Of these, 82 fellows, predominantly females (n = 60; 73 %), filled out the pre-test. On an average, 42 fellows (SD = 12) responded to the questions daily and 57 % of them answered the questions within 24 h. Thirty fellows completed the post-test. The median number of correct responses on the pre-test was 5 (IQR 3-6), compared to 8 (IQR 6-9) in the post-test. There was a significant improvement (p-value < 0.0001) in fellows' performance in the post-test when compared with the pre-test following our intervention.

Conclusions

Text-messaging based curriculum for exam preparation is feasible and can improve test performance. Fellows find receiving a daily high yield multiple choice question via text-message as a useful tool for exam preparation.

Relationship Between the American Board of Ophthalmology Maintenance of Certification Program and Actions Against the Medical License

PURPOSE

To evaluate the likelihood of disciplinary actions against medical licenses of ophthalmologists who maintained board certification through successful completion of the American Board of Ophthalmology Maintenance of Certification program compared with ophthalmologists who did not maintain certification.

METHODS

This was a retrospective cohort study of ophthalmologists certified by the American Board of Ophthalmology from 1992 to 2012 with time-limited certificates. Rates and severity of disciplinary actions against medical licenses were analyzed among ophthalmologists who did and did not maintain certification.

RESULTS

Of 9111 ophthalmologists who earned initial board certification between 1992 and 2012, 8073 (88.6%) maintained their certification and 1038 (11.4%) did not maintain their certification. A total of 234 license actions were identified in the study group. Among ophthalmologists who did not maintain board certification, the risk of a license action was more than 2 times that of those who maintained board certification (hazard ratio = 2.34, 95% CI=1.73-3.18). License actions were significantly higher in men than in women (hazard ratio = 2.02, 95% CI=1.43-2.86). Ophthalmologists who had a lapse in their certification had a higher severity of disciplinary actions (χ² = 9.21, p <.01) than ophthalmologists who maintained their certification.

CONCLUSIONS

This study supports prior literature in other specialties demonstrating a higher risk of disciplinary licensure actions in physicians who did not maintain board certification as compared with those who did. Physicians who did not maintain certification were also more likely to have actions against their license reflecting a higher severity violation. NOTE: Publication of this article is sponsored by the American Ophthalmological Society.

Volume- and quality-controlled certification system promotes centralization of complex hepato-pancreatic-biliary surgery

BACKGROUND

Centralization of complex surgeries has made little progress when it only considers the minimum number of surgical procedures. We aim to assess the impact of certification system of Japanese Society of Hepato-Biliary-Pancreatic Surgery (JSHBPS) on centralization and surgical quality of advanced hepato-pancreatic-biliary (HPB) surgery.

METHODS

The National Clinical Database was used to review 20 111 patients who underwent pancreatoduodenectomy (PD) and 9666 who underwent advanced hepatectomy defined as hepatectomy of more than one section during 2019 and 2020. JSHPBS certifies hospitals based on the annual number of advanced HPB surgeries and the surgical quality. Minimum numbers of surgeries for board-certified A and B institutions are 50 and 30, respectively. Short-term outcomes were compared among institutions.

RESULTS

In 2020, 69.4% (7007/10090) and 72.9% (3433/4710) of patients underwent PD and advanced hepatectomy at board-certified institutions. In-hospital mortality rates after PD was 0.9% at certified A institutions, 1.4% at B institutions, and 2.7% at non-certified institutions (p < .001). The odds ratio (OR) of risk-adjusted mortality after PD compared with non-certified institutions was 0.39 (confidence interval [CI]: 0.30-0.50, p < .001) at certified A institutions, and 0.54 at certified B institutions (CI: 0.40-0.73, p < .001). In-hospital mortality rates after advanced hepatectomy was 1.7% at certified A institutions, 2.3% at B institutions, and 3.2% at non-certified institutions (p < .001). The OR of risk-adjusted mortality after advanced hepatectomy compared with non-certified institutions was 0.57 at certified A institutions (CI: 0.41-0.78, p < .001).

CONCLUSION

The volume- and quality-controlled certification system of JSHBPS reduces surgical mortality after advanced HPB surgeries.

Novel method to link surgical trainee performance data to patient outcomes

BACKGROUND

A significant roadblock in surgical education research has been the inability to compare trainee performance to the outcomes of those surgeons after they enter independent practice. We describe the feasibility of an innovative method to link trainee performance data with patient outcomes.

METHODS

We extracted surgeon NPI numbers from Medicare claims data for common general surgery procedures between 2007 and 2017. Next, American Board of Surgery (ABS) trainee performance data was cross-referenced with additional resources to supplement NPI data. The patient and trainee datasets were linked using NPI number and a linkage rate was calculated.

RESULTS

We identified 12,952 unique surgeons in the Medicare file. Medicare surgeons were matched with ABS records by NPI number, with 96.2% (n = 12,460) of surgeons linked successfully.

CONCLUSIONS

We demonstrated a novel process to link patient outcomes to trainee performance. This innovation can enable future research investigating the relationship between surgical trainee performance and patient outcomes in independent practice.

Resident self-assessment of common endocrine procedures

BACKGROUND

General surgery residency graduates are expected to be proficient in straightforward endocrine operations. This study aimed to elucidate residents' self-assessment of their ability to perform common endocrine procedures.

METHODS

A fourteen-question survey was emailed to general surgery residents from seven U.S. residency programs regarding their self-assessed ability to perform each step of a straightforward thyroidectomy and parathyroidectomy. Demographics and perceived ability to perform the various procedures were collected.

RESULTS

A minority of respondents (17, 27.9%) agreed they could complete a straightforward thyroidectomy for benign disease, with only 11.7% (n = 7) agreeing they could complete a straightforward thyroidectomy for malignant disease. 26.2% (n = 16) of respondents agreed they could complete a straightforward parathyroidectomy. Completed number of cases was significantly associated with greater self-assessed ability to perform the endocrine operations (p = 0.02).

CONCLUSIONS

Most general surgery residents surveyed did not feel capable of performing common, straightforward endocrine procedures. Although confidence in operative ability increased with PGY-level and number of cases completed, the majority of PGY-5 residents still did not feel able to perform a thyroidectomy for malignant disease unassisted.

Why Competency Standardization Matters for Improvement: An Assessment of the Healthcare Quality Workforce

ABSTRACT

Healthcare transformation requires a healthcare quality workforce with the requisite expertise to lead, oversee, and implement positive change within healthcare organizations. The National Association for Healthcare Quality (NAHQ) Competency Framework, which spans 29 competencies across 8 domains of healthcare quality, outlines the specific knowledge and skills needed to advance personal and organizational quality goals. This study describes 1,671 responses to the NAHQ Professional Assessment survey from a diverse group of healthcare quality professionals representative of NAHQ's professional community. Results show that two-thirds of respondents indicated they are working in 4 or more competency domains, with 85% reporting working in Performance and Process Improvement. Results also indicate that individuals who hold the Certified Professional in Healthcare Quality (CPHQ) certification perform work at more advanced levels across all eight domains of the competency framework. This was statistically significant for 13 of the 29 competencies including all three competency statements within the Performance and Process Improvement domain. Healthcare organizations need a workforce with specialized quality and safety expertise to advance quality goals, and this study suggests that those who invest in continued professional development by attaining the CPHQ certification may be better positioned to contribute meaningfully to advance these goals and improve organizational outcomes.