The Association Between Maintaining American Board of Emergency Medicine Certification and State Medical Board Disciplinary Actions

Association between the American Board of Emergency Medicine Oral Certifying Examination and Future State Medical Board Disciplinary Actions

Objectives

The American Board of Emergency Medicine (ABEM) requires a written examination (the Qualifying Examination) followed by the Oral Certifying Examination (OCE) to obtain ABEM certification. Maintaining ABEM certification is associated with fewer state medical board (SMB) disciplinary actions. We sought to determine the association between poor initial performance on the OCE and subsequent severe SMB disciplinary action.

Methods

We included physicians who completed US categorical emergency medicine residencies in 2016 and earlier. We classified OCE performance as good (passed on first attempt) and poor (never passed or required > 1 attempt to pass). We obtained data on physician SMB disciplinary actions from the National Practitioner Data Bank that were limited to actions that denied licensure or altered the status of a medical license (eg, suspension). We determined the association between poor OCE performance and subsequent severe SMB disciplinary action.

Results

Of 34,871, 93.5% passed the OCE on the first attempt, 6.1% required multiple attempts, and 0.3% never passed. Of the physicians (93.5%) with good OCE performance, 1.0% received a severe SMB action. Among physicians with poor OCE performance, 2.3% received a severe action; and of those who never passed, 1.7% received a severe action (Table 1). Poor OCE performance was associated with an increased odds of severe SMB disciplinary action (OR 2.21, 95% CI: 1.57-3.12).

Conclusion

Physicians with poor OCE performance exhibited higher odds of experiencing a subsequent severe SMB disciplinary action. The OCE may have utility as a predictor of future professionalism or clinical performance.

Design, delivery and effectiveness of health practitioner regulation systems: an integrative review

BACKGROUND

Health practitioner regulation (HPR) systems are increasingly recognized as playing an important role in supporting health workforce availability, accessibility, quality, and sustainability, while promoting patient safety. This review aimed to identify evidence on the design, delivery and effectiveness of HPR to inform policy decisions.

METHODS

We conducted an integrative analysis of literature published between 2010 and 2021. Fourteen databases were systematically searched, with data extracted and synthesized based on a modified Donabedian framework.

FINDINGS

This large-scale review synthesized evidence from a range of academic (n = 410) and grey literature (n = 426) relevant to HPR. We identified key themes and findings for a series of HPR topics organized according to our structures-processes-outcomes conceptual framework. Governance reforms in HPR are shifting towards multi-profession regulators, enhanced accountability, and risk-based approaches; however, comparisons between HPR models were complicated by a lack of a standardized HPR typology. HPR can support government workforce strategies, despite persisting challenges in cross-border recognition of qualifications and portability of registration. Scope of practice reform adapted to modern health systems can improve access and quality. Alternatives to statutory registration for lower-risk health occupations can improve services and protect the public, while standardized evaluation frameworks can aid regulatory strengthening. Knowledge gaps remain around the outcomes and effectiveness of HPR processes, including continuing professional development models, national licensing examinations, accreditation of health practitioner education programs, mandatory reporting obligations, remediation programs, and statutory registration of traditional and complementary medicine practitioners.

CONCLUSION

We identified key themes, issues, and evidence gaps valuable for governments, regulators, and health system leaders. We also identified evidence base limitations that warrant caution when interpreting and generalizing the results across jurisdictions and professions. Themes and findings reflect interests and concerns in high-income Anglophone countries where most literature originated. Most studies were descriptive, resulting in a low certainty of evidence. To inform regulatory design and reform, research funders and governments should prioritize evidence on regulatory outcomes, including innovative approaches we identified in our review. Additionally, a systematic approach is needed to track and evaluate the impact of regulatory interventions and innovations on achieving health workforce and health systems goals.

Relationship Between the American Board of Ophthalmology Maintenance of Certification Program and Actions Against the Medical License

PURPOSE

To evaluate the likelihood of disciplinary actions against medical licenses of ophthalmologists who maintained board certification through successful completion of the American Board of Ophthalmology Maintenance of Certification program compared with ophthalmologists who did not maintain certification.

METHODS

This was a retrospective cohort study of ophthalmologists certified by the American Board of Ophthalmology from 1992 to 2012 with time-limited certificates. Rates and severity of disciplinary actions against medical licenses were analyzed among ophthalmologists who did and did not maintain certification.

RESULTS

Of 9111 ophthalmologists who earned initial board certification between 1992 and 2012, 8073 (88.6%) maintained their certification and 1038 (11.4%) did not maintain their certification. A total of 234 license actions were identified in the study group. Among ophthalmologists who did not maintain board certification, the risk of a license action was more than 2 times that of those who maintained board certification (hazard ratio = 2.34, 95% CI=1.73-3.18). License actions were significantly higher in men than in women (hazard ratio = 2.02, 95% CI=1.43-2.86). Ophthalmologists who had a lapse in their certification had a higher severity of disciplinary actions (χ² = 9.21, p <.01) than ophthalmologists who maintained their certification.

CONCLUSIONS

This study supports prior literature in other specialties demonstrating a higher risk of disciplinary licensure actions in physicians who did not maintain board certification as compared with those who did. Physicians who did not maintain certification were also more likely to have actions against their license reflecting a higher severity violation. NOTE: Publication of this article is sponsored by the American Ophthalmological Society.

An Examination of the Associations Among USMLE Step 3 Scores and the Likelihood of Disciplinary Action in Practice

PURPOSE

As the last examination in the United States Medical Licensing Examination (USMLE) sequence, Step 3 provides a safeguard before physicians enter into unsupervised practice. There is, however, little validity research focusing on Step 3 scores beyond examining its associations with other educational and professional assessments thought to cover similar content. This study examines the associations between Step 3 scores and subsequent receipt of disciplinary action taken by state medical boards for problematic behavior in practice. It analyzes Step 3 total, Step 3 computer-based case simulation (CCS), and Step 3 multiple-choice question (MCQ) scores.

METHOD

The final sample included 275,392 board-certified physicians who graduated from MD-granting medical schools and who passed Step 3 between 2000 and 2017. Cross-classified multilevel logistic regression models were used to examine the effects of Step 3 scores on the likelihood of receiving a disciplinary action, controlling for other USMLE scores and accounting for jurisdiction and specialty.

RESULTS

Results showed that physicians with higher Step 3 total, CCS, and MCQ scores tended to have lower chances of receiving a disciplinary action, after accounting for other USMLE scores. Specifically, a 1-standard-deviation increase in Step 3 total, CCS, and MCQ score was associated with a 23%, 11%, and 17% decrease in the odds of receiving a disciplinary action, respectively. The effect of Step 2 CK score on the likelihood of receiving a disciplinary action was statistically significant, while the effect of Step 1 score became statistically nonsignificant when other Step scores were included in the analysis.

CONCLUSIONS

Physicians who perform better on Step 3 are less likely to receive a disciplinary action from a state medical board for problematic behavior in practice. These findings provide some validity evidence for the use of Step 3 scores when making medical licensure decisions in the United States.

Associations between initial American Board of Internal Medicine certification and maintenance of certification status of attending physicians and in-hospital mortality of patients with acute myocardial infarction or congestive heart failure: a retrospective cohort study of hospitalisations in Pennsylvania, USA

OBJECTIVE

To determine whether internists' initial specialty certification and the maintenance of that certification (MOC) is associated with lower in-hospital mortality for their patients with acute myocardial infarction (AMI) or congestive heart failure (CHF).

DESIGN

Retrospective cohort study of hospitalisations in Pennsylvania, USA, from 2012 to 2017.

SETTING

All hospitals in Pennsylvania.

PARTICIPANTS

All 184 115 hospitalisations for primary diagnoses of AMI or CHF where the attending physician was a self-designated internist.

PRIMARY OUTCOME MEASURE

In-hospital mortality.

RESULTS

Of the 2575 physicians, 2238 had initial certification and 820 were eligible for MOC. After controlling for patient demographics and clinical characteristics, hospital-level factors and physicians' demographic and medical school characteristics, both initial certification and MOC were associated with lower mortality. The adjusted OR for initial certification was 0.835 (95% CI 0.756 to 0.922; p<0.001). Patients cared for by physicians with initial certification had a 15.87% decrease in mortality compared with those cared for by non-certified physicians (mortality rate difference of 5.09 per 1000 patients; 95% CI 2.12 to 8.05; p<0.001). The adjusted OR for MOC was 0.804 (95% CI 0.697 to 0.926; p=0.003). Patients cared for by physicians who completed MOC had an 18.91% decrease in mortality compared with those cared for by MOC lapsed physicians (mortality rate difference of 6.22 per 1000 patients; 95% CI 2.0 to 10.4; p=0.004).

CONCLUSIONS

Initial certification was associated with lower mortality for AMI or CHF. Moreover, for patients whose physicians had initial certification, an additional advantage was associated with its maintenance.

Doctors' attitudes to maintenance of professional competence: A scoping review

CONTEXT

Recent decades have seen the international implementation of programmes aimed at assuring the continuing competence of doctors. Maintenance of Professional Competence (MPC) programmes aim to encourage doctors' lifelong learning and ensure high-quality, safe patient care; however, programme requirements can be perceived as bureaucratic and irrelevant to practice, leading to disengagement. Doctors' attitudes and beliefs about MPC are critical to translating regulatory requirements into committed and effective lifelong learning. We aimed to summarise knowledge about doctors' attitudes to MPC to inform the development of MPC programmes and identify under-researched areas.

METHODS

We undertook a scoping review following Arksey and O'Malley, including sources of evidence about doctors' attitudes to MPC in the United States, the United Kingdom, Canada, Australia, New Zealand and Ireland, and using the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) as a guide.

RESULTS

One hundred and twenty-five sources of evidence were included in the review. One hundred and two were peer-reviewed publications, and 23 were reports. Most were from the United Kingdom or the United States and used quantitative or mixed methods. There was agreement across jurisdictions that MPC is a good idea in theory but doubt that it achieves its objectives in practice. Attitudes to the processes of MPC, and their impact on learning and practice were mixed. The lack of connection between MPC and practice was a recurrent theme. Barriers to participation were lack of time and resources, complexity of the requirements and a lack of flexibility in addressing doctors' personal and professional circumstances.

CONCLUSIONS

Overall, the picture that emerged is that doctors are supportive of the concept of MPC but have mixed views on its processes. We highlight implications for research and practice arising from these findings.

Multiple United States Medical Licensing Examination Attempts and the Estimated Risk of Disciplinary Actions Among Graduates of U.S. and Canadian Medical Schools

PURPOSE

The United States Medical Licensing Examination (USMLE) recently announced 2 policy changes: shifting from numeric score reporting on the Step 1 examination to pass/fail reporting and limiting examinees to 4 attempts for each Step component. In light of these policies, exam measures other than scores, such as the number of examination attempts, are of interest. Attempt limit policies are intended to ensure minimum standards of physician competency, yet little research has explored how Step attempts relate to physician practice outcomes. This study examined the relationship between USMLE attempts and the likelihood of receiving disciplinary actions from state medical boards.

METHOD

The sample population was 219,018 graduates from U.S. and Canadian MD-granting medical schools who passed all USMLE Step examinations by 2011 and obtained a medical license in the United States, using data from the NBME and the Federation of State Medical Boards. Logistic regressions estimated how attempts on Steps 1, 2 Clinical Knowledge (CK), and 3 examinations influenced the likelihood of receiving disciplinary actions by 2018, while accounting for physician characteristics.

RESULTS

A total of 3,399 physicians (2%) received at least 1 disciplinary action. Additional attempts needed to pass Steps 1, 2 CK, and 3 were associated with an increased likelihood of receiving disciplinary actions (odds ratio [OR]: 1.07, 95% confidence interval [CI]: 1.01, 1.13; OR: 1.09, 95% CI: 1.03, 1.16; OR: 1.11, 95% CI: 1.04, 1.17, respectively), after accounting for other factors.

CONCLUSIONS

Physicians who took multiple attempts to pass Steps 1, 2 CK, and 3 were associated with higher estimated likelihood of receiving disciplinary actions. This study offers support for licensure and practice standards to account for physicians' USMLE attempts. The relatively small effect sizes, however, caution policy makers from placing sole emphasis on this relationship.

Evaluation of a High Stakes Physician Competency Assessment: Lessons for Assessor Training, Program Accountability, and Continuous Improvement

INTRODUCTION

There is a dearth of evidence evaluating postlicensure high-stakes physician competency assessment programs. Our purpose was to contribute to this evidence by evaluating a high-stakes assessment for assessor inter-rater reliability and the relationship between performance on individual assessment components and overall performance. We did so to determine if the assessment tools identify specific competency needs of the assessed physicians and contribute to our understanding of physician dyscompetence more broadly.

METHOD

Four assessors independently reviewed 102 video-recorded assessments and scored physicians on seven assessment components and overall performance. Inter-rater reliability was measured using intraclass correlation coefficients using a multiple rater, consistency, two-way random effect model. Analysis of variance with least-significant difference post-hoc analyses examined if the mean component scores differed significantly by quartile ranges of overall performance. Linear regression analysis determined the extent to which each component score was associated with overall performance.

RESULTS

Intraclass correlation coefficients ranged between 0.756 and 0.876 for all components scored and was highest for overall performance. Regression indicated that individual component scores were positively associated with overall performance. Levels of variation in component scores were significantly different across quartile ranges with higher variability in poorer performers.

DISCUSSION

High-stake assessments can be conducted reliably and identify performance gaps of potentially dyscompetent physicians. Physicians who performed well tended to do so in all aspects evaluated, whereas those who performed poorly demonstrated areas of strength and weakness. Understanding that dyscompetence rarely means a complete or catastrophic lapse competence is vital to understanding how educational needs change through a physician's career.

A Blueprint for Productive Maintenance of Certification, But Is the American Board of Internal Medicine up to the Challenge?

The future of the American Board of Internal Medicine Maintenance of Certification (MOC) program is at a crossroads. The current MOC program lacks a clear visible mission, adds to modern health care’s onerous bureaucracy, and thus pulls physicians from the most important humanistic aspects of their profession. The aim of the MOC program should be to promote the best patient care by ensuring certified physicians maintain core skills through continuous education and evaluation. The program should focus on education and be designed with the rigorous obligations of practicing physicians in mind. Moving forward, the American Board of Internal Medicine should cocreate MOC with the physician community and apply innovative adult education techniques. Over time, data-driven methods and member feedback should be used to provide continuous program improvement. This review describes the origins of the current state of MOC, explores its evidence base, provides examples of model programs for the maintenance of complex professional skills, and outlines guiding principles for the future of MOC.