Certifying examination performance and patient outcomes following acute myocardial infarction

Reducing patients' rate of frequent attendance through a training intervention for physicians

BACKGROUND

Frequent attendance is a common issue for primary care health centres. The phenomenon affects the quality of care, increases doctors' workloads and can lead to burnout.This study presents the results of an educational intervention for primary care physicians, aimed at helping them to decrease the prevalence rate of excessive attendance by patients at their centres.

METHODS

A training programme was carried out for 11 primary care doctors in Barcelona who had patient lists totalling 20,064 patients. The goal of the training was to provide the participating physicians with techniques to curb frequent attendance. Additionally, the programme sought to offer them strategies to prevent professional burnout and tools to better organize their everyday medical practice. The study used a quasi-experimental design for an evaluation of an educational intervention, featuring a pre-test assessment (before the training programme) and a post-test assessment (after the training programme), as well as comparison with a control group that did not undergo the training. The study assessed the effects of the programme on the rates of frequent attendance of patients served by the participating physicians. These rates were compared with those registered by the patients seen by the control group physicians over the same period.

RESULTS

Among the group of physicians who received the training, the mean prevalence of patients who qualified as frequent attenders decreased from 22% prior to the training programme to 8% after completion of the programme. In other words, 14% of patients (2,809) limited the frequency of their visits to primary care physicians after their physicians had completed the training programme. Meanwhile, the study recorded an average decrease of 3.1 visits per year by the patients of the physicians who had undergone the training. Statistically significant differences between this group and the control group were observed.

CONCLUSIONS

The educational intervention proved effective at helping primary care physicians to decrease their patients' rates of frequent attendance. It also contributes to the impact research of continuing education on doctors and their patients. We need to increase primary care spending from the current 14% to the 25%, to address this problem, among others.

Residency program factors affecting ACVS board examination pass rates from 2017 to 2021: A survey of ACVS small animal surgery residency programs

OBJECTIVE

To identify small animal surgery (SAS) residency program factors associated with board examination pass rates for the American College of Veterinary Surgeons (ACVS) board certification examinations.

STUDY DESIGN

Online survey.

SAMPLE POPULATION

Program directors of ACVS-registered SAS residency programs.

METHODS

An online survey was distributed to eligible 2021 ACVS SAS residency program directors. Respondents provided program information and Phase I and Phase II board examination pass rate data from 2017 to 2021. Programs were grouped based on overall combined board examination pass rates as "excellent" (≥90%) versus "adequate" (<90%). Data were analyzed using recursive partitioning.

RESULTS

Responses from 36 ACVS SAS residency programs (18 private practice [PP], 18 academic programs [AP]) were included. Pass rates were considered "excellent" for 17/36 programs (11/17 AP, 6/17 PP). The strongest indicator of "excellent" programs was most mentors participating in rounds >75% of the time. Of those, the second indicator of "excellent" programs was a soft tissue caseload of <7 surgeries/resident/week. If the soft tissue caseload was >7 surgeries/resident/week, a high orthopedic caseload (>9 surgeries/resident/week) was indicative of "excellent" programs. If most mentors were not participating in rounds >75% of the time, APs were more likely to be "excellent."

CONCLUSION

SAS APs may have higher ACVS board examination pass rates. Mentor involvement, caseload, and residency practice type may influence board examination pass rates.

CLINICAL IMPACT

ACVS SAS residency programs' board examination pass rates may improve by optimizing mentor involvement and caseload regardless of practice type.

New evidence on the validity of the selection methods for recruitment to general practice training: a cohort study

BACKGROUND

Selection into UK-based GP training has used the Multi-Specialty Recruitment Assessment (MSRA) and a face-to-face selection centre (SC). The MSRA comprises of a situational judgement test and clinical problem-solving test. The SC was suspended during the COVID-19 pandemic. Evidence is needed to guide national and international selection policy.

AIM

To evaluate the validity of GP training selection.

DESIGN & SETTING

A retrospective cohort study using data from UK-based national recruitment to GP training, from 2015-2021.

METHOD

Data were available for 32 215 GP training applicants. The ability of scores from the specialty selection process to predict subsequent performance in the Clinical Skills Assessment (CSA) of the Membership of the Royal College of General Practitioners examination was modelled using path analysis. The effect sizes for sex, professional family background, and world region of qualification were estimated.

RESULTS

All component scores of the selection process demonstrated statistically significant independent relationships with CSA performance (P<0.001), thus establishing their predictive validity. All were sensitive to demographic factors. The SC scores had the weakest relationship with future CSA performance. However, for candidates with MSRA scores below the lowest quartile, the relative contribution of the SC scores to predicting CSA performance was similar to that observed for MSRA components.

CONCLUSION

The MSRA has predictive validity in this context. Re-instituting an SC for those with relatively low MSRA scores should be considered. However, the relative costs and potential advantages and disadvantages should be carefully weighed.

Accreditation Council for Graduate Medical Education Milestone Training Ratings and Surgeons' Early Outcomes

Importance

National data on the development of competence during training have been reported using the Accreditation Council for Graduate Medical Education (ACGME) Milestones system. It is now possible to consider longitudinal analyses that link Milestone ratings during training to patient outcomes data of recent graduates.

Objective

To evaluate the association of in-training ACGME Milestone ratings in a surgical specialty with subsequent complication rates following a commonly performed operation, endovascular aortic aneurysm repair (EVAR).

Design, Setting, and Participants

This study of patient outcomes followed EVAR in the Vascular Quality Initiative (VQI) registry (4213 admissions from 208 hospitals treated by 327 surgeons). All surgeons included in this study graduated from ACGME-accredited training programs from 2015 through 2019 and had Milestone ratings 6 months prior to graduation. Data were analyzed from December 1, 2021, through September 15, 2023. Because Milestone ratings can vary with program, they were corrected for program effect using a deviation score from the program mean.

Exposure

Milestone ratings assigned to individual trainees 6 months prior to graduation, based on judgments of surgical competence.

Main Outcomes and Measures

Surgical complications following EVAR for patients treated by recent graduates during the index hospitalization, obtained using the nationwide Society for Vascular Surgery Patient Safety Organization's VQI registry, which includes 929 participating centers in 49 US states.

Results

The study included outcomes for 4213 patients (mean [SD] age, 73.25 [8.74] years; 3379 male participants [80.2%]). Postoperative complications included 9.5% major (400 of 4213 cases) and 30.2% minor (1274 of 4213 cases) complications. After adjusting for patient risk factors and site of training, a significant association was identified between individual Milestone ratings of surgical trainees and major complications in early surgical practice in programs with lower mean Milestone ratings (odds ratio, 0.50; 95% CI; 0.27-0.95).

Conclusions and Relevance

In this study, Milestone assessments of surgical trainees were associated with subsequent clinical outcomes in their early career. Although these findings represent one surgical specialty, they suggest Milestone ratings can be used in any specialty to identify trainees at risk for future adverse patient outcomes when applying the same theory and methodology. Milestones data should inform data-driven educational interventions and trainee remediation to optimize future patient outcomes.

Associations between initial American Board of Internal Medicine certification and maintenance of certification status of attending physicians and in-hospital mortality of patients with acute myocardial infarction or congestive heart failure: a retrospective cohort study of hospitalisations in Pennsylvania, USA

OBJECTIVE

To determine whether internists' initial specialty certification and the maintenance of that certification (MOC) is associated with lower in-hospital mortality for their patients with acute myocardial infarction (AMI) or congestive heart failure (CHF).

DESIGN

Retrospective cohort study of hospitalisations in Pennsylvania, USA, from 2012 to 2017.

SETTING

All hospitals in Pennsylvania.

PARTICIPANTS

All 184 115 hospitalisations for primary diagnoses of AMI or CHF where the attending physician was a self-designated internist.

PRIMARY OUTCOME MEASURE

In-hospital mortality.

RESULTS

Of the 2575 physicians, 2238 had initial certification and 820 were eligible for MOC. After controlling for patient demographics and clinical characteristics, hospital-level factors and physicians' demographic and medical school characteristics, both initial certification and MOC were associated with lower mortality. The adjusted OR for initial certification was 0.835 (95% CI 0.756 to 0.922; p<0.001). Patients cared for by physicians with initial certification had a 15.87% decrease in mortality compared with those cared for by non-certified physicians (mortality rate difference of 5.09 per 1000 patients; 95% CI 2.12 to 8.05; p<0.001). The adjusted OR for MOC was 0.804 (95% CI 0.697 to 0.926; p=0.003). Patients cared for by physicians who completed MOC had an 18.91% decrease in mortality compared with those cared for by MOC lapsed physicians (mortality rate difference of 6.22 per 1000 patients; 95% CI 2.0 to 10.4; p=0.004).

CONCLUSIONS

Initial certification was associated with lower mortality for AMI or CHF. Moreover, for patients whose physicians had initial certification, an additional advantage was associated with its maintenance.

The critical role of direct observation in entrustment decisions

BACKGROUND

Entrustment decisions may be retrospective (based on past experiences with a trainee) or real-time (based on direct observation). We investigated judgments of entrustment based on assessor prior knowledge of candidates and based on systematic direct observation, conducted in an objective structured clinical exam (OSCE).

METHODS

Sixteen faculty examiners provided 287 retrospective and real-time entrustment ratings of 16 cardiology trainees during OSCE stations in 2019 and 2020. Reliability and validity of these ratings were assessed by comparing correlations across stations as a measure of reliability, differences across postgraduate years as an index of construct validity, correlation to standardized in-training exam (ITE) as a measure of criterion validity, and reclassification of entrustment as a measure of consequential validity.

RESULTS

Both retrospective and real-time assessments were highly reliable (all intra-class correlations >0.86). Both increased with a year of postgraduate training. Real-time entrustment ratings were significantly correlated with standardized ITE scores; retrospective ratings were not. Real-time ratings explained 37% (2019) and 46% (2020) of variance in examination scores vs. 21% (2019) and 7% (2020) for retrospective ratings. Direct observation resulted in a different level of entrustment compared with retrospective ratings in 44% of cases (p = <0.001).

CONCLUSIONS

Ratings based on direct observation made unique contributions to entrustment decisions.

Coverage of obesity and obesity disparities on American Board of Medical Specialties (ABMS) examinations

Obesity is a widespread disease which adversely impacts all organ systems and disproportionately affects African Americans and other minority groups. Physicians across medical specialties must possess current knowledge of obesity as an important, distinct disease with biological and social causes. Coverage of obesity on board certification examinations, which influence standards in medical knowledge and practice in each specialty, has not previously been examined. The member boards of the American Board of Medical Specialties offer a content outline or "blueprint" detailing material tested. We parsed the 24 available general certification exam blueprints for mentions of obesity and related keywords. We categorized blueprints into three tiers: mention of obesity (Tier 1), mention of related terminology but not obesity (Tier 2), and no mention of obesity or related terminology (Tier 3). We analyzed mentions of obesity and related terms by blueprint word count and procedural versus non-procedural specialties. Six (25.0%) of 24 board exam blueprints mentioned obesity (Tier 1), fifteen (62.5%) mentioned related terminology only (Tier 2), and three (12.5%) mentioned neither obesity nor related terminology (Tier 3). There was no significant difference in obesity-related mentions between procedural and non-procedural specialties (X2, p = .50). None of the blueprints included racial/ethnic disparities related to obesity. Word count was not significantly correlated with mentions of obesity in linear regression (p = .42). The absence of any mention of obesity on most content outlines and of racial/ethnic disparities on all content outlines indicates need for increased coverage of the diagnosis, prevention, and treatment of obesity across all board examinations.

Curriculum Transition From Lecture-Based to Team-Based Learning is Associated With Improved Performance on Internal Medicine In-Training Examination

BACKGROUND

Team-based learning (TBL) is an alternative to traditional lectures in graduate medical education, but evidence is scarce regarding its impact on knowledge acquisition and standardized testing performance.

OBJECTIVE

We examined the association between resident performance on the Internal Medicine In-Training Examination (IM-ITE) and these 2 educational methods.

METHODS

In 2013, the internal medicine residency program at Albany Medical College transitioned from a lecture-based curriculum to TBL. Residents enrolled in academic years 2011-2012 and 2012-2013 comprised the lecture cohort, and those enrolled in 2015-2016 and 2016-2017 the TBL cohort. Covariates included the type of medical school attended, gender, and United States Medical Licensing Examination Step 2 Clinical Knowledge scores. We performed univariate analysis and multivariable regression to determine the association between covariates and ITE scores.

RESULTS

Of 120 residents, 60 were in the lecture cohort and 60 in the TBL cohort. The IM-ITE percent correct scores were higher with TBL than lecture (PGY-1 61.0% vs 55.0%, P < .001; PGY-2 69.0% vs 59.7%, P < .001; PGY-3 73.2% vs 61.7%, P < .001). In a multivariable regression analysis of 3 PGYs combined, the transition from lecture to TBL resulted in an increase in IM-ITE Z-score of 0.415 (P < .001), equivalent to 0.415 SD, when including the effects of all covariates.

CONCLUSIONS

Compared to a lecture-based curriculum, TBL was associated with improved resident medical knowledge acquisition as evidenced by higher IM-ITE scores.

Specific Disease Knowledge as Predictor of Susceptibility to Availability Bias in Diagnostic Reasoning: a Randomized Controlled Experiment

BACKGROUND

Bias in reasoning rather than knowledge gaps has been identified as the origin of most diagnostic errors. However, the role of knowledge in counteracting bias is unclear.

OBJECTIVE

To examine whether knowledge of discriminating features (findings that discriminate between look-alike diseases) predicts susceptibility to bias.

DESIGN

Three-phase randomized experiment. Phase 1 (bias-inducing): Participants were exposed to a set of clinical cases (either hepatitis-IBD or AMI-encephalopathy). Phase 2 (diagnosis): All participants diagnosed the same cases; 4 resembled hepatitis-IBD, 4 AMI-encephalopathy (but all with different diagnoses). Availability bias was expected in the 4 cases similar to those encountered in phase 1. Phase 3 (knowledge evaluation): For each disease, participants decided (max. 2 s) which of 24 findings was associated with the disease. Accuracy of decisions on discriminating features, taken as a measure of knowledge, was expected to predict susceptibility to bias.

PARTICIPANTS

Internal medicine residents at Erasmus MC, Netherlands.

MAIN MEASURES

The frequency with which higher-knowledge and lower-knowledge physicians gave biased diagnoses based on phase 1 exposure (range 0-4). Time to diagnose was also measured.

KEY RESULTS

Sixty-two physicians participated. Higher-knowledge physicians yielded to availability bias less often than lower-knowledge physicians (0.35 vs 0.97; p = 0.001; difference, 0.62 [95% CI, 0.28-0.95]). Whereas lower-knowledge physicians tended to make more of these errors on subjected-to-bias than on not-subjected-to-bias cases (p = 0.06; difference, 0.35 [CI, - 0.02-0.73]), higher-knowledge physicians resisted the bias (p = 0.28). Both groups spent more time to diagnose subjected-to-bias than not-subjected-to-bias cases (p = 0.04), without differences between groups.

CONCLUSIONS

Knowledge of features that discriminate between look-alike diseases reduced susceptibility to bias in a simulated setting. Reflecting further may be required to overcome bias, but succeeding depends on having the appropriate knowledge. Future research should examine whether the findings apply to real practice and to more experienced physicians.

Which older adults receive sleep medicine specialty care? Predictors of being seen by a board-certified sleep medicine provider

STUDY OBJECTIVES

The aim of this study was to characterize older adult Medicare beneficiaries seen by board-certified sleep medicine providers (BCSMPs) and identify predictors of being seen by a BCSMP.

METHODS

Our data source was a random 5% sample of Medicare administrative claims data (2006-2013). BCSMPs were identified using a cross-matching procedure based on national provider identifiers available within the Medicare database and assigned based on the first sleep disorder diagnosis received. Sleep disorders (insomnia, sleep-related breathing disorders, hypersomnias, circadian rhythm sleep-wake disorders, parasomnias, and restless legs syndrome) were operationalized as International Classification of Disease, Ninth Revision, Clinical Modification diagnostic codes. The number of sleep disorders per beneficiary was computed and compared between BCSMPs and nonspecialists. Logistic regression was used to identify medical and demographic predictors of being seen by a BCSMP.

RESULTS

A total of 57,209 beneficiaries received one or more sleep disorder diagnoses during the study period. Of these, 1,279 (2.2%) were initially diagnosed by a BCSMP. Relative to individuals seen by nonspecialists, beneficiaries treated by a BCSMP were more likely to have two or more sleep disorders (9.0% vs 24.1%, P < .001). The most common diagnosis assigned by BCSMPs was obstructive sleep apnea (70.4% of patients seen by BCSMPs were diagnosed with obstructive sleep apnea). The most common diagnosis assigned by nonspecialists was insomnia (48.2% of patients seen by nonspecialists were diagnosed with insomnia). In a fully adjusted regression model, male sex (odds ratio [OR] 1.53; 95% confidence interval [CI] 1.36, 1.72), asthma (OR 1.50; 95% CI 1.30, 1.73), and heart failure (OR 1.24; 95% CI 1.10, 1.41) were positively associated with being treated by a BCSMP. Conversely, depression (OR 0.85, 95% CI 0.73, 1.00), anxiety (OR 0.69, 95% CI .59, .82), Alzheimer and related dementias (OR 0.80, 95% CI .65, .99), and anemia (OR .88, 95% CI .78, .99) were associated with a reduced likelihood of being seen by a BCSMP.

CONCLUSIONS

Relative to older adults seen by nonspecialists, those seen by BCSMPs are more medically but less psychiatrically complex and are diagnosed with a greater number of sleep disorders. These results suggest the possibility that medically complex patients are referred for specialty care, whereas psychiatrically complex patients might be seen at the nonspecialist level. Further, these results demonstrate the value of board certification in sleep medicine in caring for complex sleep patients.

Correlations Between the USMLE Step Examinations, American College of Physicians In-Training Examination, and ABIM Internal Medicine Certification Examination

PURPOSE

To assess the correlations between United States Medical Licensing Examination (USMLE) performance, American College of Physicians Internal Medicine In-Training Examination (IM-ITE) performance, American Board of Internal Medicine Internal Medicine Certification Exam (IM-CE) performance, and other medical knowledge and demographic variables.

METHOD

The study included 9,676 postgraduate year (PGY)-1, 11,424 PGY-2, and 10,239 PGY-3 internal medicine (IM) residents from any Accreditation Council for Graduate Medical Education-accredited IM residency program who took the IM-ITE (2014 or 2015) and the IM-CE (2015-2018). USMLE scores, IM-ITE percent correct scores, and IM-CE scores were analyzed using multiple linear regression, and IM-CE pass/fail status was analyzed using multiple logistic regression, controlling for USMLE Step 1, Step 2 Clinical Knowledge, and Step 3 scores; averaged medical knowledge milestones; age at IM-ITE; gender; and medical school location (United States or Canada vs international).

RESULTS

All variables were significant predictors of passing the IM-CE with IM-ITE scores having the strongest association and USMLE Step scores being the next strongest predictors. Prediction curves for the probability of passing the IM-CE based solely on IM-ITE score for each PGY show that residents must score higher on the IM-ITE with each subsequent administration to maintain the same estimated probability of passing the IM-CE.

CONCLUSIONS

The findings from this study should support residents and program directors in their efforts to more precisely identify and evaluate knowledge gaps for both personal learning and program improvement. While no individual USMLE Step score was as strongly predictive of IM-CE score as IM-ITE score, the combined relative contribution of all 3 USMLE Step scores was of a magnitude similar to that of IM-ITE score.

The medical decision-making process in the time of the coronavirus pandemic

The disease pandemic caused by the novel coronavirus has triggered significant changes in the medical decision-making process relating to critically ill patients. Admissions to intensive care units have suddenly increased, but many of these patients do not present with clinical manifestations related to the viral infection but rather exacerbation of preexisting diseases. In this context, we must prevent intuitive decision-making and insecurity from leading us to exhaust the available critical-care beds before they are truly necessary, while still recognizing the importance of rapid decision-making in emergency situations. One of the best ways to achieve this goal may be by practicing metacognition and establishing ways for regular feedback to be provided to professionals engaged in inherently rapid decision-making processes.

Association Between Participation and Performance in MOCA Minute and Actions Against the Medical Licenses of Anesthesiologists

BACKGROUND

In January 2016, as part of the Maintenance of Certification in Anesthesiology (MOCA) program, the American Board of Anesthesiology launched MOCA Minute, a web-based longitudinal assessment, to supplant the former cognitive examination. We investigated the association between participation and performance in MOCA Minute and disciplinary actions against medical licenses of anesthesiologists.

METHODS

All anesthesiologists with time-limited certificates (ie, certified in 2000 or after) who were required to register for MOCA Minute in 2016 were followed up through December 31, 2016. The incidence of postcertification prejudicial license actions was compared between those who did and did not register and compared between registrants who did and did not meet the MOCA Minute performance standard.

RESULTS

The cumulative incidence of license actions was 1.2% (245/20,006) in anesthesiologists required to register for MOCA Minute. Nonregistration was associated with a higher incidence of license actions (hazard ratio, 2.93 [95% confidence interval {CI}, 2.15-4.00]). For the 18,534 (92.6%) who registered, later registration (after June 30, 2016) was associated with a higher incidence of license actions. In 2016, 16,308 (88.0%) anesthesiologists met the MOCA Minute performance standard. Of those not meeting the standard (n = 2226), most (n = 2093, 94.0%) failed because they did not complete the required 120 questions. Not meeting the standard was associated with a higher incidence of license actions (hazard ratio, 1.92 [95% CI, 1.36-2.72]).

CONCLUSIONS

Both timely participation and meeting performance standard in MOCA Minute are associated with a lower likelihood of being disciplined by a state medical board.

Practice patterns of board-certified sleep medicine providers: a national analysis among older adult Medicare beneficiaries

STUDY OBJECTIVES

To examine the proportion of Medicare beneficiaries with sleep disorders who were evaluated by board-certified sleep medicine providers (BCSMPs).

METHODS

Using a random 5% sample of Medicare administrative claims data (2007-2011), BCSMPs were identified by employing a novel cross-matching approach based on National Provider Identifiers available within the Medicare database. Sleep disorders were included based partially on the International Classification of Sleep Disorders, Third Edition (insomnia, sleep-related breathing disorders, hypersomnias, circadian rhythm sleep-wake disorders, parasomnias, and restless legs syndrome), and operationalized as International Classification of Diseases, Ninth Revision, Clinical Modification diagnostic codes. The proportion of beneficiaries with each disorder who were seen by BCSMPs and nonspecialists was computed.

RESULTS

Among older adult Medicare beneficiaries with sleep disorders, the most common sleep disorder was insomnia (n = 65,033), and the least common sleep disorder was narcolepsy (n = 784). Individuals with central sleep apnea (n = 1,561) were most likely to be treated by a BCSMP (63.9% of beneficiaries with central sleep apnea), and individuals diagnosed with insomnia were least likely to be treated by a BCSMP (16.4% of beneficiaries with insomnia). Most BCSMPs treated beneficiaries with obstructive sleep apnea (84.9% of BCSMPs) and insomnia (75.8% of BCSMPs).

CONCLUSIONS

BCSMPs are involved in the care of a substantial proportion of Medicare beneficiaries with sleep disorders.

Opinions of general and adult congenital heart disease cardiologists on care for adults with congenital heart disease in Belgium: a qualitative study

BACKGROUND

The growing adult congenital heart disease (CHD) population requires efficient healthcare organisation. It has been suggested that clinically appropriate care be provided for individual patients on the least complex level possible, in order to alleviate saturation of special care programmes.

METHODS

Semi-structured interviews with 10 general and 10 adult CHD cardiologists were conducted to elucidate opinions on healthcare organisation in Belgium. A particular focus was placed on the potential role of general cardiologists. The software program NVivo 12 facilitated thematic analysis.

RESULTS

A discrepancy existed between how general cardiologists thought about congenital care and what adult CHD cardiologists considered the minimum knowledge required to adequately treat patients. Qualitative data were categorised under the following themes: knowledge dissemination, certification, (de)centralisation of care, the role of adult CHD cardiologists, the role of dedicated nurse specialists, and patient referral. It appeared to be pivotal to organise care in such a way that providing basic care locally does not impede the generation of sufficient patient volume, and to continue improving communications between different care levels when there is no referral back. Moreover, practical knowledge is best disseminated locally. Cardiologists' opinions on certification and on the role of dedicated nurse specialists were mixed.

CONCLUSION

On the basis of the results, we propose five recommendations for improving the provision of care to adults with CHD. A multidimensional approach to defining the role of different healthcare professionals, to improving communication channels, and to effectively sensitising healthcare professionals is needed to improve the organisation of care.

Association between Performance in a Maintenance of Certification Program and Disciplinary Actions against the Medical Licenses of Anesthesiologists

WHAT WE ALREADY KNOW ABOUT THIS TOPIC

WHAT THIS ARTICLE TELLS US THAT IS NEW: BACKGROUND:: In 2000, the American Board of Anesthesiology (Raleigh, North Carolina) began issuing time-limited certificates requiring renewal every 10 yr through a maintenance of certification program. This study investigated the association between performance in this program and disciplinary actions against medical licenses.

METHODS

The incidence of postcertification prejudicial license actions was compared (1) between anesthesiologists certified between 1994 and 1999 (non-time-limited certificates not requiring maintenance of certification) and those certified between 2000 and 2005 (time-limited certificates requiring maintenance of certification); (2) within the non-time-limited cohort, between those who did and did not voluntarily participate in maintenance of certification; and (3) within the time-limited cohort, between those who did and did not complete maintenance of certification requirements within 10 yr.

RESULTS

The cumulative incidence of license actions was 3.8% (587 of 15,486). The incidence did not significantly differ after time-limited certificates were introduced (hazard ratio = 1.15; 95% CI, 0.95 to 1.39; for non-time-limited cohort compared with time-limited cohort). In the non-time-limited cohort, 10% (n = 953) voluntarily participated in maintenance of certification. Maintenance of certification participation was associated with a lower incidence of license actions (hazard ratio = 0.60; 95% CI, 0.38 to 0.94). In the time-limited cohort, 90% (n = 5,329) completed maintenance of certification requirements within 10 yr of certificate issuance. Not completing maintenance of certification requirements (n = 588) was associated with a higher incidence of license actions (hazard ratio = 4.61; 95% CI, 3.27 to 6.51).

CONCLUSIONS

These findings suggest that meeting maintenance of certification requirements is associated with a lower likelihood of being disciplined by a state licensing agency. The introduction of time-limited certificates in 2000 was not associated with a significant change in the rate of license actions.

What we measure … and what we should measure in medical education

CONTEXT

As the practice of medicine evolves, the knowledge, skills and attitudes required to provide patient care will continue to change. These competency-based changes will necessitate the restructuring of assessment systems. High-quality assessment programmes are needed to fulfil health professions education's contract with society.

OBJECTIVES

We discuss several issues that are important to consider when developing assessments in health professions education. We organise the discussion along the continuum of medical education, outlining the tension between what has been deemed important to measure and what should be measured. We also attempt to alleviate some of the apprehension associated with measuring evolving competencies by discussing how emerging technologies, including simulation and artificial intelligence, can play a role.

METHODS

We focus our thoughts on the assessment of competencies that, at least historically, have been difficult to measure. We highlight several assessment challenges, discuss some of the important issues concerning the validity of assessment scores, and argue that medical educators must do a better job of justifying their use of specific assessment strategies.

DISCUSSION

As in most professions, there are clear tensions in medicine in relation to what should be assessed, who should be responsible for administering assessment content, and how much evidence should be gathered to support the evaluation process. Although there have been advances in assessment practices, there is still room for improvement. From the student's, resident's and practising physician's perspectives, assessments need to be relevant. Knowledge is certainly required, but there are other qualities and attributes that are important, and perhaps far more important. Research efforts spent now on delineating what makes a good physician, and on aligning new and upcoming assessment tools with the relevant competencies, will ensure that assessment practices, whether aimed at establishing competence or at fostering learning, are effective with respect to their primary goal: to produce qualified physicians.

Decentralizing the delivery of HIV pre-exposure prophylaxis (PrEP) through family physicians and sexual health clinic nurses: a dissemination and implementation study protocol

BACKGROUND

Gay, bisexual and other men who have sex with men (gbMSM) in Canada continue to experience high rates of incident HIV. Pre-exposure prophylaxis (PrEP, the regular use of anti-HIV medication) reduces HIV acquisition and could reduce incidence. However, there are too few physicians with expertise in HIV care to meet the projected demand for PrEP. To meet demand and achieve greater public health impact, PrEP delivery could be 'decentralized' by incorporating it into front-line prevention services provided by family physicians (FPs) and sexual health clinic nurses.

METHODS

This PrEP decentralization project will use two strategies. The first is an innovative knowledge dissemination approach called 'Patient-Initiated CME' (PICME), which aims to empower individuals to connect their family doctors with online, evidence-based, continuing medical education (CME) on PrEP. After learning about the project through community agencies or social/sexual networking applications, gbMSM interested in PrEP will use a uniquely coded card to access an online information module that includes coaching on how to discuss their HIV risk with their FP. They can provide their physician a link to the accredited CME module using the same card. The second strategy involves a pilot implementation program, in which gbMSM who do not have a FP may bring the card to designated sexual health clinics where trained nurses can deliver PrEP under a medical directive. These approaches will be evaluated through quantitative and qualitative methods, including: questionnaires administered to patients and physicians at baseline and at six months; focus groups with patients, FPs, and sexual health clinic staff; and review of sexual health clinic charts. The primary objective is to quantify the uptake of PrEP achieved using each decentralization strategy. Secondary objectives include a) characterizing barriers and facilitators to PrEP uptake for each strategy, b) assessing fidelity to core components of PrEP delivery within each strategy, c) measuring patient-reported outcomes including satisfaction with clinician-patient relationships, and d) conducting a preliminary costing analysis.

DISCUSSION

This study will assess the feasibility of a novel strategy for disseminating knowledge about evidence-based clinical interventions, and inform future strategies for scale-up of an underutilized HIV prevention tool.

Effectiveness of Written and Oral Specialty Certification Examinations to Predict Actions against the Medical Licenses of Anesthesiologists

BACKGROUND

The American Board of Anesthesiology administers written and oral examinations for its primary certification. This retrospective cohort study tested the hypothesis that the risk of a disciplinary action against a physician's medical license is lower in those who pass both examinations than those who pass only the written examination.

METHODS

Physicians who entered anesthesiology training from 1971 to 2011 were followed up to 2014. License actions were ascertained via the Disciplinary Action Notification Service of the Federation of State Medical Boards.

RESULTS

The incidence rate of license actions was relatively stable over the study period, with approximately 2 to 3 new cases per 1,000 person-years. In multivariable models, the risk of license actions was higher in men (hazard ratio = 1.88 [95% CI, 1.66 to 2.13]) and lower in international medical graduates (hazard ratio = 0.73 [95% CI, 0.66 to 0.81]). Compared with those passing both examinations on the first attempt, those passing neither examination (hazard ratio = 3.60 [95% CI, 3.14 to 4.13]) and those passing only the written examination (hazard ratio = 3.51 [95% CI, 2.87 to 4.29]) had an increased risk of receiving an action from a state medical board. The risk was no different between the latter two groups (P = 0.81), showing that passing the oral but not the written primary certification examination is associated with a decreased risk of subsequent license actions. For those with residency performance information available, having at least one unsatisfactory training record independently increased the risk of license actions.

CONCLUSIONS

These findings support the concept that an oral examination assesses domains important to physician performance that are not fully captured in a written examination.

A preliminary investigation to explore the cognitive resources of physicians experiencing difficulty in training

BACKGROUND

Treating patients is complex, and research shows that there are differences in cognitive resources between physicians who experience difficulties, and those who do not. It is possible that differences in some cognitive resources could explain the difficulties faced by some physicians. In this study, we explore differences in cognitive resources between different groups of physicians (that is, between native (UK) physicians and International Medical Graduates (IMG); those who continue with training versus those who were subsequently removed from the training programme); and also between physicians experiencing difficulties compared with the general population.

METHODS

A secondary evaluation was conducted on an anonymised dataset provided by the East Midlands Professional Support Unit (PSU). One hundred and twenty one postgraduate trainee physicians took part in an Educational Psychology assessment through PSU. Referrals to the PSU were mainly on the basis of problems with exam progression and difficulties in communication skills, organisation and confidence. Cognitive resources were assessed using the Wechsler Adult Intelligence Scale (WAIS-IV). Physicians were categorised into three PSU outcomes: 'Continued in training', 'Removed from training' and 'Active' (currently accessing the PSU).

RESULTS

Using a one-sample Z test, we compared the referred physician sample to a UK general population sample on the WAIS-IV and found the referred sample significantly higher in Verbal Comprehension (VCI; z = 8.78) and significantly lower in Working Memory (WMI; z = -4.59). In addition, the native sample were significantly higher in Verbal Comprehension than the UK general population sample (VCI; native physicians: z = 9.95, p < .001, d = 1.25), whilst there was a lesser effect for the difference between the IMG sample and the UK general population (z = 2.13, p = .03, d = 0.29). Findings also showed a significant difference in VCI scores between those physicians who were 'Removed from training' and those who 'Continued in training'.

CONCLUSIONS

Our results suggest it is important to understand the cognitive resources of physicians to provide a more focussed explanation of those who experience difficulties in training. This will help to implement more targeted interventions to help physicians develop compensatory strategies.

Internal medicine board certification and career pathways in Japan

BACKGROUND

Establishing and managing a board certification system is a common concern for many countries. In Japan, the board certification system is under revision. The purpose of this study was to describe present status of internal medicine specialist board certification, to identify factors associated with maintenance of board certification and to investigate changes in area of practice when physicians move from hospital to clinic practice.

METHODS

We analyzed 2010 and 2012 data from the Survey of Physicians, Dentists and Pharmacists. We conducted logistic regression analysis to identify factors associated with the maintenance of board certification between 2010 and 2012. We also analyzed data on career transition from hospitals to clinics for hospital physicians with board certification.

RESULTS

It was common for physicians seeking board certification to do so in their early career. The odds of maintaining board certification were lower in women and those working in locations other than academic hospitals, and higher in physicians with subspecialty practice areas. Among hospital physicians with board certification who moved to clinics between 2010 and 2012, 95.8% remained in internal medicine or its subspecialty areas and 87.7% maintained board certification but changed their practice from a subspecialty area to more general internal medicine.

CONCLUSION

Revisions of the internal medicine board certification system must consider different physician career pathways including mid-career moves while maintaining certification quality. This will help to secure an adequate number and distribution of specialists. To meet the increasing demand for generalist physicians, it is important to design programs to train specialists in general practice.

The Causes of Errors in Clinical Reasoning: Cognitive Biases, Knowledge Deficits, and Dual Process Thinking

Contemporary theories of clinical reasoning espouse a dual processing model, which consists of a rapid, intuitive component (Type 1) and a slower, logical and analytical component (Type 2). Although the general consensus is that this dual processing model is a valid representation of clinical reasoning, the causes of diagnostic errors remain unclear. Cognitive theories about human memory propose that such errors may arise from both Type 1 and Type 2 reasoning. Errors in Type 1 reasoning may be a consequence of the associative nature of memory, which can lead to cognitive biases. However, the literature indicates that, with increasing expertise (and knowledge), the likelihood of errors decreases. Errors in Type 2 reasoning may result from the limited capacity of working memory, which constrains computational processes. In this article, the authors review the medical literature to answer two substantial questions that arise from this work: (1) To what extent do diagnostic errors originate in Type 1 (intuitive) processes versus in Type 2 (analytical) processes? (2) To what extent are errors a consequence of cognitive biases versus a consequence of knowledge deficits?The literature suggests that both Type 1 and Type 2 processes contribute to errors. Although it is possible to experimentally induce cognitive biases, particularly availability bias, the extent to which these biases actually contribute to diagnostic errors is not well established. Educational strategies directed at the recognition of biases are ineffective in reducing errors; conversely, strategies focused on the reorganization of knowledge to reduce errors have small but consistent benefits.

The ABMS MOC Part III Examination: Value, Concerns, and Alternative Formats

This article describes the presentations and discussions at a conference co-convened by the Council on Medical Education of the American Medical Association (AMA) and by the American Board of Medical Specialties (ABMS). The conference focused on the ABMS Maintenance of Certification (MOC) Part III Examination. This article, reflecting the conference agenda, covers the value of and evidence supporting the examination, as well as concerns about the cost of the examination, and-given the current format-its relevance. In addition, the article outlines alternative formats for the examination that four ABMS member boards are currently developing or implementing. Lastly, the article presents contrasting views on the approach to professional self-regulation. One view operationalizes MOC as a high-stakes, pass-fail process while the other perspective holds MOC as an organized approach to support continuing professional development and improvement. The authors hope to begin a conversation among the AMA, the ABMS, and other professional stakeholders about how knowledge assessment in MOC might align with the MOC program's educational and quality improvement elements and best meet the future needs of both the public and the physician community.

Provider Board Certification Status and Practice Patterns in Total Knee Arthroplasty

PURPOSE

The presumption that board certification directly affects the quality of clinical care is a topic of ongoing discussion in medical literature. Recent studies have demonstrated disparities in patient outcomes associated with type of anesthesia provided for total knee arthroplasty (TKA); improved outcomes are associated with neuraxial (or regional) versus general anesthesia. Whether board-certified (BC) and non-board-certified (nBC) anesthesiologists make different choices in the anesthetic they administer is unknown. The authors sought to study potential associations of board certification status with anesthesia practice patterns for TKA.

METHOD

The authors accessed records of anesthetics provided from 2010 to 2013 from the National Anesthesia Clinical Outcomes Registry database. They identified TKA cases using Clinical Classifications Software and Current Procedural Terminology codes. The authors divided practitioners into two groups: those who were BC and those who were nBC. For each of these groups, the authors compared the following: their patient populations, the hospitals in which they worked, the nature of their practices, and the anesthetics they administered to their patients.

RESULTS

BC anesthesiologists provided care for 81.7% of 97,508 patients having TKA; 18.3% were treated by nBC anesthesiologists. BC anesthesiologists administered neuraxial/regional anesthesia more frequently than nBC anesthesiologists (41.4% versus 21.2%; P < .001).

CONCLUSIONS

The rates at which regional/neuraxial anesthesia were administered for TKA were relatively low, and there were significant differences in practice patterns of BC and nBC anesthesiologists providing care for patients undergoing TKA. More research is necessary to understand the causes of these disparities.

Association of Physician Certification in Interventional Cardiology With In-Hospital Outcomes of Percutaneous Coronary Intervention

BACKGROUND

The value of American Board of Internal Medicine certification has been questioned. We evaluated the Association of Interventional Cardiology certification with in-hospital outcomes of patients undergoing percutaneous coronary intervention (PCI) in 2010.

METHODS AND RESULTS

We identified physicians who performed ≥10 PCIs in 2010 in the CathPCI Registry and determined interventional cardiology (ICARD) certification status using American Board of Internal Medicine data. We compared in-hospital outcomes of patients treated by certified and noncertified physicians using hierarchical multivariable models adjusted for differences in patient characteristics and PCI volume. Primary end points were all-cause in-hospital mortality and bleeding complications. Secondary end points included emergency coronary artery bypass grafting, vascular complications, and a composite of any adverse outcome. With 510,708 PCI procedures performed by 5175 physicians, case mix and unadjusted outcomes were similar among certified and noncertified physicians. The adjusted risks of in-hospital mortality (odds ratio, 1.10; 95% confidence interval, 1.02-1.19) and emergency coronary artery bypass grafting (odds ratio, 1.32; 95% confidence interval, 1.12-1.56) were higher in the non-ICARD-certified group, but the risks of bleeding and vascular complications and the composite end point were not statistically significantly different between groups.

CONCLUSIONS

We did not observe a consistent association between ICARD certification and the outcomes of PCI procedures. Although there was a significantly higher risk of mortality and emergency coronary artery bypass grafting in patients treated by non-ICARD-certified physicians, the risks of vascular complications and bleeding were similar. Our findings suggest that ICARD certification status alone is not a strong predictor of patient outcomes and indicate a need to enhance the value of subspecialty certification.

Constructing a more comprehensive validity argument for medical school admission testing: predicting long-term outcomes

ISSUE

The research published outside of medical education journals provides an important source of validity evidence for using cognitive ability testing in medical school admissions.

EVIDENCE

The cumulative body of validity research, consisting of thousands of studies and scores of meta-analyses, has conclusively demonstrated that a strong positive relationship exists between job performance and general mental ability.

IMPLICATIONS

Recommendations for reducing the emphasis on or eliminating the role of general mental ability in the selection process for medical schools are not based on a consideration of the wider research evidence. Admission interventions that substantially reduce the level of academic aptitude are also likely to result in reduced professional performance.

Improving the quality and safety of care on the medical ward: A review and synthesis of the evidence base

Despite its place at the heart of inpatient medicine, the evidence base underpinning the effective delivery of medical ward care is highly fragmented. Clinicians familiar with the selection of evidence-supported treatments for specific diseases may be less aware of the evolving literature surrounding the organisation of care on the medical ward. This review is the first synthesis of that disparate literature. An iterative search identified relevant publications, using terms pertaining to medical ward environments, and objective and subjective patient outcomes. Articles (including reviews) were selected on the basis of their focus on medical wards, and their relevance to the quality and safety of ward-based care. Responses to medical ward failings are grouped into five common themes: staffing levels and team composition; interdisciplinary communication and collaboration; standardisation of care; early recognition and treatment of the deteriorating patient; and local safety climate. Interventions in these categories are likely to improve the quality and safety of care in medical wards, although the evidence supporting them is constrained by methodological limitations and inadequate investment in multicentre trials. Nonetheless, with infrequent opportunities to redefine their services, institutions are increasingly adopting multifaceted strategies that encompass groups of these themes. As the literature on the quality of inpatient care moves beyond its initial focus on the intensive care unit and operating theatre, physicians should be mindful of opportunities to incorporate evidence-based practice at a ward level.

The association between residency training and internists' ability to practice conservatively

IMPORTANCE

Growing concern about rising costs and potential harms of medical care has stimulated interest in assessing physicians' ability to minimize the provision of unnecessary care.

OBJECTIVE

To assess whether graduates of residency programs characterized by low-intensity practice patterns are more capable of managing patients' care conservatively, when appropriate, and whether graduates of these programs are less capable of providing appropriately aggressive care.

DESIGN, SETTING, AND PARTICIPANTS

Cross-sectional comparison of 6639 first-time takers of the 2007 American Board of Internal Medicine certifying examination, aggregated by residency program (n = 357).

EXPOSURES

Intensity of practice, measured using the End-of-Life Visit Index, which is the mean number of physician visits within the last 6 months of life among Medicare beneficiaries 65 years and older in the residency program's hospital referral region.

MAIN OUTCOMES AND MEASURES

The mean score by program on the Appropriately Conservative Management (ACM) (and Appropriately Aggressive Management [AAM]) subscales, comprising all American Board of Internal Medicine certifying examination questions for which the correct response represented the least (or most, respectively) aggressive management strategy. Mean scores on the remainder of the examination were used to stratify programs into 4 knowledge tiers. Data were analyzed by linear regression of ACM (or AAM) scores on the End-of-Life Visit Index, stratified by knowledge tier.

RESULTS

Within each knowledge tier, the lower the intensity of health care practice in the hospital referral region, the better residency program graduates scored on the ACM subscale (P < .001 for the linear trend in each tier). In knowledge tier 4 (poorest), for example, graduates of programs in the lowest-intensity regions had a mean ACM score in the 38th percentile compared with the 22nd percentile for programs in the highest-intensity regions; in tier 2, ACM scores ranged from the 75th to the 48th percentile in regions from lowest to highest intensity. Graduates of programs in low-intensity regions tended, more weakly, to score better on the AAM subscale (in 3 of 4 knowledge tiers).

CONCLUSIONS AND RELEVANCE

Regardless of overall medical knowledge, internists trained at programs in hospital referral regions with lower-intensity medical practice are more likely to recognize when conservative management is appropriate. These internists remain capable of choosing an aggressive approach when indicated.

Relationships between the Comprehensive Osteopathic Medical Achievement Test (COMAT) subject examinations and the COMLEX-USA Level 2-Cognitive Evaluation

CONTEXT

The relationship between the Comprehensive Osteopathic Medical Achievement Test (COMAT) series of subject examinations and the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Cognitive Evaluation (COMLEX-USA Level 2-CE) has not been thoroughly examined.

OBJECTIVE

To investigate the factors associated with performance on COMAT subject examinations and how COMAT scores correlate with COMLEX-USA Level 2-CE scores.

METHODS

We examined scores of participants from 2 COMAT examination cycles in 2011 and 2012. According to surveys, most schools used COMAT scores in clerkship and clinical rotation evaluation, which were classified as being used for "high-stakes" purposes. We matched first-attempt COMAT scores with first-attempt COMLEX-USA Level 2-CE scores, and we conducted correlation analyses between the scores from the 7 COMAT subject examinations, as well as between COMAT and COMLEX-USA Level 2-CE scores. Multiple linear regression analyses were performed to investigate how much variance in COMLEX-USA Level 2-CE scores was explained by COMAT scores.

RESULTS

In 2011 and 2012, respectively, 3751 and 3786 COMAT candidates had COMLEX-USA Level 2-CE scores (53.0% and 93.9%, respectively, had ⩾1 high-stakes COMAT score). Intercorrelations between COMAT scores were low to moderate (r=0.27-0.53), as hypothesized. Correlations between COMAT and Level 2-CE scores were moderate to high, with the highest correlations for internal medicine COMAT scores (r=0.63-0.65). All regressions showed internal medicine scores as the strongest predictor of Level 2-CE performance. Groups with high-stakes scores had larger adjusted coefficients of determination than those with low-stakes scores (eg, R(2)=0.63 vs 0.52, respectively, in 2011). For 2012 candidates with high-stakes scores, all predictors were statistically significant.

CONCLUSION

The COMAT subject examination scores were moderately intercorrelated, as hypothesized, with higher correlations between COMAT and COMLEX-USA Level 2-CE scores. The COMAT performance was predictive of COMLEX-USA Level 2-CE performance.

The relationship between licensing examination performance and the outcomes of care by international medical school graduates

PURPOSE

The Step 2 Clinical Knowledge (CK) examination of the United States Medical Licensing Examination sequence is a requirement for the certification of international medical graduates (IMGs) by the Educational Commission for Foreign Medical Graduates. An association between scores on the test and the quality of care later provided by those who take it is central to its use in certification and licensure. The purpose of this study was to determine whether there is a relationship between scores on Step 2 CK and patient outcomes for IMGs.

METHOD

This is a retrospective observational study of the 60,958 hospitalizations from 2003 to 2009 in Pennsylvania where the principal diagnosis was acute myocardial infarction or congestive heart failure and the attending physician (N = 2,525) was an IMG who had taken the Step 2 CK. The main measures were the three-digit scores on Step 2 CK and in-hospital mortality.

RESULTS

After adjustment for severity of illness, physician characteristics, and hospital characteristics, performance on Step 2 CK had a statistically significant inverse relationship with mortality. Each additional point on the examination was associated with a 0.2% (95% CI: 0.1%-0.4%) decrease in mortality. The size of the effect is noteworthy, with each standard deviation (roughly 20 points) equivalent to a 4% change in mortality risk.

CONCLUSIONS

These findings provide evidence for the validity of Step 2 CK scores. Given the magnitude of its relationship with patient outcomes, the results support the use of the examination as an effective screening strategy for licensure.

[Proposal for Recertification of Medical Specialties in Colombia]

This article gives a brief overview of the aspects that justify implementing a recertification of doctors. After a description of this process in the countries where it has the most experience, the advantages of having a recertification process and the criticism of the system is also discussed. Finally, the creation of the Council on Accreditation and Recertification Colombian Medical Specialist professionals (CAMEC, in Spanish), as a product of the work of the Colombian Association of Scientific Societies, and also the draft decree of the national government is proposed in the creation of the Integrated System for Continuing Education and Continuing Professional Development in Health (SFCTHS, in Spanish).

Outcomes of cardiac surgery: associations with physician characteristics, institutional characteristics, and transfers of care

BACKGROUND

Although there are several studies of the human and system factors that influence the outcomes of cardiac surgery, it remains difficult to draw conclusions because many do not simultaneously adjust for the characteristics of patients, physicians, and institutions. The current study explores the associations between these factors and inhospital mortality, with a particular focus on whether patients had the same operating and attending physician.

METHOD AND RESULTS

This is a retrospective observational study of 114,751 hospitalizations from 2003 to 2009 in Pennsylvania that included a coronary artery bypass graft, valve surgery, or both. The study included 70 teaching and nonteaching hospitals, 289 operating physicians who were also the attending physicians for 75% of the hospitalizations, and 2654 attending physicians for the remaining hospitalizations. After adjustment, there was a 38.4% decrease (95% CI, 20.3%-56.5%) in mortality when the operating and attending physician were the same. For the operator, each procedure performed was associated with a 0.05% (95% CI, 0.04%-0.06%) decrease in mortality and each year since medical school was associated with a 0.9% (95% CI, 0.02%-1.8%) increase in mortality. For the attending, each year since medical school was associated with a 0.67% (95% CI, 0.01%-1.4%) decrease in patient mortality.

CONCLUSIONS

The findings indicated that an increase in the log odds of mortality was associated with the transfer of care between an attending and operating physician. Better patient outcomes were associated with an operator with higher volume who was closer to medical school graduation and an attending physician with more clinical experience.

Assessing clinical reasoning: moving from in vitro to in vivo

The last century saw dramatic changes in clinical practice and medical education and the concomitant rise in high-stakes, psychometrically-based examinations of medical knowledge. Higher scores on these high-stakes “in-vitro” examinations are modestly associated with better performance in clinical practice and provide a meaningful degree of assurance to the public about physicians’ competency in medical knowledge. However, results on such examinations explain only a small fraction of the wide variation currently seen in clinical practice and diagnostic errors remain a serious and vexing problem for patients and the healthcare system despite decades of high-stakes examinations. In this commentary we explore some of the limitations of high-stakes examinations in assessing clinical reasoning and propose utilizing situated cognition theory to guide research and development of innovative modes of ”in-vivo” assessments that can be used in longitudinally and continuously in clinical practice.

Pediatrician qualifications and asthma management behaviors and their association with patient race/ethnicity

OBJECTIVE

We sought to understand if pediatrician characteristics and asthma assessment and treatment varied in association with the proportion of African-American and Latino children in the pediatrician's practice.

METHODS

We conducted a cross-sectional survey of 500 American Academy of Pediatrics members between November 2005 and May 2006. Standardized vignettes were used to test how different indicators of a patient's asthma status affect pediatrician asthma assessments and recommendations. Linear and logistic regression models were used to examine the association of pediatrician assessments and treatment recommendations for these vignettes, respectively, with the proportion of reported African-American and Latino children seen in their practice.

RESULTS

There were 270 respondents (response rate = 54%). Based on pediatrician-reported percentage of minority patients, there were no differences in board certification status, recognition of poorly controlled asthma nor in the likelihood of appropriately increasing long-term controller medications to treat poorly controlled asthma (p > 0.05 for all analyses).

CONCLUSIONS

Caring primarily for minority children by AAP pediatricians appears unrelated to training qualifications or in their reported knowledge of how to appropriately assess and treat asthma. Therefore, studies of asthma care disparities should focus on understanding the knowledge-base of non-AAP pediatric providers who care for minority populations and exploring other potential contributory provider-level factors (e.g. communication skills).

The effectiveness of continuing medical education for specialist recertification

Evolving professional, social and political pressures highlight the importance of lifelong learning for clinicians. Continuing medical education (CME) facilitates lifelong learning and is a fundamental factor in the maintenance of certification. The type of CME differs between surgical and non-surgical specialties. CME methods of teaching include lectures, workshops, conferences and simulation training. Interventions involving several modalities, instructional techniques and multiple exposures are more effective. The beneficial effects of CME can be maintained in the long term and can improve clinical outcome. However, quantitative evidence on validity, reliability, efficacy and cost-effectiveness of various methods is lacking. This is especially evident in urology. The effectiveness of CME interventions on maintenance of certification is also unknown. Currently, many specialists fulfil mandatory CME credit requirements opportunistically, therefore erroneously equating number of hours accumulated with competence. New CME interventions must emphasize actual performance and should correlate with clinical outcomes. Improved CME practice must in turn lead to continuing critical reflection, practice modification and implementation with a focus towards excellent patient care.

Changes in hospitals' credentialing requirements for board certification from 2005 to 2010

OBJECTIVE

In 2005, we conducted a study of the prevalence of board certification requirements for hospital privileging and found that one-third of hospitals did not require pediatricians to be board certified. In 2010, the American Board of Pediatrics implemented the Maintenance of Certification (MOC) program. To examine changes in the policies of hospitals regarding requirements for board certification, we surveyed privileging personnel at hospitals across the country.

STUDY DESIGN

Telephone survey between April 2010 and June 2010 of privileging personnel at a random sample of 220 hospitals.

RESULTS

Of the 220 hospitals, 23 were ineligible because they had no pediatricians on staff, and 26 hospitals refused to participate. The remaining 154 hospitals completed the survey, resulting in a 78% participation rate. Compared with our findings in 2005, in 2010 a greater proportion of hospitals now require board certification for general pediatricians (80% vs 67%, P = 0.141) and pediatric subspecialists (86% vs 71%, P = 0.048). Among these hospitals, a larger proportion (24% vs 4%) now requires board certification for all pediatricians at the point of initial privileging. However, a greater proportion of hospitals reported that they make exceptions to their board certification policies (99% vs 41%).

CONCLUSION

In the 5 years since our previous study, a larger proportion of hospitals now require pediatricians to be board certified, although the proportion of hospitals that make exceptions to this policy has increased twofold. Hospitals appear to be incorporating the MOC program into their privileging policies.

Cognitive Aspect of Diagnostic Errors

Diagnostic errors can result in tangible harm to patients. Despite our advances in medicine, the mental processes required to make a diagnosis exhibits shortcomings, causing diagnostic errors. Cognitive factors are found to be an important cause of diagnostic errors. With new understanding from psychology and social sciences, clinical medicine is now beginning to appreciate that our clinical reasoning can take the form of analytical reasoning or heuristics. Different factors like cognitive biases and affective influences can also impel unwary clinicians to make diagnostic errors. Various strategies have been proposed to reduce the effect of cognitive biases and affective influences when clinicians make diagnoses; however evidence for the efficacy of these methods is still sparse. This paper aims to introduce the reader to the cognitive aspect of diagnostic errors, in the hope that clinicians can use this knowledge to improve diagnostic accuracy and patient outcomes. Keywords: Affective influence, Analytical, Diagnostic errors, Heuristics, Reflective practice

Specialty board certification in the United States: issues and evidence

BACKGROUND

The American Board of Medical Specialties (ABMS) certification and maintenance of certification (MOC) programs strive to provide the public with guidance about a physician's competence. This study summarizes the literature on the effectiveness of these programs.

METHOD

A literature search was conducted for studies published between 1986 and April 2013 and limited to ABMS certification. A modified version of Kirkpatrick's 4 levels of program evaluation included the reaction of stakeholders to certification, the extent to which physicians are encouraged to improve, the relationship between performance in the programs and nonclinical external measures of physician competence, and the relationship of performance in the programs with clinical quality measures.

RESULTS

Patients' and hospitals' value of board certification and physician participation in MOC are high. Physicians are conflicted as to whether the effort involved is worth its value. Self-reported evidence shows improvement in knowledge, practice infrastructure, communication with patients and peers, and clinical care. Certification performance is generally related to nonclinical external measures such as types of training, practice characteristics, demographics, and disciplinary actions. In general, physicians who are board certified provide better patient care, albeit the results have modest effect sizes and are not unequivocal.

CONCLUSIONS

Certification boards should continuously try to improve their programs in response to feedback from stakeholders, changes in the way physicians practice, as well as the growth in the fields of measurement and technology. Keeping pace with these changes in a responsible and evidence-based way is important.

American Board of Medical Specialties Maintenance of Certification: theory and evidence regarding the current framework

The American Board of Medical Specialties Maintenance of Certification Program (ABMS MOC) is designed to provide a comprehensive approach to physician lifelong learning, self-assessment, and quality improvement (QI) through its 4-part framework and coverage of the 6 competencies previously adopted by the ABMS and the Accreditation Council for Graduate Medical Education (ACGME). In this article, the theoretical rationale and exemplary empiric data regarding the MOC program and its individual parts are reviewed. The value of each part is considered in relation to 4 criteria about the relationship of the competencies addressed within that part to (1) patient outcomes, (2) physician performance, (3) validity of the assessment or educational methods utilized, and (4) learning or improvement potential. Overall, a sound theoretical rationale and a respectable evidence base exists to support the current structure and elements of the MOC program. However, it is incumbent on the ABMS and ABMS member boards to continue to examine their programs moving forward to assure the public and the profession that they are meeting expectations, are clinically relevant, and provide value to patients and participating physicians, and to refine and improve them as ongoing research indicates.

Assessing medical knowledge of emergency medicine residents

The Accreditation Council for Graduate Medical Education (ACGME) requires that emergency medicine (EM) residency graduates are competent in the medical knowledge (MK) core competency. EM educators use a number of tools to measure a resident's progress toward this goal; it is not always clear whether these tools provide a valid assessment. A workshop was convened during the 2012 Academic Emergency Medicine consensus conference "Education Research in Emergency Medicine: Opportunities, Challenges, and Strategies for Success" where assessment for each core competency was discussed in detail. This article provides a description of the validity evidence behind current MK assessment tools used in EM and other specialties. Tools in widespread use are discussed, as well as emerging methods that may form valid assessments in the future. Finally, an agenda for future research is proposed to help address gaps in the current understanding of MK assessment.

Do early career indicators of clinical skill predict subsequent career outcomes and practice characteristics for general internists?

OBJECTIVE

To study relationships between clinical skill measures assessed at the beginning of general internists' careers and their career outcomes and practice characteristics.

DATA SOURCES

General Internist Community Tracking Study Physician Survey respondents (2000-2001, 2004-2005) linked with residency program evaluations and American Board of Internal Medicine board certification examination score records; n = 2,331.

STUDY DESIGN

Cross-sectional regressions of career outcome and practice characteristic measures on board examination scores/success, residency evaluations interacted with residency type, and potential confounding variables.

PRINCIPAL FINDINGS

Failure to achieve board certification was associated with $27,206 (18 percent, p < .05) less income and 14.9 percent more minority patients relative to physicians scoring in the bottom quartile on their initial examination who eventually became certified (p < .01). Other skill measures were not associated with income. Scoring in the top rather than bottom quartile on the board certification examination was associated with 9 percent increased likelihood of reporting high career satisfaction (p < .05). Among physicians trained in community hospital residency programs, lower evaluations were associated with 14.5 percent higher share of minority patients (p < .05). Both skill measures were associated with practice type.

CONCLUSIONS

There are associations between early career skill measures and career outcomes. In addition, minority patients are more likely to be treated by physicians with lower early career clinical skills measures than nonminority patients.

Relationship of electronic medical knowledge resource use and practice characteristics with Internal Medicine Maintenance of Certification Examination scores

BACKGROUND

Maintenance of certification examination performance is associated with quality of care. We aimed to examine relationships between electronic medical knowledge resource use, practice characteristics and examination scores among physicians recertifying in internal medicine.

METHODS

We conducted a cross-sectional study of 3,958 United States physicians who took the Internal Medicine Maintenance of Certification Examination (IM-MOCE) between January 1, 2006 and December 31, 2008, and who held individual licenses to one or both of two large electronic knowledge resource programs. We examined associations between physicians' IM-MOCE scores and their days of electronic resource use, practice type (private practice, residency teaching clinic, inpatient, nursing home), practice model (single or multi-specialty), sex, age, and medical school location.

RESULTS

In the 365 days prior to the IM-MOCE, physicians used electronic resources on a mean (SD, range) of 20.3 (36.5, 0-265) days. In multivariate analyses, the number of days of resource use was independently associated with increased IM-MOCE scores (0.07-point increase per day of use, p = 0.02). Increased age was associated with decreased IM-MOCE scores (1.8-point decrease per year of age, p < 0.001). Relative to physicians working in private practice settings, physicians working in residency teaching clinics and hospital inpatient practices had higher IM-MOCE scores by 29.1 and 20.0 points, respectively (both p < 0.001).

CONCLUSIONS

Frequent use of electronic resources was associated with modestly enhanced IM-MOCE performance. Physicians involved in residency education clinics and hospital inpatient practices had higher IM-MOCE scores than physicians working in private practice settings.

Evaluating the quality of care provided by graduates of international medical schools

One-quarter of practicing physicians in the United States are graduates of international medical schools. The quality of care provided by doctors educated abroad has been the subject of ongoing concern. Our analysis of 244,153 hospitalizations in Pennsylvania found that patients of doctors who graduated from international medical schools and were not U.S. citizens at the time they entered medical school had significantly lower mortality rates than patients cared for by doctors who graduated from U.S. medical schools or who were U.S. citizens and received their degrees abroad. The patient population consisted of those with congestive heart failure or acute myocardial infarction. We found no significant mortality difference when comparing all international medical graduates with all U.S. medical school graduates.