Pediatric board review course for residents "at risk"

Taking Leave During Residency: Types of Absences and Subsequent Delays and Variations in Physical Medicine and Rehabilitation Medical Board Pass Rates

ABSTRACT

There is limited literature examining the reasons physical medicine and rehabilitation residents take an extended leave of absence during residency and the impact of leave on board examination performance. Such information could better inform leave policies, help guide residency program directors, and potentially destigmatize taking leave. Study objectives were to describe the characteristics of physical medicine and rehabilitation residents who take leave during residency, compare differences in part I (written) and part II (oral) certification examination performance, and determine the prevalence of delays in taking board examinations. Study methodology was a retrospective analysis of deidentified information from the American Board of Physical Medicine and Rehabilitation database between 2008 and 2020. Results indicated four reasons for extended leave of absence: medical, parental, academic/remediation, and unspecified personal reasons. Residents who took an extended leave of absence for medical or parental reasons had similar or better odds of passing their part I and part II examinations compared with those without leave. Residents who took leave for academic/remediation concerns or unspecified personal reasons had lower odds of passing their board examinations. Examination delays for those taking parental or personal health leaves did not affect board pass rates. Further investigation is needed to identify how to support residents on academic/remediation and unspecified leaves during training.

Post graduate remediation programs in medicine: a scoping review

BACKGROUND

Recognizing that physicians may struggle to achieve knowledge, skills, attitudes and or conduct at one or more stages during their training has highlighted the importance of the 'deliberate practice of improving performance through practising beyond one's comfort level under guidance'. However, variations in physician, program, contextual and healthcare and educational systems complicate efforts to create a consistent approach to remediation. Balancing the inevitable disparities in approaches and settings with the need for continuity and effective oversight of the remediation process, as well as the context and population specific nature of remediation, this review will scrutinise the remediation of physicians in training to better guide the design, structuring and oversight of new remediation programs.

METHODS

Krishna's Systematic Evidence Based Approach is adopted to guide this Systematic Scoping Review (SSR in SEBA) to enhance the transparency and reproducibility of this review. A structured search for articles on remediation programs for licenced physicians who have completed their pre-registration postings and who are in training positions published between 1st January 1990 and 31st December 2021 in PubMed, Scopus, ERIC, Google Scholar, PsycINFO, ASSIA, HMIC, DARE and Web of Science databases was carried out. The included articles were concurrently thematically and content analysed using SEBA's Split Approach. Similarities in the identified themes and categories were combined in the Jigsaw Perspective and compared with the tabulated summaries of included articles in the Funnelling Process to create the domains that will guide discussions.

RESULTS

The research team retrieved 5512 abstracts, reviewed 304 full-text articles and included 101 articles. The domains identified were characteristics, indications, frameworks, domains, enablers and barriers and unique features of remediation in licenced physicians in training programs.

CONCLUSION

Building upon our findings and guided by Hauer et al. approach to remediation and Taylor and Hamdy's Multi-theories Model, we proffer a theoretically grounded 7-stage evidence-based remediation framework to enhance understanding of remediation in licenced physicians in training programs. We believe this framework can guide program design and reframe remediation's role as an integral part of training programs and a source of support and professional, academic, research, interprofessional and personal development.

Relationship Between Standardized Test Scores and Board Certification Exams in a Combined Internal Medicine/Pediatrics Residency Program

Background

Combined Internal Medicine/Pediatrics (Med/Peds) residencies rely on categorical program data to predict pass rates for the American Board of Internal Medicine Certifying Exam (ABIM-CE) and the American Board of Pediatrics Certifying Exam (ABP-CE). There is insufficient literature describing what best predicts a Med/Peds resident passing board exams. In this study, we aimed to determine how standardized test scores predict performance on ABIM-CE and ABP-CE for Med/Peds residents.

Methodology

We analyzed prior exam scores for 91/96 (95%) residents in a Med/Peds program from 2008 to 2017. Scores from the United States Medical Licensing Examination (USMLE) Steps 1 and 2 Clinical Knowledge (CK) and In-Training Exams in Internal Medicine (ITE-IM) and Pediatrics (ITE-P) were analyzed with the corresponding ABIM-CE and ABP-CE first-time scores. Linear and logistic regression were applied to predict board scores/passage.

Results

USMLE 1 and 2 CK, ITE-IM, and ITE-P scores had a linear relationship with both ABIM-CE and ABP-CE scores. In the linear regression, adjusted R² values showed low-to-moderate predictive ability (R² = 0.11-0.35), with the highest predictor of ABIM-CE and ABP-CE being USMLE Step 1 (0.35) and Postgraduate Year 1 (PGY-1) ITE-IM (0.33), respectively. Logistic regression showed odds ratios of passing board certifications ranging from 1.05 to 1.53 per point increase on the prior exam score. The PGY-3 ITE-IM was the best predictor of passing both certifying exams.

Conclusions

In one Med/Peds program, USMLE Steps 1 and 2 and all ITE-IM and ITE-P scores predicted certifying exam scores and passage. This provides Med/Peds-specific data to allow individualized resident counseling and guide programmatic improvements targeted to board performance.

A Systematic Review of the Relationship Between In-Training Examination Scores and Specialty Board Examination Scores

BACKGROUND

In-training examinations (ITEs) are intended for low-stakes, formative assessment of residents' knowledge, but are increasingly used for high-stake purposes, such as to predict board examination failures.

OBJECTIVE

The aim of this review was to investigate the relationship between performance on ITEs and board examination performance across medical specialties.

METHODS

A search of the literature for studies assessing the strength of the relationship between ITE and board examination performance from January 2000 to March 2019 was completed. Results were categorized based on the type of statistical analysis used to determine the relationship between ITE performance and board examination performance.

RESULTS

Of 1407 articles initially identified, 89 articles underwent full-text review, and 32 articles were included in this review. There was a moderate-strong relationship between ITE and board examination performance, and ITE scores significantly predict board examination scores for the majority of studies. Performing well on an ITE predicts a passing outcome for the board examination, but there is less evidence that performing poorly on an ITE will result in failing the associated specialty board examination.

CONCLUSIONS

There is a moderate to strong correlation between ITE performance and subsequent performance on board examinations. That the predictive value for passing the board examination is stronger than the predictive value for failing calls into question the "common wisdom" that ITE scores can be used to identify "at risk" residents. The graduate medical education community should continue to exercise caution and restraint in using ITE scores for moderate to high-stakes decisions.

Exploring Pediatric Resident Attitudes and Preferences for Board Exam Preparation

Objective The American Board of Pediatrics Certifying Exam (ABP CE) is a high stakes exam and is important for employment and fellowship opportunities in pediatrics. Although research has suggested interventions which may improve scores, little research has focused on resident perception of preparation for the ABP CE. In this study, we aimed to better define pediatric residents' attitudes and preferences regarding preparation for the ABP CE. Methods Pediatric residents from one residency program were invited to participate in focus groups to discuss their attitudes and preferences on board exam preparation for the ABP CE. Focus groups were audio recorded and transcribed. Conventional content analysis was used to analyze qualitative data. From the transcripts, authors developed codes through an iterative process, which were then organized into categories. These categories were then grouped into themes. Results Nineteen residents participated in three focus groups. Focus group transcription analysis resulted in 49 codes, which were sorted into 26 categories. The categories were then grouped into four key themes: 1) the ABP CE is not immediately important early in residency; 2) more personalized guidance is preferred; 3) consistent board preparation focus from the residency program is valued; 4) learning format is important. Conclusions Residents believe preparation for the ABP CE increases in importance as they progress through residency, and they desire more personalized, consistent, and structured focus from their training program related to ABP CE preparation. Attention to these perceptions can help guide pediatric residency program leadership in developing effective board exam preparation strategies and curricula for their residents.

Interventions for undergraduate and postgraduate medical learners with academic difficulties: A BEME systematic review: BEME Guide No. 56

Background: Clinical teachers often struggle to report unsatisfactory trainee performance, partly because of a lack of evidence-based remediation options. Objectives: To identify interventions for undergraduate (UG) and postgraduate (PG) medical learners experiencing academic difficulties, link them to a theory-based framework and provide literature-based recommendations around their use. Methods: This systematic review searched MEDLINE, CINAHL, EMBASE, ERIC, Education Source and PsycINFO (1990-2016) combining these concepts: medical education, professional competence/difficulty and educational support. Original research/innovation reports describing intervention(s) for UG/PG medical learners with academic difficulties were included. Data extraction employed Michie's Behavior Change Techniques (BCT) Taxonomy and program evaluation models from Stufflebeam and Kirkpatrick. Quality appraisal used the Mixed Methods Appraisal Tool (MMAT). The authors synthesized extracted evidence by adapting the GRADE approach to formulate recommendations. Results: Sixty-eight articles met the inclusion criteria, most commonly addressing knowledge (66.2%), skills (53.9%) and attitudinal problems (26.2%), or learner personal issues (41.5%). The most common BCTs were Shaping knowledge, Feedback/monitoring, and Repetition/substitution. Quality appraisal was variable (MMAT 0-100%). A thematic content analysis identified 109 interventions (UG: n = 84, PG: n = 58), providing 24 strong, 48 moderate, 26 weak and 11 very weak recommendations. Conclusion: This review provides a repertoire of literature-based interventions for teaching/learning, faculty development, and research purposes.

Remediating doctors' performance to restore patient safety: a realist review protocol

INTRODUCTION

Underperformance by doctors poses a risk to patient safety. Remediation is an intervention designed to remedy underperformance and return a doctor to safe practice. Remediation is widely used across healthcare systems globally, and has clear implications for both patient safety and doctor retention. Yet, there is a poor evidence base to inform remediation programmes. In particular, there is a lack of understanding as to why and how a remedial intervention may work to change a doctor's practice. The aim of this research is to identify why, how, in what contexts, for whom and to what extent remediation programmes for practising doctors work to support patient safety.

METHODS AND ANALYSIS

Realist review is an approach to evidence synthesis that seeks to develop programme theories about how an intervention works to produce its effects. The initial search strategy will involve: database and grey literature searching, citation searching and contacting authors. The evidence search will be extended as the review progresses and becomes more focused on the development of specific aspects of the programme theory. The development of the programme theory will involve input from a stakeholder group consisting of professional experts in the remediation process and patient representatives. Evidence synthesis will use a realist logic of analysis to interrogate data in order to develop and refine the initial programme theory into a more definitive realist programme theory of how remediation works. The study will follow and be reported according to Realist And Meta-narrative Evidence Syntheses-Evolving Standards (RAMESES).

ETHICS AND DISSEMINATION

Ethical approval is not required. Our dissemination strategy will include input from our stakeholder group. Customised outputs will be developed using the knowledge-to-action cycle framework, and will be targeted to: policy-makers; education providers and regulators, the National Health Service, doctors and academics.

PROSPERO REGISTRATION NUMBER

CRD42018088779.

Correlation Between Scores on Weekly Quizzes and Performance on the Annual Resident In-Service Examination

Context: Medical residency education relies heavily on the use of written and oral testing as a means of assessing a learner’s knowledge acquisition. In the United States, osteopathic emergency medicine residents take an annual specialty-based resident in-service examination (RISE) for this purpose. Their performance on the RISE helps direct educators’ approach to teaching and training.

Objectives: To determine the correlative strength of residents’ cumulative performance on a series of weekly in-house quizzes with their performance on the RISE.

Methods: In this prospective study, emergency medicine residents took a series of 15 quizzes between August 2013 and January 2014. The quizzes were administered using slides integrated with an audience-response system. Quizzes comprised questions gathered from various question banks and commercial test review resources specific to the specialty of emergency medicine. Effort was made to select questions covering topics tested on the RISE. Scores from each of the quizzes were recorded, and these data were analyzed for correlation with residents’ scores on the RISE.

Results: Sixteen emergency medicine residents from all 4 postgraduate years participated in the study. For various reasons (vacation, illness, away rotations), not all 16 residents participated in each quiz. The mean participation rate over all 15 quizzes was 76.7%, with a mean quiz score of 57.8%. A correlation analysis was conducted between the achieved RISE score and the mean quiz score (excluding any quizzes not taken). Graphical analysis revealed a sufficiently linear relationship between the 2 variables, with no outliers. Both variables were normally distributed, as assessed by the Shapiro-Wilks test (P>.05). A strong positive correlation was found between RISE score and mean quiz score (r[14]=0.75; P=.001), with the mean quiz score over the quizzes taken explaining about 57% of the variance in the achieved RISE score.

Conclusions: The results of this study imply that performance on weekly didactic quizzes may be strongly predictive of RISE performance and as such tracking these data may provide insight to educators and learners as to the most effective direction of their educational efforts.

Bureaucrapathologies: Galloping Regulosis, Assessment Degradosis, and Other Unintended Organizational Maladies in Post-Graduate Medical Education

As decadelong observers of evolving administrative regulations governing academic medicine, the authors have identified several organizational disorders they define as "bureaucrapathologies," pathological conditions caused by dysfunctional bureaucratic processes that generate excesses of wasted time, effort, and other resources. Appearing wherever bureaucratic organizations exist, they have become particularly egregious in health care, research, and education. In past decades, graduate medical education has been beset by proliferating assessment requirements accompanied by corresponding documentation requirements imposed by academic educational regulatory agencies (specifically the Accreditation Council on Graduate Medical). Although originating from the best of intentions, these largely untested, unvalidated, and unfunded mandates generate burdensome personnel, time, and resource requirements. As they trickle down organizational levels, the intentions of the originators are inevitably degraded. As motivations and incentives of lower level administrators and faculty differ considerably from those at higher levels, we inevitably encounter debatable assessment practices yielding results of dubious reliability and validity. These processes invariably lead to proliferating reports and paperwork. All of this raises serious questions about the benefits vs. harms of these enterprises. In our view, these pathogenic processes can be recognized as diagnosable subtypes of bureaucrapathology. Here the authors briefly describe two, Galloping Regulosis and Assessment Degradosis, which reflect on their pathogenesis and offer preliminary thoughts for potential remedies. Several other recently identified bureaucrapathological syndromes awaiting future delineation are noted.

The remediation challenge: theoretical and methodological insights from a systematic review

OBJECTIVES

Remediation is usually offered to medical students and doctors in training who underperform on written or clinical examinations. However, there is uncertainty and conflicting evidence about the effectiveness of remediation. The aim of this systematic review was to synthesise the available evidence to clarify how and why remediation interventions may have worked in order to progress knowledge on this topic.

METHODS

The MEDLINE, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), ERIC (Educational Resources Information Centre), Web of Science and Scopus databases were searched for papers published from 1984 to April 2012, using the search terms 'remedial teaching', 'education', 'medical', 'undergraduate'/or 'clinical clerkship'/or 'internship and residency', 'at risk' and 'struggling'. Only studies that included an intervention, then provided retest data, and reported at least one outcome measure of satisfaction, knowledge, skills or effects on patients were eligible for inclusion. Studies of practising doctors were excluded. Data were abstracted independently in duplicate for all items. Coding differences were resolved through discussion.

RESULTS

Thirty-one of 2113 studies met the review criteria. Most studies were published after 2000 (n=24, of which 12 were published from 2009 onwards), targeted medical students (n=22) and were designed to improve performance on an immediately subsequent examination (n=22). Control or comparison groups, conceptual frameworks, adequate sample sizes and long-term follow-up measures were rare. In studies that included long-term follow-up, improvements were not sustained. Intervention designs tended to be highly complex, but their design or reporting did not enable the identification of the active components of the remedial process.

CONCLUSIONS

Most remediation interventions in medical education focus on improving performance to pass a re-sit of an examination or assessment and provide no insight into what types of extra support work, or how much extra teaching is critical, in terms of developing learning. More recent studies are generally of better quality. Rigorous approaches to developing and evaluating remediation interventions are required.