Predicting Performance on the American Board of Physical Medicine and Rehabilitation Written Examination Using Resident Self-Assessment Examination Scores

Enhancing Examination Success: the Cumulative Benefits of Self-Assessment Questions and Virtual Patient Cases

PURPOSE

Research on the learning benefits of the feedback-rich formative assessment environment of virtual patient cases (VPCs) has largely been limited to single institutions and focused on discrete clinical skills or topical knowledge. To augment current understanding, we designed a multi-institutional study to explore the distinct and cumulative effects of VPC formative assessments and optional self-assessment questions (SAQs) on exam performance.

METHOD

In this correlational study, we examined the records of 1,692 students on their family medicine (FM) clerkship at 20 medical schools during the 2014-2015 academic year. Schools utilized an established online curriculum, which included family medicine VPCs, embedded formative assessments, context-rich SAQs corresponding with each VPC, and an associated comprehensive family medicine exam. We used mixed-effects modeling to relate the student VPC composite formative assessment score, SAQ completion, and SAQ performance to students' scores on the FM final examination.

RESULTS

Students scored higher on the final exam when they performed better on the VPC formative assessments, completed associated SAQs, and scored higher on those SAQs. Students' SAQ completion enhanced examination performance above that explained by engagement with the VPC formative assessments alone.

CONCLUSIONS

This large-scale, multi-institutional study furthers the body of research on the effect of formative assessments associated with VPCs on exam performance and demonstrates the added benefit of optional associated SAQs. Findings highlight opportunities for future work on the broader impact of formative assessments for learning, exploring the benefits of integrating VPCs and SAQs, and documenting effects on clinical performance and summative exam scores.

Taking Leave During Residency: Types of Absences and Subsequent Delays and Variations in Physical Medicine and Rehabilitation Medical Board Pass Rates

ABSTRACT

There is limited literature examining the reasons physical medicine and rehabilitation residents take an extended leave of absence during residency and the impact of leave on board examination performance. Such information could better inform leave policies, help guide residency program directors, and potentially destigmatize taking leave. Study objectives were to describe the characteristics of physical medicine and rehabilitation residents who take leave during residency, compare differences in part I (written) and part II (oral) certification examination performance, and determine the prevalence of delays in taking board examinations. Study methodology was a retrospective analysis of deidentified information from the American Board of Physical Medicine and Rehabilitation database between 2008 and 2020. Results indicated four reasons for extended leave of absence: medical, parental, academic/remediation, and unspecified personal reasons. Residents who took an extended leave of absence for medical or parental reasons had similar or better odds of passing their part I and part II examinations compared with those without leave. Residents who took leave for academic/remediation concerns or unspecified personal reasons had lower odds of passing their board examinations. Examination delays for those taking parental or personal health leaves did not affect board pass rates. Further investigation is needed to identify how to support residents on academic/remediation and unspecified leaves during training.

Relationships Between the ACGME Resident and Faculty Surveys and Program Pass Rates on the ABIM Internal Medicine Certification Examination

PURPOSE

The Accreditation Council for Graduate Medical Education (ACGME) has surveyed residents since 2003, and faculty since 2012. Surveys are designed to assess program functioning and specify areas for improvement. The purpose of this study was to assess the association of the ACGME's resident and faculty surveys with residency-program-specific performance on the American Board of Internal Medicine (ABIM) certification exam.

METHOD

Data were available from residents and faculty in 375 U.S. ACGME-accredited internal medicine programs from the 2012-2013, 2013-2014, and 2014-2015 academic years. Analysis of variance and correlations were used to examine the relationship between noncompliance with ACGME program requirements as assessed by the resident and faculty surveys, and ABIM program pass rates.

RESULTS

Noncompliance reported on the resident and faculty surveys was highest for programs not meeting the ACGME program requirement of an 80% pass rate on the ABIM certification examination. This relationship was significant for overall noncompliance, both within the resident (P < .001) and faculty (P < .05) surveys, for many areas within the two surveys (correlations ranged between -.07 and -.25, and P values ranged between .20 and < .001), and for the highest levels of noncompliance across areas of the resident (P < .001) and faculty (P < .04) surveys.

CONCLUSIONS

ACGME resident and faculty surveys were significantly associated with ABIM program pass rates, supporting the importance of these surveys within the ACGME's Next Accreditation System.