Workplace-based assessments of entrustable professional activities in a psychiatry core clerkship: an observational study

From Practice to Proficiency: Evaluation of a Novel Workplace-Based Assessment (WBA) in a Postgraduate Chemical Pathology Residency Program in Pakistan

Introduction

A workplace-based assessment (WBA) model was implemented in the postgraduate (PG) residency program of Chemical Pathology at the Department of Pathology & Laboratory Medicine, Aga Khan University (AKU). PGs were assessed using direct observation of practical skills (DOPS), evaluation of clinical events (ECE) and case-based discussion (CBD) on a virtual learning environment (VLE) platform.

Objectives

To evaluate WBA frequency, case mix, feedback, and satisfaction levels of faculty and PGs of Chemical Pathology at AKU.

Methods

Data from January 2019 to June 2023 was assessed. Tool utilization and case mix frequencies were calculated. PG and faculty satisfaction levels, as well as feedback and discussion time, were averaged. A thematic analysis was conducted on descriptive comments.

Results

Out of 911 WBAs attempted, 79.1% (n=730) were CBDs, 10.8% (n=98) were DOPS, and 9.1% (n=83) were ECEs, showing a well-distributed case mix. Average satisfaction levels for CBD, ECE, and DOPS among both PGs and faculty were 8.38, 8.48, and 8.59, and 8.20, 8.36, and 8.46, respectively. Faculty feedback averaged 8.40, 8.65, and 7.85 minutes for CBD, ECE, and DOPS, respectively. Discussion times averaged 9.37, 9.52, and 13.36 minutes for CBD, ECE, and DOPS, respectively. Suggestions for development were noted in 20.82% (n=225) of CBDs, 21.69% (n=18) of ECEs, and 16.32% (n=16) of DOPS. Positives were documented in 40% (n=292) of CBDs, 28.92% (n=24) of ECEs, and 7.14% (n=7) of DOPS.

Conclusion

This study evaluated a web-based WBA model in chemical pathology training, suggesting its applicability in diverse pathology specialties and regional training programs.

Entrustable professional activities in undergraduate dental education: A practical model for development and validation

OBJECTIVE

Entrustable professional activities (EPAs) are tasks that a person who is qualified or is in the process to be credentialed are allowed to engage. There are several levels of entrustment based on degrees of supervision assigned to each EPA. This paper aims to describe the process and outcome of creating EPAs; validate EPAs relevant to undergraduate dental training.

METHODS

A draft set of EPA statements was developed based on the consensus of an expert panel. These were then mapped to the nationally determined minimum experience thresholds (clinical and procedural experiences/competencies) and aligned to task-based instructional strategy. The EPAs were validated to improve the relevance by using a criterion-based rubric.

RESULTS

An end-to-end process workflow led to the development of an EPA-based educational framework to bridge the gaps in the curriculum. The process identified a total of 41 EPAs and out of which, 10 EPAs were notated as core EPAs and will be subjected to structured workplace-based assessment complying to the national standards. The validation exercise rated core EPAs with an overall score matching close to the cut-off of 4.07 (Equal rubric).

CONCLUSION

The end-to-end process workflow provided the opportunity to elaborate a structured process for the development of EPAs for undergraduate dental education. As validation is a continuous process, feedback from implementation will inform the next steps.

Commentary on alignment of medical student assessment and vocational training in psychiatry and addiction medicine

OBJECTIVE

To comment upon the potential for alignment of medical student assessment and vocational specialist training through the RANZCP-CanMEDS model of Entrustable Professional Activities (EPAs) and Workplace-Based Assessments (WBAs). We discuss a specific post hoc example of such an alignment in an Australian graduate medical school in Psychiatry and Addiction Medicine.

CONCLUSIONS

Vocational training models of assessment, such as the RANZCP specialist training program for psychiatrists, can potentially be mapped to medical student education in formative and summative assessment through CanMEDs-based EPAs and WBAs, to assist in transition to specialist training.

Subjective achievement from psychiatry rotation in the Japanese postgraduate residency system: a longitudinal questionnaire study

BACKGROUND

Psychiatry rotation has been mandatory in the Japanese postgraduate residency system since 2020. Some psychiatry-related competency items are stipulated as mandatory for residents. The current study aimed to clarify whether psychiatry rotation affected residents' subjective achievement of these competency items.

METHODS

This longitudinal study was conducted among postgraduate residents who completed a rotation in the psychiatry department at Nagasaki University Hospital across two academic years (2020-2021). The survey was administered at the start and at the end of the psychiatry rotation. Residents evaluated their subjective understanding and confidence regarding initiating treatment for these competency items using a six-point Likert scale. The average scores for each item were compared between pre-rotation and post-rotation.

RESULTS

In total, 99 residents (91.7%) responded to this survey. Residents had significantly higher scores at post-rotation compared with pre-rotation in all psychiatry-related competency items in both subjective understanding and confidence in initiating treatment. Additionally, strong effect sizes were found for many items.

CONCLUSION

Residents improved learning about psychiatry-related competency items through psychiatry rotation. This finding suggests that it is reasonable for psychiatry rotation to be mandatory in the current Japanese postgraduate residency system. The importance of psychiatry is likely to increase in both undergraduate and postgraduate medical education in the future. It is necessary to continuously update educational strategies to meet changing social needs over time. As this study was conducted at a single institution, a multi-center study is needed to expand the current findings.

Evidence-oriented teaching of geriatric psychiatry: a narrative literature synthesis and pilot evaluation of a clerkship seminar

INTRODUCTION

The field of geriatric psychiatry has in recent decades developed into an independent discipline, incorporating elements of psychiatry, neurology and internal medicine. In view of demographic changes, this field is becoming increasingly relevant for primary care and undergraduate medical training. So far, however, there is little educational guidance for instructional design of geriatric psychiatry in undergraduate medical education.

PROJECT DESCRIPTION

A narrative literature review of medical education studies in the field of geriatric psychiatry was conducted. Student evaluations of a geriatric psychiatry clerkship seminar were analyzed, followed by a target group analysis. Results informed the iterative development of new clerkship seminar content and structure. This was implemented and evaluated over several academic cycles. Learning material was made available via the open-source learning management system "ILIAS".

RESULTS

A total of 29 medical education articles were identified and evaluated. The previous seminar in geriatric psychiatry at our university hospital was rated below average (Likert item overall rating of 4.3/6 compared to other seminars with an average overall rating of 5.2, p<0.001). An evidence-oriented revision of the content and instructional design was implemented. Activation of learners, self-reference effect, and audience questioning were used during the lecture. Additionally, two geriatric psychiatry case scenarios were adapted for discussion. We saw continuous improvement of student evaluations of the revised course, reaching a rating improvement of 5.3 out of 6 (p<0.01, U=135.5 Cohen's d=1.28).

CONCLUSION

A systematic approach was used to develop a geriatric psychiatry clerkship seminar, based on medical education evidence, for undergraduate medical students, resulting in better student evaluations. The teaching materials can be adapted for local implementation at other teaching hospitals. Future studies should also explore effects regarding higher learning outcomes.