Does faculty development influence the quality of in-training evaluation reports in pharmacy?

Nudging clinical supervisors to provide better in-training assessment reports

INTRODUCTION

In-training assessment reports (ITARs) summarize assessment during a clinical placement to inform decision-making and provide formal feedback to learners. Faculty development is an effective but resource-intensive means of improving the quality of completed ITARs. We examined whether the quality of completed ITARs could be improved by 'nudges' from the format of ITAR forms.

METHODS

Our first intervention consisted of placing the section for narrative comments at the beginning of the form, and using prompts for recommendations (Do more, Keep doing, Do less, Stop doing). In a second intervention, we provided a hyperlink to a detailed assessment rubric and shortened the checklist section. We analyzed a sample of 360 de-identified completed ITARs from six disciplines across the three academic years where the different versions of the ITAR were used. Two raters independently scored the ITARs using the Completed Clinical Evaluation Report Rating (CCERR) scale. We tested for differences between versions of the ITAR forms using a one-way ANOVA for the total CCERR score, and MANOVA for the nine CCERR item scores.

RESULTS

Changes to the form structure (nudges) improved the quality of information generated as measured by the CCERR instrument, from a total score of 18.0/45 (SD 2.6) to 18.9/45 (SD 3.1) and 18.8/45 (SD 2.6), p = 0.04. Specifically, comments were more balanced, more detailed, and more actionable compared with the original ITAR.

DISCUSSION

Nudge interventions, which are inexpensive and feasible, should be included in multipronged approaches to improve the quality of assessment reports.

Impact of faculty development programme on self-efficacy, competency and attitude towards medical education in Bhutan: a mixed-methods study

BACKGROUND

Soon after Bhutan's first medical university was established in 2012, Faculty Development Programmes (FDPs) were adopted for efficient delivery of postgraduate medical curriculum. Medical education was an additional responsibility for the clinicians who already had multi-dimensional roles in the healthcare system where there is acute shortage of healthcare professionals. We studied the impact of FDPs on postgraduate medical education in Bhutan.

METHODS

This was a mixed-methods study with a quantitative (cohort study - quasi-experimental with 18 participants) and concurrent explanatory qualitative component (focused group discussion (FGD) with 11 teaching faculty members). The 18 participants were given a structured FDP designed by the University. The FGD assessed teacher self-efficacy and competency using standard tools before and after the FDP. Thematic analysis of the FGD explored the impact of FDPs in the delivery of postgraduate residency programmes.

RESULTS

There were significant increase in the teacher self-efficacy (31 vs 34, p = 0.009) and competency scores (56 vs 64, p = 0.011). There were significant improvements in self-efficacy in the domain of the teaching relevant subject contents and developing creative ways to cope with system constraints. In teaching-learning assessments, there was a significant appreciation of the effectiveness of lectures and tutorials and the use of essay questions. The FGD demonstrated the acceptance of FDPs and its importance in quality improvement of postgraduate medical education, professional development of teachers and improvement of their communication skills. The teachers have now migrated from the conventional methods of teaching to workplace-based teaching and assessment. The FDPs also resulted in review and revision of postgraduate medical curriculum soon after the first batch graduated in 2018. Lack of adequate support from relevant stakeholders and lack of a medical education centre in the University were seen as major challenges.

CONCLUSIONS

The FDPs have brought tangible professionalization of postgraduate medical education at an early stage of the medical university. There is a need for continued efforts to strengthen, sustain and consolidate the gains made thus far.