The mini-PAT as a multi-source feedback tool for trainees in child and adolescent psychiatry: assessing whether it is fit for purpose

How psychiatrists’ attitudes towards multi-source feedback including patient feedback influenced the educational value: a qualitative study

Background:  Multi-source feedback (MSF) is well-established in psychiatric training. However, evidence on the educational impact is not definitive and there is scanty evidence exploring its value for professional development of psychiatry trainees in the United Kingdom (UK). Evidence suggests the MSF tool currently used is not suitable for specialist trainees. This qualitative research project explored psychiatric doctors’ attitudes towards MSF with patient feedback, to determine how this influenced the feedback’s educational usefulness. Methods: A qualitative study using a phenomenological approach based on a constructivist approach. Purposive sampling identified trainee psychiatrists who completed a more extensive MSF, including patient feedback, than they currently use. They discussed their results in supervised sessions to plan how to use the feedback. Semi-structured interviews were conducted separately with trainees and their supervisors following completion of MSF. The data was analysed thematically. The study was completed in 2020. Results: Seven trainees and five supervisors participated. Four themes were identified. Most had positive opinions about the educational usefulness of MSF, including patient feedback, and made changes to their behaviour following the feedback. Interviewees valued patient feedback and identified it as important in psychiatry. Most valued their patient feedback over their colleague feedback. The complexities of patient feedback in psychiatry and how this may influence the educational usefulness of the feedback were discussed in detail. Conclusions: Findings suggest a need to review the current system of MSF in psychiatry in order to maximise educational benefits. In particular, this research points to the benefits of psychiatric trainees engaging with patient feedback.

Psychometric properties of the mini peer assessment tools (Mini-PAT) in emergency medicine residents

Background: A few studies have been done regarding the validity and reliability of the Mini-Peer Assessment Tool across various specialties. This study was conducted to determine the reliability, content and construct validity of Mini-Peer Assessment Tool to assess the competency of emergency medicine residents. Methods: This study was carried out to investigate the psychometric properties of the mini-PAT tool to evaluate the professional competencies of emergency medicine residents in educational hospitals affiliated to Tehran University of Medical Sciences. The initial Mini-Peer Assessment Tool was translated into Persian. After that, the content validity index and content validity ratio determined by consulting 12 professors of emergency medicine. The construct validity was determined with exploratory factor analysis and investigation of the correlation coefficient on 31 self and 248 peer assessment cases. The reliability of the mini peer assessment tool was determined by internal consistency and item deletion by using Cronbach's alpha coefficient. Reliability was also assessed by determining the agreement between the two tools of self-assessment and peer assessment by using the diagram Bland and Altman. Results: The results showed content validity ratio (CVR) of the items ranged from 0.56 to 0.83, and the content validity index (CVI) of the items ranged from 0.72 to 0.90. The reliability of the self-assessment and peer-assessment tools were 0.83 and 0.95 respectively and there was a relative agreement between the self-assessment method and the peer assessment method. Finally, the tool underwent exploratory factor analysis resulting extraction into two factors namely 'clinical competencies' and 'human interactions' in the peer assessment tool. In the self-assessment tool, the factors of 'good practice' and 'technical competence' were extracted. Conclusion: The results of the present study provided evidence of the adequacy of content validity, reliability of the contextually customized mini-peer assessment tool in assessing the competencies of emergency medicine residents.

The professionalism of psychiatry registrars as perceived by patients and various health practitioners at Weskoppies Psychiatric Hospital, Pretoria

Objectives

Amidst calls for improved professionalism, this study examined the professionalism of psychiatry registrars at Weskoppies Hospital as evaluated by their patients, themselves, their consultants and other health practitioners. The second objective was to examine the perceived importance of aspects of professionalism and compare these descriptively among the various health practitioners and patients.

Method

Participants completed the Professionalism Mini-Evaluation Exercise Questionnaire in evaluating the professionalism of the registrar. The number of questionnaires completed by patients, allied health practitioners, consultant psychiatrists and psychiatry registrars were, respectively, 100, 50, 25 and 20; thus, 195 in total. This previously validated questionnaire consists of 21 items that enquire about doctor-patient relationship skills, reflective skills, time management and interprofessional relationship skills. Participants also ranked the three items of the questionnaire that they considered most important.

Results

Highly statistically significant differences among four groups emerged for both the total and all four subscale scores, with patients generally rating the professionalism of registrars as lower, and not meeting with expectations. All four groups ranked ‘listened actively to patient’ and ‘showed interest in patient as person’ highly. Patients and allied health practitioners ranked the attribute ‘was on time’ highly, while consultants and registrars did not.

Conclusion

Improving professionalism requires that the attributes ‘listening actively to patient’ and ‘recognising and meeting patient needs’ are taken seriously. Doing so requires that patients also evaluate the professionalism of registrars, rather than relying merely on the evaluation by consultants as being sufficient.