Acceptability and educational impact of a peer feedback model for significant event analysis

Doctors' attitudes to maintenance of professional competence: A scoping review

CONTEXT

Recent decades have seen the international implementation of programmes aimed at assuring the continuing competence of doctors. Maintenance of Professional Competence (MPC) programmes aim to encourage doctors' lifelong learning and ensure high-quality, safe patient care; however, programme requirements can be perceived as bureaucratic and irrelevant to practice, leading to disengagement. Doctors' attitudes and beliefs about MPC are critical to translating regulatory requirements into committed and effective lifelong learning. We aimed to summarise knowledge about doctors' attitudes to MPC to inform the development of MPC programmes and identify under-researched areas.

METHODS

We undertook a scoping review following Arksey and O'Malley, including sources of evidence about doctors' attitudes to MPC in the United States, the United Kingdom, Canada, Australia, New Zealand and Ireland, and using the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) as a guide.

RESULTS

One hundred and twenty-five sources of evidence were included in the review. One hundred and two were peer-reviewed publications, and 23 were reports. Most were from the United Kingdom or the United States and used quantitative or mixed methods. There was agreement across jurisdictions that MPC is a good idea in theory but doubt that it achieves its objectives in practice. Attitudes to the processes of MPC, and their impact on learning and practice were mixed. The lack of connection between MPC and practice was a recurrent theme. Barriers to participation were lack of time and resources, complexity of the requirements and a lack of flexibility in addressing doctors' personal and professional circumstances.

CONCLUSIONS

Overall, the picture that emerged is that doctors are supportive of the concept of MPC but have mixed views on its processes. We highlight implications for research and practice arising from these findings.

Self-evaluation and peer-feedback of medical students' communication skills using a web-based video annotation system. Exploring content and specificity

OBJECTIVE

Self-evaluation and peer-feedback are important strategies within the reflective practice paradigm for the development and maintenance of professional competencies like medical communication. Characteristics of the self-evaluation and peer-feedback annotations of medical students' video recorded communication skills were analyzed.

METHOD

Twenty-five year 4 medical students recorded history-taking consultations with a simulated patient, uploaded the video to a web-based platform, marked and annotated positive and negative events. Peers reviewed the video and self-evaluations and provided feedback. Analyzed were the number of marked positive and negative annotations and the amount of text entered. Topics and specificity of the annotations were coded and analyzed qualitatively.

RESULTS

Students annotated on average more negative than positive events. Additional peer-feedback was more often positive. Topics most often related to structuring the consultation. Students were most critical about their biomedical topics. Negative annotations were more specific than positive annotations. Self-evaluations were more specific than peer-feedback and both show a significant correlation. Four response patterns were detected that negatively bias specificity assessment ratings.

CONCLUSION

Teaching students to be more specific in their self-evaluations may be effective for receiving more specific peer-feedback.

PRACTICE IMPLICATIONS

Videofragmentrating is a convenient tool to implement reflective practice activities like self-evaluation and peer-feedback to the classroom in the teaching of clinical skills.

Peer assessment among first year medical students in anatomy

Peer assessment has been shown to be an effective tool to promote professionalism in medical students. Peer assessment may be particularly useful in anatomy dissection laboratory as the required close collaboration and long hours of anatomy laboratory provide students insights into their peers' work habits and interpersonal skills. The objective of this study was to quantitatively and qualitatively analyze the use of a validated peer assessment tool in Gross Anatomy. Students in a first year medical school class evaluated three members of their dissection group using an online survey tool. The mid-course and end-of-course evaluation included open-ended comments, as well as a five-point scale that measured three work habits, two interpersonal attributes and one overall score. All 267 students completed the assignment. The overall score and four of the five other assessed categories showed significant improvement from the mid- to end-of-course evaluations. Quantitative and qualitative data also revealed significant improvement among the students who received the lowest mid-course assessments. Seventy-six percent of the class agreed with the statement: "Based on the feedback I received, I made a change in how I worked with or taught my peers." The use of this peer assessment tool used by students in anatomy was associated with improvements in work habits and interpersonal attributes, particularly by the cohort of students who received the lowest mid-course feedback. Peer assessment offers students an opportunity to improve their interpersonal skills and work habits.

Can the theoretical domains framework account for the implementation of clinical quality interventions?

BACKGROUND

The health care quality improvement movement is a complex enterprise. Implementing clinical quality initiatives requires attitude and behaviour change on the part of clinicians, but this has proven to be difficult. In an attempt to solve this kind of behavioural challenge, the theoretical domains framework (TDF) has been developed. The TDF consists of 14 domains from psychological and organisational theory said to influence behaviour change. No systematic research has been conducted into the ways in which clinical quality initiatives map on to the domains of the framework. We therefore conducted a qualitative mapping experiment to determine to what extent, and in what ways, the TDF is relevant to the implementation of clinical quality interventions.

METHODS

We conducted a thematic synthesis of the qualitative literature exploring clinicians' perceptions of various clinical quality interventions. We analysed and synthesised 50 studies in total, in five domains of clinical quality interventions: clinical quality interventions in general, structural interventions, audit-type interventions, interventions aimed at making practice more evidence-based, and risk management interventions. Data were analysed thematically, followed by synthesis of these themes into categories and concepts, which were then mapped to the domains of the TDF.

RESULTS

Our results suggest that the TDF is highly relevant to the implementation of clinical quality interventions. It can be used to map most, if not all, of the attitudinal and behavioural barriers and facilitators of uptake of clinical quality interventions. Each of these 14 domains appeared to be relevant to many different types of clinical quality interventions. One possible additional domain might relate to perceived trustworthiness of those instituting clinical quality interventions.

CONCLUSIONS

The TDF can be usefully applied to a wide range of clinical quality interventions. Because all 14 of the domains emerged as relevant, and we did not identify any obvious differences between different kinds of clinical quality interventions, our findings support an initially broad approach to identifying barriers and facilitators, followed by a "drilling down" to what is most contextually salient. In future, it may be possible to establish a model of clinical quality policy implementation using the TDF.

Barriers and attitudes influencing non-engagement in a peer feedback model to inform evidence for GP appraisal

BACKGROUND

The UK general practitioner (GP) appraisal system is deemed to be an inadequate source of performance evidence to inform a future medical revalidation process. A long-running voluntary model of external peer review in the west of Scotland provides feedback by trained peers on the standard of GP colleagues' core appraisal activities and may 'add value' in strengthening the robustness of the current system in support of revalidation. A significant minority of GPs has participated in the peer feedback model, but a clear majority has yet to engage with it. We aimed to explore the views of non-participants to identify barriers to engagement and attitudes to external peer review as a means to inform the current appraisal system.

METHODS

We conducted semi-structured interviews with a sample of west of Scotland GPs who had yet to participate in the peer review model. A thematic analysis of the interview transcriptions was conducted using a constant comparative approach.

RESULTS

13 GPs were interviewed of whom nine were males. Four core themes were identified in relation to the perceived and experienced 'value' placed on the topics discussed and their relevance to routine clinical practice and professional appraisal: 1. Value of the appraisal improvement activity. 2. Value of external peer review. 3. Value of the external peer review model and host organisation and 4. Attitudes to external peer review.

CONCLUSIONS

GPs in this study questioned the 'value' of participation in the external peer review model and the national appraisal system over the standard of internal feedback received from immediate work colleagues. There was a limited understanding of the concept, context and purpose of external peer review and some distrust of the host educational provider. Future engagement with the model by these GPs is likely to be influenced by policy to improve the standard of appraisal and contractual related activities, rather than a self-directed recognition of learning needs.

Peer mentoring in doctor performance assessment: strategies, obstacles and benefits

CONTEXT

Mentors are increasingly involved in doctor performance assessments. Mentoring seems to be a key determinant in achieving the ultimate goal of those assessments, namely, improving doctor performance. Little is known, however, about how mentors perceive and fulfil this role.

OBJECTIVE

The aim of this paper is to expand understanding of the role of mentors in performance assessment.

METHODS

Thirty-eight mentors undertook formative performance assessments of their peers in a pilot study. A mixed-methods design was used, consisting of a postal survey (n = 28) and qualitative interviews with a subset of mentors (n = 11). Individual semi-structured interviews were completed and transcripts were analysed by two researchers using a grounded theory approach.

RESULTS

The results of the survey showed that 89% of mentors intended to continue in their mentorship role. Interviews revealed that mentors used several strategies in the assessments, including: contrasting and collating information; posing reflective questions, and goal setting. Mentors experienced difficulty in disregarding their views of the doctors evaluated. Some mentors noticed obstacles with specific interview skills such as 'paying attention to their colleagues' strengths' and 'enabling doctors to find their own solutions'. Mentors reported that they and their organisations benefited from the assessments. The perceived benefits included: improved interview skills; increased solidarity, and increased mutual respect.

CONCLUSIONS

The study provides insights into what mentors can do to increase the chance that externally derived information is integrated into doctors' self-assessments. Mainly, mentors used strategies aimed at effectively delivering feedback and encouraging reflection. However, we found that mentors who took part in our study appeared to struggle with a number of obstacles related to: time investment; familiarity with the doctor assessed, and the acquiring of specific interview skills.

Verifying appraisal evidence using feedback from trained peers: views and experiences of Scottish GP appraisers

BACKGROUND

GP appraisal is currently considered inadequate because it lacks robustness. Objective assessment of appraisal evidence is needed to enable judgements on professional performance to be made.

AIM

To determine GP appraisers' views of the acceptability, feasibility, and educational impact of external peer feedback received on three core appraisal activities undertaken as part of this study.

DESIGN OF STUDY

Independent peer review and cross-sectional postal questionnaire study.

SETTING

NHS Scotland.

METHOD

One of three core appraisal activities (criterion audit, significant event analysis, or video of consultations) was undertaken by GP appraisers and subjected to peer review by trained colleagues. A follow-up postal questionnaire elicited participants' views on the potential acceptability, feasibility, and educational impact of this approach.

RESULTS

Of 164 appraisers, 80 agreed to participate; 67/80 (84%) submitted one of three appraisal materials for peer review and returned completed questionnaires. For significant event analyses (n = 44), most responders believed the peer feedback method was feasible (100%) and fair (92.5%) and would add value to appraisal (95.5%). Peer feedback on criterion audits (n = 15) was believed to be acceptable and fair (93.3%) and it was thought it would be a useful educational tool (100%). Completing a consultation video (n = 8) was perceived to be feasible as part of normal general practice (n = 5). It was unanimously agreed that assessment of videos by peers has educational impact and would help improve appraisal.

CONCLUSION

This group of GP appraisers strongly supported the role of external and independent feedback by trained peers as one approach to strengthening the existing appraisal process.