Using a 'peer assessment questionnaire' in primary medical care

BACKGROUND

Periodic assessment of clinician performance or 'revalidation' is being actively considered to reassure the public that doctors are 'up to date and fit to practice'. There is, therefore, increasing interest in how to assess individual clinician performance in a valid and reliable way. The use of peer assessment questionnaires is one of the methods being considered and investigated by the General Medical Council in the UK.

AIM

To test the feasibility of using a peer assessment questionnaire in a primary care setting, and consider the related issues of validity and reliability and compare the results to previous studies.

DESIGN

Cross-sectional survey in a volunteer sample.

SETTING

General practice in the UK.

METHOD

GPs who volunteered to take part in an evaluation of a pilot appraisal implementation scheme were recruited by appraisers. These volunteers (GP subjects) chose 15 colleagues to complete a 'peer assessment' questionnaire that asked peers to make judgements about their clinical skills and other characteristics, such as 'compassion', 'integrity' and 'responsibility'.

RESULTS

Of the 207 practitioners that agreed to be appraised, 113 completed the optional task of implementing the peer questionnaire. Of the 1271 raters, 1189 provided data about their roles and 33.6% of these were GPs. The data revealed significant levels of items where peers were 'unable to evaluate' the issues posed in the questionnaire (ranging from 13.7-1.8%). These rates differed from those obtained in studies based in the US where mean scores were slightly higher. Although the overall results are broadly similar to those previously obtained, there are sufficient differences to suggest that there are contextual issues influencing the interpretation of the items and therefore the scoring process.

CONCLUSION

The volunteer sample in this study found no major obstacles to the implementation of the peer assessment questionnaire. While it is not possible to generalise from this selected volunteer sample, the use of peer assessment questionnaires appears feasible and may be acceptable to clinical practitioners. However, concern remains about the validity of such instruments and that their development did not fully consider issues of procedural justice or whether the overall purpose of the tools was to be formative, summative, or both.

Organizational assessment in general practice: a systematic review and implications for quality improvement

BACKGROUND

Quality improvement of organizational aspects in general practice is receiving increasing attention. In particular, the impact of effective organization on preventative care has been recognized. Organizational assessments are typically used as part of professionally led accreditation schemes where there is a tension between externally led quality assurance and internally led quality improvement. The aim of this article is to inform the debate by reviewing the international-peer-reviewed literature on organizational assessments used in general practice settings.

DESIGN

Systematic literature review.

METHODS

The literature was searched for articles relating to organizational assessment. Titles and abstracts were examined by two independent reviewers and relevant articles obtained. Bibliographies were examined for follow-up references. Data were extracted on the development and use of assessment methods.

RESULTS

Thirteen papers describing five organizational assessment instruments were included for detailed appraisal.

CONCLUSION

This review discovered a developing field containing different approaches to the measurement of organizational aspects of general practice. Whilst professionally led accreditation is well-developed and dependent on externally led quality assurance, approaches to internally led quality improvement are less well-developed. There is a need for organizational assessment tools designed for the purpose of stimulating internal development.

Reduction in uptake of PSA tests following decision aids: systematic review of current aids and their evaluations

A man's decision to have a prostate-specific antigen (PSA) test should be an informed one. We undertook a systematic review to identify and appraise PSA decision aids and evaluations. We searched 15 electronic databases and hand-searched key journals. We also contacted key authors and organisations. All decision aids and evaluations that discussed PSA were included, with meta-analyses performed on two outcomes from the evaluations: PSA testing and patient knowledge of PSA and related issues. Seven decision aids and 11 evaluations were included. The meta-analysis showed a significantly reduced probability in PSA testing after a decision aid: -3.5% (95% confidence interval: 0.0 to 7.2%; P = 0.050). There were significant improvements in knowledge within 2 weeks after a decision aid: 19.5% (95% confidence interval: 14.2 to 24.8%; P < 0.001). The effect on knowledge was less pronounced within 12-18 months after a decision aid: 3.4% (95% confidence interval: -0.7 to 7.4%; P = 0.10). PSA decision aids improve knowledge about PSA testing, at least in the short term. Men given these decision aids seem to be less likely to have the PSA test.

PSA testing for prostate cancer: an online survey of the views and reported practice of General Practitioners in the UK

BACKGROUND

The role of Prostate Specific Antigen (PSA) testing in the early detection of prostate cancer is controversial. Current UK policy stipulates that any man who wishes to have a PSA test should have access to the test, provided he has been given full information about the benefits and limitations of testing. This study aimed to determine UK GPs' current reported practice regarding PSA testing, and their views towards informed decision-making and PSA testing.

METHOD

Online questionnaire survey, with a sample of 421 GPs randomly selected from a database of GPs across the UK.

RESULTS

95% (400/421) of GPs responded. 76% of GPs reported having performed a PSA test for an asymptomatic man at least once in the previous three months, with 13% reported having tested more than five men in this period. A majority of GPs reported they would do a PSA test for men presenting with a family history and requesting a test, for asymptomatic men requesting a test and also for men presenting with lower urinary tract symptoms. Reported testing rates were highest for men with a family history. Amongst men with lower urinary tract symptoms and men with no symptoms, reported testing rates were significantly higher for older than younger men. The majority of GPs expressed support for the current policy (67%), and favoured both the general practitioner and the man being involved in the decision making process (83%). 90% of GPs indicated that they would discuss the benefits and limitation of testing with the man, with most (61%) preferring to ask the man to make a further appointment if he decides to be tested.

CONCLUSION

This study indicates that PSA testing in asymptomatic men is a regular occurrence in the UK, and that there is general support from GPs for the current policy of making PSA tests available to 'informed' men who are concerned about prostate cancer. While most GPs indicated they would discuss the benefits and limitations prior to PSA testing, and most GPs favoured a shared approach to decision making, it is not known to what extent men are actually being informed. Research is needed to evaluate the most effective approach to assisting men in making an informed decision about whether or not to have a PSA test.

The OPTION scale: measuring the extent that clinicians involve patients in decision-making tasks

OBJECTIVE

To examine the psychometric properties of a revised scale, named 'observing patient involvement in decision making' (OPTION), by analysing its reapplication to a sample of routine primary care consultations. The OPTION instrument assesses to what degree clinicians involve patients in decision making.

DESIGN

Cross-sectional assessment of medical interaction by two calibrated raters.

SETTING

Primary care.

PARTICIPANTS

Twenty-one general practitioners provided 186 consultations for assessment.

MEASUREMENTS

Observational score using the OPTION instrument.

RESULTS

Compared with the first version of the OPTION scale, the revised scale that uses a magnitude instead of an attitude scale, when applied to the same data set, resulted in improvement in the scale's reliability and to lower scores for the levels of involvement achieved by the practitioners. Factor analysis confirms that it is acceptable to regard the scale as a single construct. Although there is moderate variability when raters are assessed on an item by item basis, the agreements between raters at the level of the overall OPTION score is high (the intraclass correlation coefficient scores for total OPTION score was 0.77), a level that is acceptable for the evaluation of a set of consultations per practitioner (e.g. between 5 and 10 consultations), where aggregate scores would be used for determining overall performance.

CONCLUSIONS

We conclude that OPTION is sufficiently reliable to be used for formal assessment at the level of the whole instrument (all 12 items).

"Of the edgelands": broadening the scope of qualitative methodology

In an editorial in a previous issue of this journal Rapport et al introduced the metaphor of the edgelands, arguing that the area between urban and rural landscapes serves to illustrate some of the difficulties of interdisciplinarity experienced by those who work in the medical humanities. In this paper the authors explore some specific issues of qualitative research methodology in health care research. The paper describes a broadening out of the scope of qualitative inquiry in social scientific research in health and social care. The paper explains why some new methodologies have emerged and how both old and new methodologies are grouped around three interlocking strands: narrative based, arts based, and redefined, methodology. In order to illustrate developments in this field, the authors present three examples of the use of these methodologies in practice: photo elicitation technique; discourse analytic, and interpretive anthropological, method. Finally the authors illustrate how these methodologies can give added value to health services research.

Voices of the terminally ill: uncovering the meaning of desire for euthanasia

This study aimed to better understand the meaning of desire for euthanasia. An hermeneutic approach was undertaken using a purposively selected sample of advanced cancer patients who desired euthanasia while receiving palliative care. Unstructured interviews were conducted with six participants, which were audiotaped, transcribed and analysed. This approach allowed in-depth exploration and interpretation of the patients' lived experience. The findings illustrated a timeline from previous wellness to approaching death with five major themes: (1) reality, (2) perception, (3) anticipation, (4) desire and (5) holding environment. The desire for euthanasia is not confined to physical or psychosocial concerns relating to advanced cancer, but incorporates hidden existential yearnings for connectedness, care and respect, understood within the context of the patients' lived experience. Euthanasia requests cannot be taken at face value but require in-depth exploration of their covert meaning, in order to ensure that the patients' needs are being addressed adequately.

Identifying future models for delivering genetic services: a nominal group study in primary care

Background

To enable primary care medical practitioners to generate a range of possible service delivery models for genetic counselling services and critically assess their suitability.

Methods

Modified nominal group technique using in primary care professional development workshops.

Results

37 general practitioners in Wales, United Kingdom too part in the nominal group process. The practitioners who attended did not believe current systems were sufficient to meet anticipated demand for genetic services. A wide range of different service models was proposed, although no single option emerged as a clear preference. No argument was put forward for genetic assessment and counselling being central to family practice, neither was there a voice for the view that the family doctor should become skilled at advising patients about predictive genetic testing and be able to counsel patients about the wider implications of genetic testing for patients and their family members, even for areas such as common cancers. Nevertheless, all the preferred models put a high priority on providing the service in the community, and often co-located in primary care, by clinicians who had developed expertise.

Conclusion

There is a need for a wider debate about how healthcare systems address individual concerns about genetic concerns and risk, especially given the increasing commercial marketing of genetic tests.

Arriving at the postmodern medical consultation

The analysis of the medical consultation is characterised by mainly prescriptive attempts to recommend 'best practice'. As the role of the individual in society has gained prominence, the power relationships in medical practice have had to change to reflect the increasing recognition of autonomy and self-determination. Medical discourse is at a junction, having to relinquish authoritarianism and grapple with the concept of sharing information and decisions in an area where complex and uncertain data exist, albeit often without full disclosure. The writing of Foucault and Lyotard and the concept of postmodernism fit well with the idea that the consultation between doctors and patients is increasingly becoming a contested space that occupies multiple voices, such as that of the media, the pharmaceutical industry, government-led guidelines and that of the profession. Creating the circumstances and the means for creating an effective dialogue in the postmodern consultation is the prime task for physicians.

Shared decision making and risk communication in practice: a qualitative study of GPs' experiences

BACKGROUND

Important barriers to the wider implementation of shared decision making remain. The experiences of professionals who are skilled in this approach may identify how to overcome these barriers.

AIMS

To identify the experiences and views of professionals skilled in shared decision making and risk communication, exploring the opportunities and challenges for implementation.

DESIGN OF STUDY

Qualitative study.

SETTING

Gwent Health Authority.

METHOD

Exit interviews using focus group methodology with 20 GPs who had been in practice between 1 and 10 years, and participated in an explanatory trial lasting 6 months. The trial interventions comprised training in shared decision-making skills and the use of risk communication materials. The doctors consulted with up to 48 patients each (mean = 40, half of them audiotaped) for the study.

RESULTS

The GPs indicated positive attitudes towards involving patients and described positive effects on their consultations. However, the frequency of applying the new skills and tools was limited outside the trial. Doctors were selective about when they felt greater patient involvement was appropriate and feasible, rather than seeking to apply the approaches to the majority of consultations. They felt they often responded to consumer preferences for low levels of involvement in decision making. Time limitations were important in not implementing the approach more widely.

CONCLUSION

The promotion of 'patient involvement' appears likely to continue. Professionals appear receptive to this, and willing to acquire the relevant skills. Strategies for wider implementation of patient involvement could address how consultations are scheduled in primary care, and raise consumers' expectations or desires for involvement.

Assessing organisational development in primary medical care using a group based assessment: the Maturity Matrix

OBJECTIVE

To design and develop an instrument to assess the degree of organisational development achieved in primary medical care organisations.

DESIGN

An iterative development, feasibility and validation study of an organisational assessment instrument.

SETTING

Primary medical care organisations.

PARTICIPANTS

Primary care teams and external facilitators.

MAIN OUTCOME MEASURES

Responses to an evaluation questionnaire, qualitative process feedback, hypothesis testing, and quantitative psychometric analysis (face and construct validity) of the results of a Maturity Matrix assessment in 55 primary medical care organisations.

RESULTS

Evaluations by 390 participants revealed high face validity with respect to its usefulness as a review and planning tool at the practice level. Feedback from facilitators suggests that it helped practices to prioritise their organisational development. With respect to construct validity, there was some support for the hypothesis that training and non-training status affected the degree and pattern of organisational development. The size of the organisation did not have a significant impact on the degree of organisational development.

CONCLUSION

This practice based facilitated group evaluation method was found to be both useful and enjoyable by the participating organisations. Psychometric validation revealed high face validity. Further developments are in place to ensure acceptability for summative work (benchmarking) and formative feedback processes (quality improvement).

Assessing organisational development in primary medical care using a group based assessment: the Maturity Matrix

OBJECTIVE

To design and develop an instrument to assess the degree of organisational development achieved in primary medical care organisations.

DESIGN

An iterative development, feasibility and validation study of an organisational assessment instrument.

SETTING

Primary medical care organisations.

PARTICIPANTS

Primary care teams and external facilitators.

MAIN OUTCOME MEASURES

Responses to an evaluation questionnaire, qualitative process feedback, hypothesis testing, and quantitative psychometric analysis (face and construct validity) of the results of a Maturity Matrix assessment in 55 primary medical care organisations.

RESULTS

Evaluations by 390 participants revealed high face validity with respect to its usefulness as a review and planning tool at the practice level. Feedback from facilitators suggests that it helped practices to prioritise their organisational development. With respect to construct validity, there was some support for the hypothesis that training and non-training status affected the degree and pattern of organisational development. The size of the organisation did not have a significant impact on the degree of organisational development.

CONCLUSION

This practice based facilitated group evaluation method was found to be both useful and enjoyable by the participating organisations. Psychometric validation revealed high face validity. Further developments are in place to ensure acceptability for summative work (benchmarking) and formative feedback processes (quality improvement).

Patient-based outcome results from a cluster randomized trial of shared decision making skill development and use of risk communication aids in general practice

BACKGROUND

Shared decision-making (SDM) between professionals and patients is increasingly advocated from ethical principles. Some data are accruing about the effects of such approaches on health or other patient-based outcomes. These effects often vary substantially between studies.

OBJECTIVE

Our aim was to evaluate the effects of training GPs in SDM, and the use of simple risk communication aids in general practice, on patient-based outcomes.

METHODS

A cluster randomized trial with crossover was carried out with the participation of 20 recently qualified GPs in urban and rural general practices in Gwent, South Wales. A total of 747 patients with known atrial fibrillation, prostatism, menorrhagia or menopausal symptoms were invited to a consultation to review their condition or treatments. After baseline, participating doctors were randomized to receive training in (i) SDM skills; or (ii) the use of simple risk communication aids, using simulated patients. The alternative training was then provided for the final study phase. Patients were randomly allocated to a consultation during baseline or intervention 1 (SDM or risk communication aids) or intervention 2 phases. A randomly selected half of the consultations took place in 'research clinics' to evaluate the effects of more time for consultations, compared with usual surgery time. Patient-based outcomes were assessed at exit from consultation and 1 month follow-up. These were: COMRADE instrument (principal measures; subscales of risk communication and confidence in decision), and a range of secondary measures (anxiety, patient enablement, intention to adhere to chosen treatment, satisfaction with decision, support in decision making and SF-12 health status measure). Multilevel modelling was carried out with outcome score as the dependent variable, and follow-up point (i.e. exit or 1 month later for each patient), patient and doctor levels of explanatory variables.

RESULTS

No statistically significant changes in patient-based outcomes due to the training interventions were found: COMRADE risk communication score increased 0.7 [95% confidence interval (CI) -0.92 to 2.32] after risk communication training and 0.9 (95% CI -0.89 to 2.35) after SDM training; and COMRADE satisfaction with communication score increased by 1.0 (95% CI -1.1 to 3.1) after risk communication, and decreased by 0.6 (95% CI 2.7 to -1.5) after SDM training. Patients' confidence in the decision (2.1 increase, 95% CI 0.7-3.5, P < 0.01) and expectation to adhere to chosen treatments (0.7 increase, 95% CI 0.04-1.36, P < 0.05) were significantly greater among patients seen in the research clinics (when more time was available) compared with usual surgery time. Most outcomes deteriorated between exit and 1 month later. There was no interaction between intervention effects.

CONCLUSION

Patients can be more involved in treatment decisions, and risks and benefits of treatment options can be explained in more detail, without adversely affecting patient-based outcomes. SDM and risk communication may be advocated from values and ethical principles even without evidence of health gain or improvement in patient-based outcomes, but the resources required to enhance these professional skills must also be taken into consideration. These data also indicate the benefits of extra consultation time.

Involving patients in decision making and communicating risk: a longitudinal evaluation of doctors' attitudes and confidence during a randomized trial

BACKGROUND

Important barriers to the wider implementation of shared decision making (SDM) and risk communication in practice remain. The attitudes of professionals undergoing training in these approaches may inform how to overcome these barriers, but there are few such data yet available.

AIM

To identify the attitudes of professionals during participation in a large practice-based intervention study with substantial individual exposure to SDM and risk communication, and to assess their confidence with these approaches and reported frequency of implementing them.

SETTING AND PARTICIPANTS

Twenty general practitioners (GPs) who had been in practice between 1 and 10 years, and participated in an explanatory trial lasting 6 months. The trial interventions comprised training in SDM skills and the use of risk communication materials. The doctors consulted with up to 48 patients each (mean = 40, half of them audio-taped) for the study.

METHODS

Questionnaire assessments before and after each training stage.

RESULTS

The GPs indicated positive attitudes towards involving patients and towards the training interventions. They indicated that the risk information packs were applicable but had used them only occasionally with patients outside the trial. No statistically significant changes were associated with the specific interventions in terms of doctors' confidence in discussing risk information after the risk communication intervention, or attitudes to patient involvement after the SDM intervention. Most attitudes and confidence ratings showed positive changes during the course of the trial as a cohort effect. Such positive changes were associated with female doctors more than male doctors, but not with MRCGP (postgraduate vocational) qualification. Time constraints remained important throughout the study in not implementing the approach more frequently.

CONCLUSIONS

Professionals appear receptive to patient involvement, and willing to acquire the relevant skills. SDM and risk communication training did not appear to contribute differentially to this. Practical barriers such as time constraints should probably be addressed with greater priority than the precise content of training or continuing professional development initiatives if 'involvement' is to become a commoner experience for patients in primary care.

Resource effects of training general practitioners in risk communication skills and shared decision making competences

RATIONALE, AIMS AND OBJECTIVES

Involving patients more in decisions about their own care requires doctors to be trained in effective ways of communicating information and in developing competences to negotiate levels of patient involvement which are most appropriate for each case. The aim of this study was to determine the cost of such training and identify which service resource variables are subsequently affected.

METHODS

An explanatory cluster randomized crossover trial was carried out which involved training general practitioners (GPs) in the use of risk communication (RC) tools, shared decision making (SDM) competences or both. Continuing care by GPs of patients with one of four chronic conditions (menopausal symptoms, menorrhagia, atrial fibrillation, prostatism) was reviewed before and after training. Cost of training was assessed by prospective monitoring of resources used. Data on prescribing, referrals and investigations were collected via questionnaires to participating practitioners. Data on follow-up GP consultations were extracted from medical records. Three two-level logistic models were performed to investigate the probability of training having an effect on prescribing, referrals and investigations ordered at the review consultation.

RESULTS

Training cost pound 1218 per practitioner which increased the cost of a consultation by pound 2.89. Training in SDM or combined with RC significantly affected the probability of a prescription being issued to women with menopausal symptoms and menorrhagia (although RC on its own had no effect) but did not significantly affect prescribing for patients with prostatism or atrial fibrillation. It did not significantly affect the probability of investigations, referrals or follow-up GP visits for any of the conditions.

CONCLUSION

Unless training has a major influence on consultation length, it is unlikely to have any major impacts on cost.

Achieving involvement: process outcomes from a cluster randomized trial of shared decision making skill development and use of risk communication aids in general practice

BACKGROUND

A consulting method known as 'shared decision making' (SDM) has been described and operationalized in terms of several 'competences'. One of these competences concerns the discussion of the risks and benefits of treatment or care options-'risk communication'. Few data exist on clinicians' ability to acquire skills and implement the competences of SDM or risk communication in consultations with patients.

OBJECTIVE

The aims of this study were to evaluate the effects of skill development workshops for SDM and the use of risk communication aids on the process of consultations.

METHODS

A cluster randomized trial with crossover was carried out with the participation of 20 recently qualified GPs in urban and rural general practices in Gwent, South Wales. A total of 747 patients with known atrial fibrillation, prostatism, menorrhagia or menopausal symptoms were invited to a consultation to review their condition or treatments. Half the consultations were randomly selected for audio-taping, of which 352 patients attended and were audio-taped successfully. After baseline, participating doctors were randomized to receive training in (i) SDM skills or (ii) the use of simple risk communication aids, using simulated patients. The alternative training was then provided for the final study phase. Patients were allocated randomly to a consultation during baseline or intervention 1 (SDM or risk communication aids) or intervention 2 phases. A randomly selected half of the consultations were audio-taped from each phase. Raters (independent, trained and blinded to study phase) assessed the audio-tapes using a validated scale to assess levels of patient involvement (OPTION: observing patient involvement), and to analyse the nature of risk information discussed. Clinicians completed questionnaires after each consultation, assessing perceived clinician-patient agreement and level of patient involvement in decisions. Multilevel modelling was carried out with the OPTION score as the dependent variable, and rater, consultation and clinician levels of data, standardized by rater within clinician.

RESULTS

Following each of the interventions, the clinicians significantly increased their involvement of patients in decision making (OPTION score increased by 10.6 following risk communication training [95% confidence interval (CI) 7.9 -13.3; P < 0.001] and by 12.9 after SDM skill development (95% CI 10 -15.8, P < 0.001), a moderate effect size. The level of involvement achieved by the risk communication aids was significantly increased by the subsequent introduction of the skill development workshops (7.7 increase in OPTION score, 95% CI 3.4-12; P < 0.001). The alternative sequence (skills followed by risk communication aids) did not achieve this effect. The use of most risk information formats increased after the provision of specific risk communication aids (P < 0.001). Clinicians using the risk communication tools perceived significantly higher patient and clinician agreement on treatment (P < 0.001), patient satisfaction with information (P < 0.01), clinician satisfaction with decision (P < 0.01) and general overall satisfaction with the consultation (P < 0.001) than those who were exposed to SDM skill development workshops.

CONCLUSIONS

These clinicians were able to acquire the skills to implement SDM competences and to use risk communication aids. Each intervention provided independent effects. Further progress towards greater patient involvement in health care decision making is possible, and skill development in this area should be incorporated into postgraduate professional development programmes.

Organisational change theory and the use of indicators in general practice

General practices are making greater use of indicators to help shape and develop organisational arrangements supporting the delivery of health care. Debate continues concerning what exactly such indicators should measure and how they should be used to achieve improvement. Organisational theories can provide an analytical backdrop to inform the design of indicators, critique their construction, and evaluate their use. Systems theory, organisational development, social worlds theory, and complexity theory each has a practical contribution to make to our understanding of how indicators work in prompting quality improvements and why they sometimes don't. This paper argues that systems theory exerts the most influence over the use of indicators. It concludes that a strategic framework for quality improvement should take account of all four theories, recognising the multiple realities that any one approach will fail to reflect.