The use of patient-level outcomes to inform treatment

Surgeon's experiences of receiving peer benchmarked feedback using patient-reported outcome measures: a qualitative study

Background

The use of patient-reported outcome measures (PROMs) to provide healthcare professionals with peer benchmarked feedback is growing. However, there is little evidence on the opinions of professionals on the value of this information in practice. The purpose of this research is to explore surgeon’s experiences of receiving peer benchmarked PROMs feedback and to examine whether this information led to changes in their practice.

Methods

This qualitative research employed a Framework approach. Semi-structured interviews were undertaken with surgeons who received peer benchmarked PROMs feedback. The participants included eleven consultant orthopaedic surgeons in the Republic of Ireland.

Results

Five themes were identified: conceptual, methodological, practical, attitudinal, and impact. A typology was developed based on the attitudinal and impact themes from which three distinct groups emerged. ‘Advocates’ had positive attitudes towards PROMs and confirmed that the information promoted a self-reflective process. ‘Converts’ were uncertain about the value of PROMs, which reduced their inclination to use the data. ‘Sceptics’ had negative attitudes towards PROMs and claimed that the information had no impact on their behaviour. The conceptual, methodological and practical factors were linked to the typology.

Conclusion

Surgeons had mixed opinions on the value of peer benchmarked PROMs data. Many appreciated the feedback as it reassured them that their practice was similar to their peers. However, PROMs information alone was considered insufficient to help identify opportunities for quality improvements. The reasons for the observed reluctance of participants to embrace PROMs can be categorised into conceptual, methodological, and practical factors. Policy makers and researchers need to increase professionals’ awareness of the numerous purposes and benefits of using PROMs, challenge the current methods to measure performance using PROMs, and reduce the burden of data collection and information dissemination on routine practice.

Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers

BACKGROUND

Many patient-reported outcome measures have been developed in the past two decades, playing an increasingly important role in palliative care. However, their routine use in practice has been slow and difficult to implement.

AIM

To systematically identify facilitators and barriers to the implementation of patient-reported outcome measures in different palliative care settings for routine practice, and to generate evidence-based recommendations, to inform the implementation process in clinical practice.

DESIGN

Systematic literature review and narrative synthesis.

DATA SOURCES

Medline, PsycInfo, Cumulative Index to Nursing and Allied Health Literature, Embase and British Nursing Index were systematically searched from 1985. Hand searching of reference lists for all included articles and relevant review articles was performed.

RESULTS

A total of 3863 articles were screened. Of these, 31 articles met the inclusion criteria. First, data were integrated in the main themes: facilitators, barriers and lessons learned. Second, each main theme was grouped into either five or six categories. Finally, recommendations for implementation on outcome measures at management, health-care professional and patient levels were generated for three different points in time: preparation, implementation and assessment/improvement.

CONCLUSIONS

Successful implementation of patient-reported outcome measures should be tailored by identifying and addressing potential barriers according to setting. Having a coordinator throughout the implementation process seems to be key. Ongoing cognitive and emotional processes of each individual should be taken into consideration during changes. The educational component prior to the implementation is crucial. This could promote ownership and correct use of the measure by clinicians, potentially improving practice and the quality of care provided through patient-reported outcome measure data use in clinical decision-making.

Modelling candidate effectiveness indicators for mental health services

OBJECTIVE

Although Australia and the UK have both made efforts to systematize outcome measurement in mental health services, surprisingly little attention has been paid to how best to analyse routine outcome data in order to determine how services are performing.

METHODS

Outcome data collected in acute inpatient and ambulatory mental health services across Australia during the 2006-2007 financial year were used. three approaches to measuring effectiveness were explored: effect size (ES); the reliable change index (RCI); and standard error of measurement (SEM).

RESULTS

The most conservative results were produced by the RCI and the least conservative by the medium ES statistic and the SEM. By way of example, only 38.0% of inpatient admission-discharge periods of care showed significant improvement for adults when the RCI was used, whereas 67.4% and 72.9% did so when the medium ES and the SEM statistics were used, respectively.

CONCLUSIONS

In any routine outcome measurement exercise, the degree of effectiveness demonstrated by services will depend on the specific statistical indicator used to judge effectiveness. Routine outcome measurement has the potential to answer a range of crucial performance-related questions, but only if the same metric is used. Discussion of the appropriate statistical approach to take to facilitate cross-service, cross-area and even cross-national comparisons warrants attention.

Do adults in contact with Australia's public sector mental health services get better?

This paper describes the outcomes of episodes of care for adults in public sector mental health services across Australia, with a view to informing the debate on service quality. Health of the Nation Outcome Scales (HoNOS) change scores and effect sizes were calculated for 14,659 acute inpatient episodes and 23,692 community episodes. The results showed that people in contact with public sector mental health services generally do get better, although the magnitude of improvement depends on the setting and episode type. This confirmatory finding is particularly positive, given current community concerns about the quality and effectiveness of mental health services.

Routine measurement of outcomes in Australia's public sector mental health services

Objective

This paper describes the Australian experience to date with a national 'roll out' of routine outcome measurement in public sector mental health services.

Methods

Consultations were held with 123 stakeholders representing a range of roles.

Results

Australia has made an impressive start to nationally implementing routine outcome measurement in mental health services, although it still has a long way to go. All States/Territories have established data collection systems, although some are more streamlined than others. Significant numbers of clinicians and managers have been trained in the use of routine outcome measures, and thought is now being given to ongoing training strategies. Outcome measurement is now occurring 'on the ground'; all States/Territories will be reporting data for 2003–04, and a number have been doing so for several years. Having said this, there is considerable variability regarding data coverage, completeness and compliance. Some States/Territories have gone to considerable lengths to 'embed' outcome measurement in day-to-day practice. To date, reporting of outcome data has largely been limited to reports profiling individual consumers and/or aggregate reports that focus on compliance and data quality issues, although a few States/Territories have begun to turn their attention to producing aggregate reports of consumers by clinician, team or service.

Conclusion

Routine outcome measurement is possible if it is supported by a co-ordinated, strategic approach and strong leadership, and there is commitment from clinicians and managers. The Australian experience can provide lessons for other countries.

What outcomes to measure in routine mental health services, and how to assess them: a systematic review

OBJECTIVE

Routine outcome assessment in adult mental health services involves the on-going assessment of patient-level outcomes. Use of outcomes to inform treatment is widely recommended, but seldom implemented. The goals of this review were (i) to identify principles that have been proposed for implementing routine outcome assessment, (ii) to identify the full range of outcome domains that have been proposed for assessment, and (iii) to synthesize proposals for specific outcome domains into emergent categories.

METHOD

A systematic review of published and unpublished research was undertaken, using electronic databases, research registers, conference proceedings, expert informants and the World Wide Web. For goal (i) studies were included that proposed principles for implementing routine outcome assessment. For goal (ii) studies were included that identified at least two patient-level outcome domains for patients using adult mental health services and made some reference to a broader literature base.

RESULTS

Six thousand four hundred publications matched initial search criteria. Seven distinct sets of principles for choosing patient-level outcomes were located, which showed a fair degree of consensus. Sixteen outcome domain proposals were identified, which were synthesized into seven emergent categories: wellbeing, cognition/emotion, behaviour, physical health, interpersonal, society and services.

CONCLUSIONS

The findings from this review were used to develop a four-step method for adult mental health services wishing to implement routine outcome assessment.

Feasibility, usefulness, limitations and perspectives of routine outcome assessment: the South Verona Outcome Project

Closing the gap between research and clinical practice is nowadays considered a priority in outcome studies. Survey studies in community settings having as their main aim the multidimensional measure of outcome of mental health care interventions, including the use of standardised instruments administered as part of the routine clinical activities in mental health services, have recently started to be planned in various countries, but have encountered several difficulties. A naturalistic, longitudinal study aimed to assess the outcome of care provided by a community-based mental health service, the South Verona Outcome Project, has been conducted in Italy starting from the beginning of the 90's and is running since then. This paper: a) describes a series of methodological aspects of the South Verona Outcome Project, such as instruments, study design, inclusion and exclusion criteria, training of the staff, and focuses on strategies used so to ensure feasibility of the assessment and good quality of the data; b) summarises some results of the study, characteristics of feed-back provided, and outputs; c) discusses the problems faced, the impact of this approach on service provision, its limitations and future perspectives.