Overcoming obstacles to routine outcome measurement. The nuts and bolts of implementing clinical audit

Measuring performance of professional role substitution models of care against traditional medical care in healthcare-A systematic review

OBJECTIVES

To identify outcome measures used to evaluate performance of healthcare professional role substitution against usual medical doctor or specialist medical doctor care to facilitate our understanding of the adequacy of these measures in assessing quality of healthcare delivery.

METHODS

Using a systematic approach, we searched Medline, Cochrane Central Register of Controlled Trials, Embase, CINAHL, and Web of Science from database inception until May 2020. Studies that presented original comparative data on at least one outcome measure were included following screening by two authors. Findings were synthesized, and outcome measures classified into six domains which included: effectiveness, safety, appropriateness, access, continuity of care, efficiency, and sustainability which were informed by the Institute of Medicine dimensions of healthcare quality, the Australian health performance framework, and Levesque and Sutherland's integrated performance measurement framework.

RESULTS

One thirty five articles met the inclusion criteria, describing 58 separate outcome measures. Safety of role substitution models of care was assessed in 80 studies, effectiveness (n = 60), appropriateness (n = 40), access (n = 36), continuity of care (n = 6), efficiency and productivity (n = 45). Two-thirds of the studies that assessed productivity and efficiency performed an economic analysis (n = 27). The quality and rigour of evaluations varied substantially across studies, with two-thirds of all studies measuring and reporting outcomes from only one or two of these domains.

CONCLUSIONS

There are a growing number of studies measuring the performance of non-medical healthcare professional substitution roles. Few have been subject to robust evaluations, and there is limited evidence on the scientific rigour and adequacy of outcomes measured. A systematic and coordinated approach is required to support healthcare settings in assessing the value of non-medical role substitution healthcare delivery models.

Setting up clinical audit in a psychodynamic psychotherapy service: a pilot study

Aims and Method

Clinical governance implies a need to engage in a demonstrable form of clinical audit. We decided to pilot a pre–post-therapy questionnaire study, involving both therapists and patients, with the aim of assessing its feasibility as a routine measure of outcome in our service. Questionnaires were chosen to reflect both the symptom profile Clinical Outcomes in Routine Evaluation (CORE) and the general level of functioning Global Assessment of Functioning Scale (GAF). The patients were also questioned about their satisfaction with the therapy.

Results

Of 53 eligible patient–therapist pairs, 26 patients and 19 therapists responded pre-treatment (overall 51% response rate). The mean (s.d.) CORE score per item was 1.93 (0.78), whereas the GAF score was 55 (15.2), somewhat belying the idea that psychotherapists only treat the ‘worried well’. The post-therapy response rate was poorer, rendering statistical analysis of change difficult to perform. The trend, however, was towards an improvement in both symptoms and level of functioning over the course of therapy.

Clinical Implications

Our patient group presents with a considerable degree of impairment. Although the response rates in this pilot study were poor, the data trend suggests that patients do benefit from the treatment offered. The aims of the study were met in that implementation of a clinical audit process within our department is feasible and the data are obtainable. The main interest, however, lies in what we learned from the process. We offer a dynamic and practical assessment of the impediments encountered, with implications for how these might be confronted.

Improving the quality of mental health services using patient outcome data: making the most of HoNOS

Efforts to assess and improve the quality of mental health services are often hampered by a lack of information on patient outcomes. Most mental health services in England have been routinely collecting Health of the Nation Outcome Scales (HoNOS) data for some time. In this article we illustrate how clinical teams have used HoNOS data to identify areas where performance could be improved. HoNOS data have the potential to give clinical teams the information they need to assess the quality of care they deliver, as well as develop and test initiatives aimed at improving the services they provide.

Effective patient–clinician interaction to improve treatment outcomes for patients with psychosis: a mixed-methods design

Background

At least 100,000 patients with schizophrenia receive care from community mental health teams (CMHTs) in England. These patients have regular meetings with clinicians, who assess them, engage them in treatment and co-ordinate care. As these routine meetings are not commonly guided by research evidence, a new intervention, DIALOG, was previously designed to structure consultations. Using a hand-held computer, clinicians asked patients to rate their satisfaction with eight life domains and three treatment aspects, and to indicate whether or not additional help was needed in each area, with responses being graphically displayed and compared with previous ratings. In a European multicentre trial, the intervention improved patients’ quality of life over a 1-year period. The current programme builds on this research by further developing DIALOG in the UK.

Research questions

(1) How can the practical procedure of the intervention be improved, including the software used and the design of the user interface? (2) How can elements of resource-oriented interventions be incorporated into a clinician manual and training programme for a new, more extensive ‘DIALOG+’ intervention? (3) How effective and cost-effective is the new DIALOG+ intervention in improving treatment outcomes for patients with schizophrenia or a related disorder? (4) What are the views of patients and clinicians regarding the new DIALOG+ intervention?

Methods

We produced new software on a tablet computer for CMHTs in the NHS, informed by analysis of videos of DIALOG sessions from the original trial and six focus groups with 18 patients with psychosis. We developed the new ‘DIALOG+’ intervention in consultation with experts, incorporating principles of solution-focused therapy when responding to patients’ ratings and specifying the procedure in a manual and training programme for clinicians. We conducted an exploratory cluster randomised controlled trial with 49 clinicians and 179 patients with psychosis in East London NHS Foundation Trust, comparing DIALOG+ with an active control. Clinicians working as care co-ordinators in CMHTs (along with their patients) were cluster randomised 1 : 1 to either DIALOG+ or treatment as usual plus an active control, to prevent contamination. Intervention and control were to be administered monthly for 6 months, with data collected at baseline and at 3, 6 and 12 months following randomisation. The primary outcome was subjective quality of life as measured on the Manchester Short Assessment of Quality of Life; secondary outcomes were also measured. We also established the cost-effectiveness of the DIALOG intervention using data from the Client Service Receipt Inventory, which records patients’ retrospective reports of using health- and social-care services, including hospital services, outpatient services and medication, in the 3 months prior to each time point. Data were supplemented by the clinical notes in patients’ medical records to improve accuracy. We conducted an exploratory thematic analysis of 16 video-recorded DIALOG+ sessions and measured adherence in these videos using a specially developed adherence scale. We conducted focus groups with patients (n = 19) and clinicians (n = 19) about their experiences of the intervention, and conducted thematic analyses. We disseminated the findings and made the application (app), manual and training freely available, as well as producing a protocol for a definitive trial.

Results

Patients receiving the new intervention showed more favourable quality of life in the DIALOG+ group after 3 months (effect size: Cohen’sd = 0.34), after 6 months (Cohen’sd = 0.29) and after 12 months (Cohen’sd = 0.34). An analysis of video-recorded DIALOG+ sessions showed inconsistent implementation, with adherence to the intervention being a little over half of the possible score. Patients and clinicians from the DIALOG+ arm of the trial reported many positive experiences with the intervention, including better self-expression and improved efficiency of meetings. Difficulties reported with the intervention were addressed by further refining the DIALOG+ manual and training. Cost-effectiveness analyses found a 72% likelihood that the intervention both improved outcomes and saved costs.

Limitations

The research was conducted solely in urban east London, meaning that the results may not be broadly generalisable to other settings.

Conclusions

(1) Although services might consider adopting DIALOG+ based on the existing evidence, a definitive trial appears warranted; (2) applying DIALOG+ to patient groups with other mental disorders may be considered, and to groups with physical health problems; (3) a more flexible use with variable intervals might help to make the intervention even more acceptable and effective; (4) more process evaluation is required to identify what mechanisms precisely are involved in the improvements seen in the intervention group in the trial; and (5) what appears to make DIALOG+ effective is that it is not a separate treatment and not a technology that is administered by a specialist; rather, it changes and utilises the existing therapeutic relationship between patients and clinicians in CMHTs to initiate positive change, helping the patients to improve their quality of life.

Future research

Future studies should include a definitive trial on DIALOG+ and test the effectiveness of the intervention with other populations, such as people with depression.

Trial registration

Current Controlled Trials ISRCTN34757603.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

Intensive case management for high-risk patients with first-episode psychosis: service model and outcomes

BACKGROUND

The first episode of psychosis is a crucial period when early intervention can alter the trajectory of the young person's ongoing mental health and general functioning. After an investigation into completed suicides in the Early Psychosis Prevention and Intervention Centre (EPPIC) programme, the intensive case management subprogramme was developed in 2003 to provide assertive outreach to young people having a first episode of psychosis who are at high risk owing to risk to self or others, disengagement, or suboptimal recovery. We report intensive case management model development, characterise the target cohort, and report on outcomes compared with EPPIC treatment as usual.

METHODS

Inclusion criteria, staff support, referral pathways, clinical review processes, models of engagement and care, and risk management protocols are described. We compared 120 consecutive referrals with 50 EPPIC treatment as usual patients (age 15-24 years) in a naturalistic stratified quasi-experimental real-world design. Key performance indicators of service use plus engagement and suicide attempts were compared between EPPIC treatment as usual and intensive case management, and psychosocial and clinical measures were compared between intensive case management referral and discharge.

FINDINGS

Referrals were predominately unemployed males with low levels of functioning and educational attainment. They were characterised by a family history of mental illness, migration and early separation, with substantial trauma, history of violence, and forensic attention. Intensive case management improved psychopathology and psychosocial outcomes in high-risk patients and reduced risk ratings, admissions, bed days, and crisis contacts.

INTERPRETATION

Characterisation of intensive case management patients validated the clinical research focus and identified a first episode of psychosis high-risk subgroup. In a real-world study, implementation of an intensive case management stream within a well-established first episode of psychosis service showed significant improvement in key service outcomes. Further analysis is needed to determine cost savings and effects on psychosocial outcomes. Targeting intensive case management services to high-risk patients with unmet needs should reduce the distress associated with pathways to care for patients, their families, and the community.

FUNDING

National Health & Medical Research Council and the Colonial Foundation.

Clinical practitioner's attitudes towards the use of Routine Outcome Monitoring within Child and Adolescent Mental Health Services: a qualitative study of two Child and Adolescent Mental Health Services

Routine Outcome Monitoring (ROM) is held as a greatly important part of practice across many Health Care Services, both in the NHS and in private practice. Yet despite this, there has been little research into the attitudes of practitioners towards ROM. This paper looks at the attitudes of 50 clinicians from two Child and Adolescent Mental Health Services in greater London. The findings showed that although the practitioners were not overwhelming positive in their attitudes to ROM, neither were they overwhelming negative, and many of their concerns involved practical issues surrounding ROM that are potentially soluble. Practitioner engagement in ROM is key if ROM is to be used constructively to reflect on practice.

Innovations in Practice: Feasibility of the development and well-being assessment as an adjunct to clinical assessment in child and adolescent mental health services

BACKGROUND

Child and Adolescent Mental Health services (CAMHS) might benefit from the use of structured diagnostic assessments as an adjunct to clinical assessment. Such assessments will only support clinical practice if their completion avoids too great a burden to parents and services, and if the resulting information is useful to practitioners.

METHOD

Parents were asked to complete the Development And Well-Being Assessment (DAWBA) before their initial appointment at a community CAMHS, and DAWBAs were disclosed to the assessing practitioners in a random half of cases. Parents and Practitioners were asked to complete a questionnaire about their experience of the DAWBA. Parents completed the experience of services questionnaire 6 months after the baseline.

RESULTS

Most parents found the interview easy to understand. Many reported that the experience of completing the interview changed the way that they thought about their child's difficulties in a positive manner. Practitioner reports were also mainly positive. The mean helpfulness score adjusted for the clustering of cases within practitioners out of 1-5 for very unhelpful to very helpful was 4.04 (95% Confidence Interval: 3.89-4.18). There was no association between practitioner access to the DAWBA and parent reported satisfaction on the Experiences of Services Questionnaire (mean difference 0.74, 95% confidence interval -0.59-02.06, p0.27).

CONCLUSIONS

With the right supporting arrangements in place, the DAWBA would be a feasible assessment tool in community CAMHS.

Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers

BACKGROUND

Many patient-reported outcome measures have been developed in the past two decades, playing an increasingly important role in palliative care. However, their routine use in practice has been slow and difficult to implement.

AIM

To systematically identify facilitators and barriers to the implementation of patient-reported outcome measures in different palliative care settings for routine practice, and to generate evidence-based recommendations, to inform the implementation process in clinical practice.

DESIGN

Systematic literature review and narrative synthesis.

DATA SOURCES

Medline, PsycInfo, Cumulative Index to Nursing and Allied Health Literature, Embase and British Nursing Index were systematically searched from 1985. Hand searching of reference lists for all included articles and relevant review articles was performed.

RESULTS

A total of 3863 articles were screened. Of these, 31 articles met the inclusion criteria. First, data were integrated in the main themes: facilitators, barriers and lessons learned. Second, each main theme was grouped into either five or six categories. Finally, recommendations for implementation on outcome measures at management, health-care professional and patient levels were generated for three different points in time: preparation, implementation and assessment/improvement.

CONCLUSIONS

Successful implementation of patient-reported outcome measures should be tailored by identifying and addressing potential barriers according to setting. Having a coordinator throughout the implementation process seems to be key. Ongoing cognitive and emotional processes of each individual should be taken into consideration during changes. The educational component prior to the implementation is crucial. This could promote ownership and correct use of the measure by clinicians, potentially improving practice and the quality of care provided through patient-reported outcome measure data use in clinical decision-making.

The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review

BACKGROUND

Allied Health Professionals today are required, more than ever before, to demonstrate their impact. However, despite at least 20 years of expectation, many services fail to deliver routine outcome measurement in practice. This systematic review investigates what helps and hinders routine outcome measurement of allied health professionals practice.

METHODS

A systematic review protocol was developed comprising: a defined search strategy for PsycINFO, MEDLINE and CINHAL databases and inclusion criteria and systematic procedures for data extraction and quality appraisal. Studies were included if they were published in English and investigated facilitators and/or barriers to routine outcome measurement by allied health professionals. No restrictions were placed on publication type, design, country, or year of publication. Reference lists of included publications were searched to identify additional papers. Descriptive methods were used to synthesise the findings.

RESULTS

960 papers were retrieved; 15 met the inclusion criteria. Professional groups represented were Physiotherapy, Occupational Therapy, and Speech and Language Therapy. The included literature varied in quality and design. Facilitators and barriers to routine outcome measurement exist at individual, managerial and organisational levels. Key factors affecting professionals' use of routine outcome measurement include: professionals' level of knowledge and confidence about using outcome measures, and the degree of organisational and peer-support professionals received with a view to promoting their work in practice.

CONCLUSIONS

Whilst the importance of routinely measuring outcomes within the allied health professions is well recognised, it has largely failed to be delivered in practice. Factors that influence clinicians' ability and desire to undertake routine outcome measurement are bi-directional: they can act as either facilitators or barriers. Routine outcome measurement may only be deliverable if appropriate action is taken at individual therapist, team, and organisational levels of an organisation.

An evaluation of the implementation of the FACE Health and Social Assessment in routine psychiatric practice

BACKGROUND

Systematic assessment of patients is seen as an important aspect of mental health care. As yet few UK services have successfully implemented outcome measures.

AIMS

To examine the feasibility of using FACE Health and Social Assessment in everyday practice and explore issues raised for staff.

METHOD

This comprised a feasibility questionnaire completed by 52 staff, 3 focus groups involving 18 staff and an analysis of patient scores on routinely collected FACE data.

RESULT

The questionnaire findings were positive yet only just over half of the patients on Assertive Outreach (AO) teams had some part of FACE completed in the previous year. The focus groups drew attention to many other calls on staff time and the importance of ongoing support. Staff were uncertain about the benefits of FACE for measuring change but believed that it promoted reflective practice. FACE subscores showed some ability to discriminate between patients on Rehabilitation and Recovery and AO teams but there were no significant differences for AO patients over time.

CONCLUSIONS

Although competing demands and low priority may underpin the failure to implement FACE it is perhaps clinical staffs lack of conviction in its contribution to improving mental health care that is the fundamental issue.

Engagement and early termination of contact with a community-based early intervention service for personality disorder in young adults

BACKGROUND

Few studies have examined factors associated with continuity of care in a community-based early intervention service for personality disorder in patients aged 16-25.

AIMS

To estimate the probability of dropping out of care in patients attending an early intervention service for personality disorder and identify patient characteristics associated with those who drop out, are discharged or continue using the service.

METHOD

A 24-month cohort of first-contact patients attending the early intervention service was followed up for 12 months to identify drop-outs, discharges and those still using the service.

RESULTS

One hundred eighty-three first contact patients were referred/self-referred during the study timescale. After 12-month follow-up, 83 (45%) were discharged, 39 (21%) still using the service and 61 (33%) dropped out. Drop out was most likely among patients aged 21-25, from higher socio-economic groups and highest during months 3-5 of service use. No discharges occurred until at least 3 months into the service, with a peak at 6-8 months.

CONCLUSIONS

Although sharing many factors predicting increased levels of drop-out, the rate of drop-out among young adults aged between 16-25 attending an early intervention service for personality disorder was equivalent to that experienced by services for adults with a long history of personality disorder diagnosis. Concerns exist, however, concerning increased rates of drop-out among patients reporting a greater number of difficulties, reporting a common mental health or substance abuse problem, and during months 3-5 of service use. Implications of the results for future service developments are discussed.

Do patients improve in general psychiatric outpatient care? Problem severity among patients and the effectiveness of a psychiatric outpatient unit

This study evaluated a routine psychiatric outpatient unit that admitted a variety of diagnoses and was staffed with a multi-professional team. The aim was to describe the problem severity among patients of a routine psychiatric outpatient unit in Sweden and to evaluate the statistical and clinical effectiveness of routine care assessed at the termination of treatment. Newly admitted patients were diagnosed according to the ICD-10 and completed questionnaires at the beginning and end of their treatment regarding symptoms (Brief Symptom Inventory) and interpersonal problems (Inventory of Interpersonal Problems); 98.6% of the patients belonged to the dysfunctional group at pre-treatment, showing that there were considerable sufferings among new patients. There were significant improvements in most of the variables. The effect size regarding symptoms (Global Severity Index) was d=0.76 and 0.43 in the interpersonal total score. No difference between different diagnostic groups was found. As for the clinical significant change, the results showed that 64.5% of the patients were improved regarding symptom remission but also that a considerable number of patients (60.5%) still belonged to the dysfunctional group after the treatment. The study indicates that the effectiveness of a routine psychiatric outpatient practice seems to be similar to results obtained from speciality research clinics and randomized controlled trial studies. However, the results also show that there is a considerable amount of patients still in the dysfunctional group after the treatment, a fact that implies that further improvements of the treatment could be made.

Introduction of routine outcome measures: staff reactions and issues for consideration

The aim of this study was to explore clinician reactions to (i) the introduction of routine outcome measures and (ii) the utility of outcomes data in clinical practice. Focus group discussions (n = 34) were conducted with mental health staff (n = 324) at approximately 8 months post implementation of routine outcome measures. A semi-structured interview schedule was used to collect data on two key issues; reactions to the introduction of outcome measures and factors influencing the utility of outcomes data in clinical practice. Data from the discussion groups were analysed using content analysis to isolate emerging themes. While the majority of participants endorsed the collection and utilization of outcomes data, many raised questions about the merits of the initiative. Ambivalence, competing work demands, lack of support from senior medical staff, questionable evidence to support the use of outcome measures, and fear of how outcomes data might be used emerged as key issues. At 8 months post implementation a significant number of clinical staff remained ambivalent about the benefits of outcome measurement and had not engaged in the process. The shift to a service model driven by outcomes and case-mix data will take time and resources to achieve. Implications for nursing staff are discussed.

Routine outcome monitoring in clinical practice: service and non-service costs of psychiatric patients attending a Community Mental Health Centre in Italy

OBJECTIVE

This study estimated service and non-service costs in a random sample of patients attending a community psychiatric mental health centre in Italy.

METHODS

A randomly selected sample of patients identified during 1 week of routine clinical activity was enrolled. Information was collected using the Italian-language checklist Questionario Economico per l'attivita clinica dei Servizi Psichiatrici (QESP). Costs were classified in two categories: service costs (from the provision of services) and non-service costs (loss of productivity and informal care).

RESULTS

One hundred and twenty patients were included. Patients suffering from schizophrenia and related disorders (n = 50) had a mean age of 44.5 years (S.D. 12.9), and patients with other diagnoses (n = 70) had a mean age of 44.9 years (S.D. 14.8). More patients in the schizophrenia group were single, living alone, had a low educational level and a higher unemployment rate. In patients suffering from schizophrenia, the monthly service cost per patient was nearly double that for patients with other diagnoses. Non-service costs associated with patients' lack of job opportunities were more than three times higher for patients with schizophrenia. Non-service costs associated with patients' and care-givers' time off work were similar in the two groups. In the multivariate analysis, being unemployed was associated with higher service costs. Younger age, length of illness and diagnosis of schizophrenia were determinants of higher non-service costs. The latter three independent variables were also associated with overall (service and non-service) costs.

CONCLUSION

The present study estimated service and non-service costs under routine circumstances to provide information on costs that community psychiatric services, patients and care-givers sustain when dealing with psychiatric problems.

Outcome measurement in pharmacological trials: validity of the Routine Assessment of Patient Progress (RAPP)

The assessment of outcomes after treatment with antipsychotic medication is fundamental to clinical care and research. The Routine Assessment of Patient Progress (RAPP) is a reliable multidimensional scale that employs nurses' ratings of symptoms and functioning in psychiatric inpatients. The present study sought to extend validity evidence for the RAPP by examining its ability to reflect changes associated with treatment by antipsychotic medications. The use of a different sample in this study also provided the opportunity to replicate earlier validity data collected on the original set of patients. Ninety-seven separate trials were conducted, involving 65 consecutive admissions to a unit that specializes in the assessment and treatment of patients with long standing severe psychiatric disorders. The RAPP, along with the Positive and Negative Syndrome Scale and global measures of severity, were administered at baseline and at the end of each trial. Both factor scores and clinically-derived subscales were analysed for sensitivity to change. Patients were globally rated as improved, unchanged or worsened at the end of the medication trial. Results indicated that the RAPP factor, clinical scale and total scores compared favourably to other outcome measures in patients rated as improved or worse. In patients rated as unchanged, RAPP scores displayed significantly less change than did the PANSS scores. These findings support the validity of the RAPP as an outcome measure in treatment trials.

Setting up clinical audit in a psychodynamic psychotherapy service: a pilot study

Aims and Method

Clinical governance implies a need to engage in a demonstrable form of clinical audit. We decided to pilot a pre–post-therapy questionnaire study, involving both therapists and patients, with the aim of assessing its feasibility as a routine measure of outcome in our service. Questionnaires were chosen to reflect both the symptom profile Clinical Outcomes in Routine Evaluation (CORE) and the general level of functioning Global Assessment of Functioning Scale (GAF). The patients were also questioned about their satisfaction with the therapy.

Results

Of 53 eligible patient–therapist pairs, 26 patients and 19 therapists responded pre-treatment (overall 51% response rate). The mean (s.d.) CORE score per item was 1.93 (0.78), whereas the GAF score was 55 (15.2), somewhat belying the idea that psychotherapists only treat the ‘worried well’. The post-therapy response rate was poorer, rendering statistical analysis of change difficult to perform. The trend, however, was towards an improvement in both symptoms and level of functioning over the course of therapy.

Clinical Implications

Our patient group presents with a considerable degree of impairment. Although the response rates in this pilot study were poor, the data trend suggests that patients do benefit from the treatment offered. The aims of the study were met in that implementation of a clinical audit process within our department is feasible and the data are obtainable. The main interest, however, lies in what we learned from the process. We offer a dynamic and practical assessment of the impediments encountered, with implications for how these might be confronted.

Outcome measures and needs assessment tools for schizophrenia and related disorders

BACKGROUND

There has been a recent trend to encourage routine outcome measurement and needs assessment as an aid to decision making in clinical practice and patient care. Standardised instruments have been developed which measure clinical symptoms of disorders such as schizophrenia, wider health related quality of life and patients' needs. Such measures might usefully be applied to aid the recognition of psychosocial problems and to monitor the course of patients' progress over time in terms of disease severity and associated deficits in health related quality of life. They might also be used to help clinicians to make decisions about treatment and to assess subsequent therapeutic impact. Such an approach is not, however, without cost and the actual benefit of the adoption of routine outcome and needs assessment in the day-to-day care of those with schizophrenia remains unclear.

OBJECTIVES

To establish the value of the routine administration of outcome measures and needs assessment tools and the feedback they provide in improving the management and outcome of patients with schizophrenia and related disorders.

SEARCH STRATEGY

The reviewers undertook electronic searches of the British Nursing Index (1994 to Sept 1999), the Cochrane Library (Issue 2, 2002), the Cochrane Schizophrenia Group Trials Register (2002), EMBASE (1980-2002), MEDLINE (1966-2002), and PsycLIT (1887-2002), together with hand searches of key journals. References of all identified studies were searched for further trials, and the reviewers contacted authors of trials.

SELECTION CRITERIA

Randomised controlled trials comparing the feedback of routine standardised outcome measurement and needs assessment, to routine care for those with schizophrenia.

DATA COLLECTION AND ANALYSIS

Reviewers evaluated data independently. Studies which randomised clinicians or clinical teams (rather than individual patients) were considered to be the most robust. However only those which took account of potential clustering effects were considered further. Where possible and appropriate, risk ratios (RR) and their 95% confidence intervals (CI) were calculated. For continuous data Weighted Mean Differences (WMD) were calculated. Data were inspected for heterogeneity.

MAIN RESULTS

No randomised data were found which addressed the specified objectives. One unpublished and one ongoing trial was identified.

REVIEWER'S CONCLUSIONS

The routine use of outcomes measures and needs assessment tools is, as yet, unsupported by high quality evidence of clinical and cost effectiveness. Clinicians, patients and policy makers alike may wish to see randomised evidence before this strategy is routinely adopted.

What outcomes to measure in routine mental health services, and how to assess them: a systematic review

OBJECTIVE

Routine outcome assessment in adult mental health services involves the on-going assessment of patient-level outcomes. Use of outcomes to inform treatment is widely recommended, but seldom implemented. The goals of this review were (i) to identify principles that have been proposed for implementing routine outcome assessment, (ii) to identify the full range of outcome domains that have been proposed for assessment, and (iii) to synthesize proposals for specific outcome domains into emergent categories.

METHOD

A systematic review of published and unpublished research was undertaken, using electronic databases, research registers, conference proceedings, expert informants and the World Wide Web. For goal (i) studies were included that proposed principles for implementing routine outcome assessment. For goal (ii) studies were included that identified at least two patient-level outcome domains for patients using adult mental health services and made some reference to a broader literature base.

RESULTS

Six thousand four hundred publications matched initial search criteria. Seven distinct sets of principles for choosing patient-level outcomes were located, which showed a fair degree of consensus. Sixteen outcome domain proposals were identified, which were synthesized into seven emergent categories: wellbeing, cognition/emotion, behaviour, physical health, interpersonal, society and services.

CONCLUSIONS

The findings from this review were used to develop a four-step method for adult mental health services wishing to implement routine outcome assessment.

Clinical databases of patients receiving antidepressants. The missing link between research and practice?

BACKGROUND

In the last 10 years the use of antidepressants has increased drastically. Unfortunately, the epidemiology of these compounds has shown significant gaps between recommendations derived from randomised controlled trials and current clinical practice.

METHODS

We argue for the need to develop and maintain clinical databases of patients receiving antidepressants as a way of bridging this situation.

RESULTS

In addition to experimental data generated in selected patients and settings, observational databases of large cohorts of typical patients, followed in typical settings, should be developed and maintained. Clinical databases could collect information on patient social and demographic characteristics, clinical symptoms, diagnosis and pharmacological and non-pharmacological treatments. In addition, they can provide accurate estimates of probabilities of different outcomes and on factors that affect outcome.

CONCLUSION

Clinical databases should not be seen as another expensive administrative task for busy doctors. Clinical databases should be developed, organised and utilised only by clinicians who are interested in monitoring their clinical practice and want to provide patients, relatives and the public with information on prognosis and outcome in their specific context of care. Maintaining clinical databases is a routine process, nested in everyday clinical activity, which aims at constituting a permanent link between research and practice.

The impact of routine outcome measurement on treatment processes in community mental health care: approach and methods of the MECCA study

Three issues characterise the background to the MECCA study: A) Throughout Europe, most patients with severe forms of psychotic disorders are cared for in the community. The challenge now is to make processes in community mental health care more effective. B) There are widespread calls to implement regular outcome measurement in routine settings. This, however, is more likely to happen, if it provides a direct benefit to clinicians and patients. C) Whilst user involvement is relatively easy to achieve on a political level, new mechanisms may have to be established to make the views of patients feed into individual treatment decisions. The MECCA study is a cluster randomised controlled trial following the same protocol in community mental health teams in six European countries. In the experimental group, patients' subjective quality of life, treatment satisfaction and wishes for different or additional help are assessed in key worker-patient meetings every two months and intended to inform the therapeutic dialogue and treatment decisions. The trial tests the hypothesis that the intervention--as compared to current best standard practice--will lead to a better outcome in terms of quality of life and other criteria in patients with psychotic disorders over a one year period. This more favourable outcome is assumed to be mediated through different treatment input based on more appropriate joint decisions or a more positive therapeutic relationship in line with a partnership model of care or both. Moreover, the study will hopefully reveal new insights into how therapeutic processes in community mental health care work and how they can be optimised.

Monitoring community psychiatric services in Italy: differences between patients who leave care and those who stay in treatment

BACKGROUND

Continuity of care has been monitored rarely in Italian community mental health centres.

AIMS

To estimate the long-term probability of leaving care in first-contact patients attending an out-patient service, and to identify patients most likely to drop out.

METHOD

All patients who had a first contact with the community mental health centre of Magenta during a 1-year recruitment period were followed up for 24 months. Patients who failed to return after the last out-patient visit were regarded as 'drop-outs'.

RESULTS

During the 1-year recruitment period 330 subjects were at their first contact. The 1-year incidence of first-contact patients was nearly 33 per 10 000 inhabitants. At follow-up, 46% of patients had dropped out. In comparison with patients with psychoses, subjects suffering from neurotic (P =0.004) and personality disorders (P=0.029) were more likely to drop out.

CONCLUSIONS

In the Italian system of community psychiatric care nearly half of the patients are no longer in contact after 2 years. Those who stay in treatment are more likely to suffer from psychosis, suggesting a commitment of Italian out-patient facilities to tackling the needs of patients with more severe disorders.

The use of patient-level outcomes to inform treatment

OBJECTIVE

The assessment of outcome for any purpose is not undertaken routinely in European mental health services. This paper discusses the merits of using outcome data to inform the planning of mental health care for individual patients, and provides practical advice to support the implementation of this new approach to working.

METHOD

The use of outcomes in North America and Europe is briefly reviewed. A conceptual basis is proposed for routine outcome assessment--the ongoing measurement and use of outcome data to inform decisions about whether to continue, change or curtail treatment. A cognitive psychology model is developed which indicates that the routine use of outcomes will improve mental health care. Perceived problems with routine outcome assessment are discussed, and principles for implementation are identified.

RESULTS

Outcomes are used mainly for generating local-level (rather than patient-level) data in North America, and rarely used in Europe. The use of outcome data routinely may facilitate reflective clinical practice, a model of decision-making which leads to a higher quality of clinical care than automated problem-solving. One issue relates to the use of standardised assessments designed for research purposes in clinical settings, and this is being addressed through the development of a new generation of outcome measures which are explicitly designed for clinical use. However, most clinicians remain unconvinced of the benefits of routine outcome assessment, and relevant research is currently underway across Europe which will address this concern. Scientific principles to maximise quality and pragmatic principles to maximise the chances of successful implementation are identified.

CONCLUSIONS

The routine use of outcomes will become increasingly prominent in European mental health services. This provides clinicians with an opportunity to improve the quality of clinical care offered to patients.

Routine outcome measurement in a child and adolescent mental health service: an evaluation of HoNOSCA. The Health of the Nation Outcome Scales for Children and Adolescents

OBJECTIVE

This paper evaluates a range of properties for a clinician-based instrument designed for routine use in a child and adolescent mental health service (CAMHS).

METHOD

The Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA) is a new outcome measure with great promise. Case vignettes were used to examine interrater reliability. HoNOSCA was implemented for routine outpatient use by multidisciplinary staff with a return rate of 84%. The 305 ratings obtained at assessment were analysed by age, gender and diagnosis. A sample of 145 paired ratings with a 3-month interval were examined for the measurement of change over time.

RESULTS

Interrater reliability of the total score indicates moderate reliability if absolute scores are used and good reliability if the total score is used for relative comparisons. Most scales have good to very good reliability. The scales discriminated between age and gender in the expected way. HoNOSCA correlated with clinicians' views of change and was sensitive to change over a 3-month period. The total score seemed a proxy for severity.

CONCLUSION

Routine outcome instruments must be explored in settings where they will be used and with realistic training. HoNOSCA appears to be of value in routine outcome measurement and although questions remain about reliability and validity, the results strongly support further investigation.

Computer aids to mental health care

Computer systems are being used increasingly to aid the assessment and self-treatment of mental health problems in adults. Systems vary hugely in the extent to which they meet all patients' assessment and therapy needs and save clinicians' time. Hardly any single system 1) performs every task required from initial screening to the end of follow-up, 2) works 100% independently of contact with a clinician or technician, and 3) is widely available and supported. Most systems use desk- or laptop computers. Some now use palmtop (hand-held) computers. A few employ computerized phone interviews (interactive voice response), usually from home. Virtual reality as a tool is embryonic. Computer aids save time in screening and outcome-tracking in a wide variety of problems. Computer aids to treatment have had promising outcomes in phobic, anxiety, panic, and obsessive-compulsive disorders, nonsuicidal depression, obesity, and smoking cessation. Some systems are installed in a few places as part of everyday clinical care. A growing number should soon be robust enough to ease the lives of many patients, practitioners, and researchers if use of the systems is carefully integrated into normal clinical practice.