Quality of mental health service care: the forgotten pathway from process to outcome

Excellence in forensic psychiatry services: international survey of qualities and correlates

BACKGROUND

Excellence is that quality that drives continuously improving outcomes for patients. Excellence must be measurable. We set out to measure excellence in forensic mental health services according to four levels of organisation and complexity (basic, standard, progressive and excellent) across seven domains: values and rights; clinical organisation; consistency; timescale; specialisation; routine outcome measures; research and development.

AIMS

To validate the psychometric properties of a measurement scale to test which objective features of forensic services might relate to excellence: for example, university linkages, service size and integrated patient pathways across levels of therapeutic security.

METHOD

A survey instrument was devised by a modified Delphi process. Forensic leads, either clinical or academic, in 48 forensic services across 5 jurisdictions completed the questionnaire.

RESULTS

Regression analysis found that the number of security levels, linked patient pathways, number of in-patient teams and joint university appointments predicted total excellence score.

CONCLUSIONS

Larger services organised according to stratified therapeutic security and with strong university and research links scored higher on this measure of excellence. A weakness is that these were self-ratings. Reliability could be improved with peer review and with objective measures such as quality and quantity of research output. For the future, studies are needed of the determinants of other objective measures of better outcomes for patients, including shorter lengths of stay, reduced recidivism and readmission, and improved physical and mental health and quality of life.

Clinical practitioner's attitudes towards the use of Routine Outcome Monitoring within Child and Adolescent Mental Health Services: a qualitative study of two Child and Adolescent Mental Health Services

Routine Outcome Monitoring (ROM) is held as a greatly important part of practice across many Health Care Services, both in the NHS and in private practice. Yet despite this, there has been little research into the attitudes of practitioners towards ROM. This paper looks at the attitudes of 50 clinicians from two Child and Adolescent Mental Health Services in greater London. The findings showed that although the practitioners were not overwhelming positive in their attitudes to ROM, neither were they overwhelming negative, and many of their concerns involved practical issues surrounding ROM that are potentially soluble. Practitioner engagement in ROM is key if ROM is to be used constructively to reflect on practice.

Community and hospital residential care: a comparative evaluation

Objective: To compare the quality of care offered by a community hospital hostel and three hospital rehabilitation facilities (two traditional rehabilitation wards and an innovative normalisation unit) for people with longterm mental illnesses.

Method: Quality of care is assessed here on three different levels: those of Input using Programme Analysis of Service Systems; Assessment of Care Environments; Process using Ward Management Practices Questionnaire; Attitudes to Treatment Questionnaire; Outcome using Rehabilitation Evaluation Hall and Baker and the Life Experiences Checklist and resident and staff questionnaires. These measures cover a range of perspectives from staff to residents, and include both standardised assessments as well as specific schedules developed for the study.

Results: On Input measures, the community hospital hostel had the best scores on the Programme Analysis of Service Systems schedule, which measures the degree to which services meet predetermined normalisation criteria. On the Assessment of Care Environments it also scored favourably against other community facilities. Process measures showed no differences between units in terms of their management practices, all scoring well, but suggested some differences in staff attitudes. Staff in the community hospital hostel had the most medical approach to care, however this was accounted for by the scores of untrained staff. Qualified nurses had a more psychological approach to care. Finally on Output measures, residents in the community hospital hostel were found to be the most disabled on the REHAB scale. Despite this, they had a significantly better quality of life as assessed by the Life Experiences Checklist. Staff in the community hospital hostel had the clearest perceptions of their roles, though there were differences again in how qualified and unqualified staff perceived their work. Residents were generally satisfied with services, though residents in the community hospital hostel and in the hospital normalisation unit had the highest satisfaction levels.

Conclusion: The results of this comparative evaluation show that a high standard of care, equal to or surpassing some of the best hospital provision, can be provided in the community. This is despite the fact that the residents in the community hospital hostel were more disabled. Community patients' quality of life is better in a number of domains than their hospital counterparts and even patients initially reticent about the move into the community report higher levels of satisfaction, especially regarding their home environment. There-were interesting differences between trained and unqualified staff in the community hospital hostel. Trained staff had a more psychological view of patient care and felt more supported and appreciated by the team than their untrained colleagues. The implications of these findings for community residential care are discussed.

What is the association between quality of treatment for depression and patient outcomes? A cohort study of adults consulting in primary care

BACKGROUND

The association between adequate treatment for a Major Depressive Episode (MDE) and improvements in depressive symptoms is not well established in naturalistic practice conditions. The main objective of this study was to examine the association between receiving at least one minimally adequate treatment for MDE (i.e. according to clinical guidelines) in the previous 12 months and evolution of depressive symptoms at 6- and 12-months. Associations with receiving pharmacotherapy and/or psychotherapy and the role of severity of depression were examined.

METHODS

This cohort study included 908 adults meeting criteria for previous-year MDE and consulting at one of 65 primary care clinics in Quebec, Canada. Multilevel analyses were performed.

RESULTS

Results show that (i) receiving at least one minimally adequate treatment for depression was associated with greater improvements in depression symptoms at 6 and at 12 months; (ii) adequate pharmacotherapy and adequate psychotherapy were both associated with greater improvements in depression symptoms, and (iii) the association between adequate treatment and improvement in depression symptoms varied as a function of severity of symptoms at the time of inclusion in the cohort with worse symptoms at the time of inclusion being associated with greater reductions at 6 and 12 months.

LIMITATIONS

Measures are self-reported. Participants were recruited at different stages over the course of their MDE.

CONCLUSIONS

This study shows that adequate treatment for depression is associated with improvements in depressive symptoms in naturalistic primary care practice conditions, but that those with more severe depressive symptoms are more likely to receive adequate treatment and improve across time.

Quality of care for major depression and its determinants: a multilevel analysis

BACKGROUND

Numerous studies highlight an important gap in the quality of care for depression in primary care. However, basic indicators were often used. Few of these studies examined factors associated with receiving adequate treatment, particularly with a simultaneous consideration of individual and organizational characteristics. The purpose of this study was to estimate the proportion of primary care patients with a major depressive episode (MDE) who receive adequate treatment and to examine the individual and organizational (i.e., clinic-level) characteristics associated with the receipt of at least one minimally adequate treatment for depression.

METHODS

The sample used for this study included 915 adults consulting a general practitioner (GP), regardless of the motive of consultation, meeting DSM-IV criteria for MDE during the 12 months preceding the survey (T1), and nested within 65 primary care clinics. Data reported in this study were obtained from the "Dialogue" project. Adherence rates for 27 quality indicators selected to cover the most important components of depression treatment were estimated. Multilevel analyses were conducted.

RESULTS

Adherence to guidelines was high (>75%) for one third of the quality indicators that were measured but was low (<60%) for nearly half of the measures. Just over half of the sample (52.2%) received at least one minimally adequate treatment for depression. At the individual level, determinants of receipt of minimally adequate care included age, having a family physician, a supplementary insurance coverage, a comorbid anxiety disorder and the severity of depression. At the clinic level, determinants included the availability of psychotherapy on-site, the use of treatment algorithms, and the mode of remuneration.

CONCLUSIONS

Our findings suggest that interventions are needed to increase the extent to which primary mental health care conforms to evidence-based recommendations. These interventions should target specific populations (i.e. the younger adults and the elderly), enhance accessibility to psychotherapy and to a regular family physician, and support primary care physicians in their clinical practice with patients suffering from depression in different ways such as developing knowledge to treat depression and adapting mode of remuneration.

Developing and comparing methods for measuring the content of care in mental health services

BACKGROUND

Measurement of what happens in mental health services is needed to describe services, identify variation in care provision and understand service outcomes. However, there is no consensus about appropriate methods or measures for content of care. Previous research has primarily used a single information source and prioritised staff over patient perspectives on content of care. This study aims to enhance understanding of how to measure content of care by developing and evaluating four instruments, each using a different measurement method.

METHODS

Development is described by four instruments-CaSPAR, CaRICE, CCCQ-S and CCCQ-P-which use researcher observation, staff-report and patient-report to measure the intensity and nature of care at services. Inter-rater reliability of CaRICE and CCCQ-S was investigated. Concordance between staff and patient perspectives was explored through assessing inter-rater agreement of CCCQ-S and CCCQ-P questionnaires. The convergence of data from the measures collected in an inpatient multi-site study was investigated.

RESULTS

CaRICE demonstrated good inter-rater reliability (κ = 0.71). CCCQ-S inter-rater reliability was poor. Concordance between staff and patient reports was low: there was a trend for patients to report less care received than staff reported had been provided. Results from CaSPAR, CaRICE and CCCQ-P exhibited divergence, indicating possible differences in patient, staff and researcher perspectives.

DISCUSSION

Information about content of care should be sought as close as possible to the point of delivery. There may be differing, valid perspectives about care provided by services. Further development of methods to measure content of care is required. Meanwhile, a multi-methods approach should be adopted, which allows inclusion of different perspectives (specifically including the patient's perspective) and triangulation of results from different measurement methods. CaSPAR, CaRICE and CCCQ-P can provide multi-perspective content of care measurement in inpatient services.

Appraising evidence for intervention effectiveness in early psychosis: conceptual framework and review of evaluation approaches

The literature that is relevant to evaluation of treatment effectiveness is large, scattered and difficult to assemble for appraisal. This scoping review first develops a conceptual framework to help organize the field, and second, uses the framework to appraise early psychosis intervention (EPI) studies. Literature searches were used to identify representative study designs, which were then sorted according to evaluation approach. The groupings provided a conceptual framework upon which a map of the field could be drawn. Key words were cross-checked against definitions in dictionaries of scientific terms and the National Library of Medicine Medical Subject Headings (MeSH) browser. Using the final list of key words as search terms, the EPI evaluation literature was appraised. Experimental studies could be grouped into two classes: efficacy and effectiveness randomized controlled trials. Non-experimental studies could be subgrouped into at least four overlapping categories: clinical epidemiological; health service evaluations; quality assurance studies; and, quasi-experimental assessments of treatment effects. Applying this framework to appraise EPI studies indicated promising evidence for the effectiveness of EPI irrespective of study design type, and a clearer picture of where future evaluation efforts should be focused. Reliance on clinical trials alone will restrict the type of information that can inform clinical practice. There is convergent evidence for the benefits of specialized EPI service functions across a range of study designs. Greater investment in health services research and quality assurance approaches in evaluating EPI effectiveness should be made, which will involve scaling up of study sizes and development of an EPI programme fidelity rating template. The degree of complexity of the evaluation field suggests that greater focus on research methodology in the training of Australasian psychiatrists is urgently needed.

Assessing the content of mental health services: a review of measures

BACKGROUND

Measurement of service content is necessary to understand what services actually provide and explain variation in service outcomes. There is no consensus about how to measure content of care in mental health services.

METHOD

Content of care measures for use in mental health services were identified through a search of electronic databases, hand searching of references from selected studies and consultation with experts in the field. Measures are presented in an organising methodological framework. Studies which introduced or cited the measures were read and investigations of empirical associations between content of care and outcomes were identified.

RESULTS

Twenty five measures of content of care were identified, which used three different data collection methods and five information sources. Seven of these measures have been used to identify links between content of care and outcomes, most commonly in Assertive Community Treatment settings.

DISCUSSION

Measures have been developed which can provide information about service content. However, there is a need for measures to demonstrate more clearly a theoretical or empirical basis, robust psychometric properties and feasibility in a range of service settings. Further comparison of the feasibility and reliability of different measurement methods is needed. Contradictory findings of associations between service content and outcomes may reflect measures' uncertain reliability, or that crucial process variables are not being measured.

CONCLUSION

Measures providing a greater depth of information about the nature of interventions are needed. In the absence of a gold standard content of care measure, a multi-methods approach should be adopted.

Sick-leave certificates granted to psychiatric outpatients with major depression

We investigated to what extent and for what reasons depressive outpatients are given certificates for sick-leaves and whether there are particular factors associated with the given sick-leaves. Our retrospective case note study was based on 213 adult psychiatric outpatients with first-time documented DSM-III-R major depression. Several sociodemographic, clinical, and treatment characteristics were detected during the follow-up of 3 months' medical care. This information was related to official registers of granted sick-leave certificates with a follow-up time of 30 months for the study patients. Sick-leave certificates were granted to 61% of patients. The mean duration of time off work due to documented illness was 142 days. Eighty-seven percent of the total sum of sick-leave days was for depressive disorders. Living with someone else, being employed, and early recognition of the illness were associated with the granted sick-leaves.

The evolution of clinical audit as a tool for quality improvement

Clinical auditing practices are recognized universally as a useful tool in evaluating and improving the quality of care provided by a health service. External auditing is a regular activity for mental health services in Australia but internal auditing activities are conducted at the discretion of each service. This paper evaluates the effectiveness of 6 years of internal auditing activities in a mental health service. A review of the scope, audit tools, purpose, sampling and design of the internal audits and identification of the recommendations from six consecutive annual audit reports was completed. Audit recommendations were examined, as well as levels of implementation and reasons for success or failure. Fifty-seven recommendations were identified, with 35% without action, 28% implemented and 33.3% still pending or in progress. The recommendations were more likely to be implemented if they relied on activity, planning and action across a selection of service areas rather than being restricted to individual departments within a service, if they did not involve non-mental health service departments and if they were not reliant on attitudinal change. Tools used, scope and reporting formats have become more sophisticated as part of the evolutionary nature of the auditing process. Internal auditing in the Barwon Health Mental Health Service has been effective in producing change in the quality of care across the organization. A number of evolutionary changes in the audit process have improved the efficiency and effectiveness of the audit.

Outcomes focused service delivery: developing an academic-management partnership

OBJECTIVE

Current mental health policies emphasise the need for services to be integrated and to develop outcomes-based evaluation systems. An overview of the challenges faced by service managers and clinical academics who develop the appropriate financial, personnel and academic infrastructure for these tasks is presented.

METHOD

By drawing on experiences within the St George Service and references to other services, we propose a model for a successful partnership between the academic and management components of a district service.

RESULTS

Major logistic impediments to the development of a partnership are identified, although the long-term scientific and service delivery benefits are highlighted. Key areas within both academic practice and managerial approaches requiring transformation are discussed.

CONCLUSIONS

A successful long-term partnership between management and an academic department within a district service may provide the opportunity for rapid progress in population-based service evaluation and health outcomes research.

Continuity of care for the severely mentally ill: concepts and measures

Very few useful measures of service functioning are as yet available for the evaluation of mental health services. Continuity of care has been identified as "the strategic first choice" for the development of such process measures. The term "continuity" has been used to refer to a variety of important aspects of service functioning, including whether services maintain contact with patients, whether patients consistently see the same staff, success of transfer between services, degree to which plans for services are followed through, integration between service providers, and comprehensiveness in meeting patient's individual needs. In this paper, we reviewed the main theoretical definitions of continuity of care for the severely mentally ill, and discussed the work that has attempted to operationalise these definitions and to apply them in the study of mental health services. We concluded that whilst continuity of care has had a central place in theoretical discussions of community service planning, progress in developing and applying practical measures has so far been sporadic and limited. Obstacles to such research have included great diversity in definitions of continuity and the confounding influence of individual patient characteristics on the relationship between service process and outcome. Despite these limitations, research in this field has indicated that developing and applying measures of continuity of care may be feasible and produce useful results with findings of some of the studies suggesting a relationship between continuity of care and individual outcome.