Evaluating mental health care reform: including the clinician, client, and family perspective

The incremental value of self-reported mental health measures in predicting functional outcomes of veterans

Research on patient-centered care supports use of patient/consumer self-report measures in monitoring health outcomes. This study examined the incremental value of self-report mental health measures relative to a clinician-rated measure in predicting functional outcomes among mental health service recipients. Participants (n = 446) completed the Behavior and Symptom Identification Scale, the Brief Symptom Inventory, and the Veterans/Rand Short Form-36 at enrollment in the study (T1) and 3 months later (T2). Global Assessment of Functioning (GAF) ratings, mental health service utilization, and psychiatric diagnoses were obtained from administrative data files. Controlling for demographic and clinical variables, results indicated that improvement based on the self-report measures significantly predicted one or more functional outcomes (i.e., decreased likelihood of post-enrollment psychiatric hospitalization and increased likelihood of paid employment), above and beyond the predictive value of the GAF. Inclusion of self-report measures may be a useful addition to performance measurement efforts.

Service user perspectives on the impact of a mental illness diagnosis

Aim – to provide a conceptual and practical analysis of the impacts of mental health diagnoses on consumers and to consider how service users might contribute to the new psychiatric classifications currently being drawn up. Methods – A search was carried out revealing a very sparse literature on this topic. Consultations with service users were conducted and the views of experts sought. Results – Diagnosis is important as it marks the formal status of psychiatric patient being conferred. Consumers react differently, and often, negatively to this. Stigma can follow from a diagnosis. The process of giving a diagnosis can range from one of negotiation and taking the person's strengths into account to the blunt allocation of an unwanted label. Consumers can be reduced to their diagnosis so it becomes their whole personhood and this can have an effect on their sense of self. However, consumers are not passive victims and have their own strategies for dealing with these issues. Conclusion – Consumers can use these experiences to make contributions to the new diagnostic classification systems and to future research.

Appraising evidence for intervention effectiveness in early psychosis: conceptual framework and review of evaluation approaches

The literature that is relevant to evaluation of treatment effectiveness is large, scattered and difficult to assemble for appraisal. This scoping review first develops a conceptual framework to help organize the field, and second, uses the framework to appraise early psychosis intervention (EPI) studies. Literature searches were used to identify representative study designs, which were then sorted according to evaluation approach. The groupings provided a conceptual framework upon which a map of the field could be drawn. Key words were cross-checked against definitions in dictionaries of scientific terms and the National Library of Medicine Medical Subject Headings (MeSH) browser. Using the final list of key words as search terms, the EPI evaluation literature was appraised. Experimental studies could be grouped into two classes: efficacy and effectiveness randomized controlled trials. Non-experimental studies could be subgrouped into at least four overlapping categories: clinical epidemiological; health service evaluations; quality assurance studies; and, quasi-experimental assessments of treatment effects. Applying this framework to appraise EPI studies indicated promising evidence for the effectiveness of EPI irrespective of study design type, and a clearer picture of where future evaluation efforts should be focused. Reliance on clinical trials alone will restrict the type of information that can inform clinical practice. There is convergent evidence for the benefits of specialized EPI service functions across a range of study designs. Greater investment in health services research and quality assurance approaches in evaluating EPI effectiveness should be made, which will involve scaling up of study sizes and development of an EPI programme fidelity rating template. The degree of complexity of the evaluation field suggests that greater focus on research methodology in the training of Australasian psychiatrists is urgently needed.

Model testing: examining parent satisfaction

The purpose of this study was to test a model predicting social-psychological determinants of parent satisfaction with children's mental health services. Model concepts were parents' characteristics, definition of situation, desired services, expectations, met desires, met expectations, and satisfaction. The sample included 120 parents of children, ages 3-18 years, admitted to one of five treatment programs. With minor changes, results of structural equation modeling indicated a good fit between the data and the proposed relationships. The strongest predictor of parent satisfaction was met expectations; however, inclusion of the desire concepts was essential in obtaining a strong, acceptable model fit.

Development and testing of a scale measuring parent satisfaction with staff interactions

This study tested the psychometric properties of the Parent Satisfaction Scale (PSS), a new scale measuring parents' satisfaction with their child's mental health services, primarily focusing on staff's interpersonal interactions. Data were collected by postal surveys 5 to 10 months after a child's admission to 1 of 5 mental health treatment programs. Completed surveys were received from parents of 120 of a potential 232 children (52% response rate). Internal consistency reliability of the PSS was strong (alpha = .96). Construct validity was supported by significant relationships between the PSS and constructs considered to be antecedent variables influencing parent satisfaction, including met expectations (r = 0.62, P < .001), met desires (r = 0.32, P < .001), and met needs (r = 0.55, P < .001). Convergent validity was supported, with the PSS having Pearson correlations of .86 (P < .001) with the Client Satisfaction Questionnaire-8 and .76 (P < .001) with parents' ratings of satisfaction with specific services received.

Mental health program monitoring: towards simplifying a complex task

Finding measures that can assess areas of expected program impact, provide valid results, and be easily integrated into routine program practices is a significant challenge. This paper is intended to assist program staff by providing an accessible inventory of measures appropriate for routine monitoring of the status and outcome of individuals using mental health outpatient and community programs. The inventory is not exhaustive, but rather includes examples of solid measures for assessing outcomes in four key domains--symptoms, functioning, quality of life, and satisfaction. These can provide a core of information, to which measurement of more in-depth issues can be added to address specific concerns.

Clinical status. Charting for outcomes in behavioral health

This article summarizes the criteria that clinical outcome data must meet to be useful in the quest for empirical effectiveness data about mental health services. Although demand is high for such data, its potential usefulness is just beginning to be tapped. The mental health field presents unique challenges for implementing outcome tracking systems, for analyzing and reporting results, and for using results to improve the processes of care. Measuring outcomes will not automatically improve care or ensure quality. It may, however, provide information that will be useful in guiding efforts to improve the quality of mental health services.

Measuring treatment outcome and client satisfaction among children and families

As the delivery and reimbursement methods for mental health services change rapidly, measuring treatment outcome and client satisfaction has become critical. This article describes a case example of a treatment outcome and client satisfaction assessment program at the Children's Health Council, a private nonprofit agency affiliated with Stanford University that provides comprehensive mental health services to children and families in Palo Alto, California. Approximately 300 families receive mental health treatment per year at the agency. The simple and inexpensive program presented herein can be used and modified by other mental health professionals and agencies struggling to develop satisfactory treatment outcome and client satisfaction evaluation programs.

Assessing behavioral health outcomes in outpatient programs: reliability and validity of the BASIS-32

The Behavior and Symptom Identification Scale (BASIS-32) was developed to assess mental health outcomes among patients with severe illness treated on inpatient programs. However, its applicability and utility to those treated in outpatient programs has not been determined. The objective of this study was to assess reliability, validity, and sensitivity to change of the BASIS-32 among mental health consumers treated in outpatient programs. A total of 407 outpatients completed the BASIS-32 and the Short Form Health Status Profile (SF-36) at the beginning of a treatment episode and again 30 to 90 days later. Outpatients reported less difficulty at intake than did inpatients, and the BASIS-32 detected statistically significant changes 30 to 90 days after beginning outpatient treatment. Factor structure and construct validity were partially confirmed on this sample of outpatient consumers. Analyses of data from a wide range of facilities and samples would add to validation efforts and to further refinement of the BASIS-32.

Assessment of the Massachusetts Medicaid managed Behavioral Health Program: year three

This evaluation of the third year of the Massachusetts Medicaid managed Mental Health/Substance Abuse Program showed that overall utilization increased slightly and expenditures were nearly the same in FY1994 compared to FY1993; however, they were lower for disabled members. Providers believed that access to care, utilization, and quality of care were the same or better than a year earlier and that the clinical review process was improved. Client severity was higher. Aftercare planning improved but gaps in services persisted. Integration of care improved. Administrative and management problems continued. Lessons for similar, more recent initiatives are discussed.

The imperative of outcome assessment in psychiatry

This report describes the current conceptualization of outcome assessment in psychiatry and focuses on how assessment instruments can be built into psychiatric facility-based practice. First, the domains of clinical assessment are outlined, with an emphasis on three elements: level of psychiatric symptoms, clinical functioning, and patient satisfaction. Examples of outcome instruments then are provided as well as the elements of their successful implementation. Finally, the value of linking outcome assessment to data on patient characteristics and service utilization are discussed in order to gain insight into the relationship between treatment and outcome. The clinical, fiscal, and regulatory imperatives emerging for outcome assessment call for its demystification and widespread application.