Measurement and psychotherapy. Evidence-based practice and practice-based evidence

Assessing the outcome of community-based psychiatric care: building a feedback loop from 'real world' health services research into clinical practice

OBJECTIVE

To describe the main characteristics of the South-Verona Outcome Project (SVOP) and to focus on its overall conceptual framework, with specific reference to the following perspectives: i) integrating evidence-based and practice-based approaches; ii) involving service professionals in routine outcome assessment; and iii) involving service users in mental health outcome assessment.

METHOD

A selective literature review of methodological and empirical papers addressing the relevance and usefulness of outcome research to routine clinical practice was performed.

RESULTS

Reviewed literature shows the need to integrate evidence-based and practice-based approaches and to involve service professionals in routine outcome assessment, by adopting a multiple perspective paradigm. Studies conducted in 'real world' health services indicate that the outcome of care is multifaceted and it can be perceived differently when different perspectives are taken into account. Such a complex picture can provide more comprehensive information on the effectiveness of care provided, to feed back positively into clinical practice.

CONCLUSION

The SVOP design and its methodological background were demonstrated to be appropriate for a detailed and routine assessment of outcome in the 'real world' of mental health services.

Linking data to decision-making: applying qualitative data analysis methods and software to identify mechanisms for using outcomes data

A multiple case study was conducted to examine how staff in child out-of-home care programs used data from an Outcomes Management System (OMS) and other sources to inform decision-making. Data collection consisted of thirty-seven semi-structured interviews with clinicians, managers, and directors from two treatment foster care programs and two residential treatment centers, and individuals involved with developing the OMS; and observations of clinical and quality management meetings. Case study and grounded theory methodology guided analyses. The application of qualitative data analysis software is described. Results show that although staff rarely used data from the OMS, they did rely on other sources of systematically collected information to inform clinical, quality management, and program decisions. Analyses of how staff used these data suggest that improving the utility of OMS will involve encouraging staff to participate in data-based decision-making, and designing and implementing OMS in a manner that reflects how decision-making processes operate.

Supportive therapy for schizophrenia

BACKGROUND

Supportive therapy is often used in everyday clinical care and in evaluative studies of other treatments.

OBJECTIVES

To estimate the effects of supportive therapy for people with schizophrenia.

SEARCH STRATEGY

We searched the Cochrane Schizophrenia Group's register of trials (January 2004), supplemented by manual reference searching and contact with authors of relevant reviews or studies.

SELECTION CRITERIA

All randomised trials involving people with schizophrenia and comparing supportive therapy with any other treatment or standard care.

DATA COLLECTION AND ANALYSIS

We reliably selected studies, quality rated these and extracted data. For dichotomous data, we estimated the relative risk (RR) fixed effect with 95% confidence intervals (CI). Where possible, we undertook intention-to-treat analyses. For statistically significant results, we calculated the number needed to treat/harm (NNT/H). We estimated heterogeneity (I-square technique) and publication bias.

MAIN RESULTS

We included 21 relevant studies. We found no significant differences in the primary outcomes between supportive therapy and standard care. There were, however, significant differences favouring other psychological or psychosocial treatments over supportive therapy. These included hospitalisation rates (3 RCTs, n=241, RR 2.12 CI 1.2 to 3.6, NNT 8) but not relapse rates (5 RCTs, n=270, RR 1.18 CI 0.9 to 1.5). We found that the results for general functioning significantly favoured cognitive behavioural therapy compared with supportive therapy in the short (1 RCT, n=70, WMD -9.50 CI -16.1 to -2.9), medium (1 RCT, n=67, WMD -12.6 CI -19.4 to -5.8) and long term (2 RCTs, n=78, SMD -0.50 CI -1.0 to -0.04), but the clinical significance of these findings based on few data is unclear. Participants were less likely to be satisfied with care if receiving supportive therapy compared with cognitive behavioural treatment (1 RCT, n=45, RR 3.19 CI 1.0 to 10.1, NNT 4 CI 2 to 736). The results for mental state and symptoms were unclear in the comparisons with other therapies. No data were available to assess the impact of supportive therapy on engagement with structured activities.

AUTHORS' CONCLUSIONS

There are insufficient data to identify a difference in outcome between supportive therapy and standard care. There are several outcomes, including hospitalisation and general mental state, indicating advantages for other psychological therapies over supportive therapy but these findings are based on a few small studies. Future research would benefit from larger trials that use supportive therapy as the main treatment arm rather than the comparator.

Chronic childhood constipation: a review of the literature and the introduction of a protocolized behavioral intervention program

OBJECTIVE

To release a newly protocolized behavioral intervention program for children with chronic constipation aged 4-18 years with guidance from literature about underlying theories from which the treatment techniques follow.

METHODS

Articles until July 2006 were identified through electronic searches in Medline, PsychInfo and Picarta. There was no limit placed on the time periods searched. Following keywords were used: constipation, encopresis, fecal incontinence, psychotherapy, emotions, randomized controlled trials, parent-child relations, parents, family, psychology, behavioral, behavioral problems, psychopathology, toilet, social, psychosocial, pain, retentive posturing, stool withholding, stool toileting refusal, shame, stress, anxiety. A filter was used to select literature referring to children 0-18 years old. Key constructs and content of sessions for a protocolized behavioral intervention program are derived from literature.

RESULTS

Seventy-one articles on chronic childhood constipation are critically reviewed and categorized into sections on epidemiology, symptomatology, etiology and consequences, treatment and effectivity, and follow-up on chronic childhood constipation. This is followed by an extensive description of our protocolized behavioral intervention program.

CONCLUSION

This is the first article on childhood constipation presenting a full and transparent description of a behavioral intervention program embedded in literature. In addition, a theoretical framework is provided that can serve as a trial paradigm to evaluate intervention effectiveness.

PRACTICE IMPLICATIONS

This article can serve as an extensive guideline in routine practice to treat chronically constipated children. By releasing our protocolized behavioral intervention program and by offering a theoretical framework we expect to provide a good opportunity to evaluate clinical effectivity by both randomized controlled trials and qualitative research methods. Findings will contribute to the implementation of an effective treatment for chronic constipation in childhood.

A biopsychosocial model as a guide for psychoeducation and treatment of depression

Effective treatment of severe or chronic unipolar depression requires the combination of pharmacological and psychotherapeutic interventions, and demands a theoretical paradigm integrating biological and psychosocial aspects of depression. Supported by recent research, we propose in our article a biopsychosocial diathesis-stress model of depression. Its basic aim is psychoeducational: to provide therapists, patients, and their environment a constructive conceptual framework to understand depressive complaints, vulnerability, and stress. The core of the model consists of the concept of psychobiological vulnerability, which is determined by risk factors-of a biogenetic, psychological, somatic, and societal nature-and by protective factors. Life events with an idiosyncratic, stress-inducing value interact with this vulnerability, triggering severe or chronic distress that affects the individual's resilience and leads to symptoms of depression. The pathogenesis of depression is symbolized by a negative downward loop, in which interactions among symptoms, vulnerability, and stressors drive the patient toward a depressive condition. Moreover, experiencing recurrent depression influences psychobiological vulnerability, the occurrence of stressors, and tremendously increases the risk of further relapse. The model stresses the self-evident integration of biological and psychological therapeutic interventions that need to focus on symptom reduction and on relapse prevention. Moreover, it offers the patient and therapist a psychoeducational context in which the individual's vulnerability and depressive symptoms can be treated. Finally, applications of the depression model as a therapeutic approach to severe depression in the phases of remoralization, symptom reduction, and relapse prevention are presented.

The effectiveness of computerized cognitive behavioural therapy in routine care

OBJECTIVES AND DESIGN

The efficacy of a Computerized Cognitive Behavioural Therapy (CCBT) package, Beating the Blues, has been demonstrated in a large randomized controlled trial. The current study tests the generalizability of this finding in a naturalistic non-randomized trial.

METHOD

219 patients with anxiety and/or depression were recruited to receive CCBT in routine care. The Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) and Work and Social Adjustment scale (WSA) were administered pre-treatment, immediately on completing treatment and at 6 months post-treatment. Single-item self-report measures of anxiety and depression were also collected during each treatment session.

RESULTS

Completer and intention-to-treat analysis demonstrated statistically and clinically significant improvements on the CORE-OM, WSA and in self-reported anxiety and depression. Intention-to-treat analysis indicated an average 0.29-point drop on the CORE-OM, equating to an uncontrolled pre-post effect size of 0.50. Research completers achieved an average 0.61-point drop equating to an uncontrolled pre-post size of 1.00 on the same measure. Where data was available (18%), these benefits were maintained at week 32 (6 months follow-up).

CONCLUSION

CCBT can be an effective first line tool within a stepped care framework for the management of common mental health problems.

Supportive therapy for schizophrenia

BACKGROUND

Supportive therapy is often used in everyday clinical care and in evaluative studies of other treatments.

OBJECTIVES

To estimate the effects of supportive therapy for people with schizophrenia.

SEARCH STRATEGY

We searched the Cochrane Schizophrenia Group's register of trials (January 2004), supplemented by manual reference searching and contact with authors of relevant reviews or studies.

SELECTION CRITERIA

All randomised trials involving people with schizophrenia and comparing supportive therapy with any other treatment or standard care.

DATA COLLECTION AND ANALYSIS

We reliably selected studies, quality rated these and extracted data. For dichotomous data, we estimated the relative risk (RR) fixed effect with 95% confidence intervals (CI). Where possible, we undertook intention-to-treat analyses. For statistically significant results, we calculated the number needed to treat/harm (NNT/H). We estimated heterogeneity (I-square technique) and publication bias.

MAIN RESULTS

We included 21 relevant studies. We found no significant differences in the primary outcomes between supportive therapy and standard care. There were, however, significant differences favouring other psychological or psychosocial treatments over supportive therapy. These included hospitalisation rates (3 RCTs, n=241, RR 2.12 CI 1.2 to 3.6, NNT 8) but not relapse rates (5 RCTs, n=270, RR 1.18 CI 0.9 to 1.5). We found that the results for general functioning significantly favoured cognitive behavioural therapy compared with supportive therapy in the short (1 RCT, n=70, WMD -9.50 CI -16.1 to -2.9), medium (1 RCT, n=67, WMD -12.6 CI -19.4 to -5.8) and long term (2 RCTs, n=78, SMD -0.50 CI -1.0 to -0.04), but the clinical significance of these findings based on few data is unclear. Participants were less likely to be satisfied with care if receiving supportive therapy compared with cognitive behavioural treatment (1 RCT, n=45, RR 3.19 CI 1.0 to 10.1, NNT 4 CI 2 to 736). The results for mental state and symptoms were unclear in the comparisons with other therapies. No data were available to assess the impact of supportive therapy on engagement with structured activities.

AUTHORS' CONCLUSIONS

There are insufficient data to identify a difference in outcome between supportive therapy and standard care. There are several outcomes, including hospitalisation and general mental state, indicating advantages for other psychological therapies over supportive therapy but these findings are based on a few small studies. Future research would benefit from larger trials that use supportive therapy as the main treatment arm rather than the comparator.

"Not prepared to care" and "raising the bar": case examples of building university-community partnerships in gerontological social work research

This paper examines university-community partnerships in gerontological social work research. Utilizing a case study approach, the process of collaboration is explored within the context of a two-phase research project conducted jointly by agency social workers, social work students and a social work faculty member. The benefits and challenges of the university-community research partnership are discussed along with the implications for gerontological social work research, education and practice.

Transforming between Beck Depression Inventory and CORE-OM scores in routine clinical practice

OBJECTIVES

The Clinical Outcomes in Routine Evaluation - Outcome Measure (CORE-OM) and the Beck Depression Inventory (BDI) are routinely used to assess emotional problems. It would be helpful to be able to compare scores when only one of the measures is available. We investigated the relationship between the measures and produced translation tables.

METHODS

Level of agreement between CORE-OM and BDI-I was assessed for 2,234 clients who had completed both measures at referral for routine secondary care. Tables for predicting between the measures were constructed using several methods, including non-linear regression and non-parametric smoothing. Results were cross-validated on a separate sample of 326 clients.

RESULTS

High correlations between the measures were obtained (r=.862 for female clients; r=.855 for male clients). Accuracy in predicting caseness is higher than predicting depression levels.

CONCLUSIONS

CORE-OM and BDI can be compared in routine clinical settings with acceptable accuracy.

Static and dynamic patient characteristics as predictors of criminal recidivism: A prospective study in a Dutch forensic psychiatric sample

If clinicians in forensic psychiatry want to reduce risk of reoffending in their patients, they require insight into dynamic risk factors, and evidence that these add predictive power to static risk indicators. Predictors need to be evaluated under clinically realistic circumstances. This study aimed to validate dynamic and static variables as predictors of reconviction in a naturalistic outcome study. Data on static and dynamic risk factors were collected for 151 patients discharged from Dutch forensic psychiatric hospitals. Community follow-up was prospective, with a 5.5 year minimum. A prediction model was developed using Cox regression analysis. The magnitude of the predictive power of this model was estimated using receiver operating characteristic (ROC) analysis. The final prediction model contained four static and no dynamic predictors. The model's ROC area under the curve was .79 (95% CI .69-.89). Clinical risk ratings were non-predictive. Post hoc analyses exploring the influence of subgroups of patients did not yield better models. It is concluded that a small set of static predictors yielded a good estimate of future reconvictions; inclusion of dynamic predictors did not add predictive power.

Training of evidence-based assessment and intervention approaches in cross-cultural contexts: challenges and solutions

Dissemination of evidence-based assessment and intervention approaches for child and adolescent with behavioral and/or emotional problems is now a priority in the field worldwide. However, developing staff competence in evidence-based assessment and intervention approaches in different countries is complicated by some environmental and economic constraints. In this paper a distance training/supervision model is discussed. We describe seven specific challenges encountered and solutions used for overcoming the obstacles in order to implement evidence-based assessment and intervention approaches in different sites in Brazil, Egypt, Israel, and Lebanon.

Transference patterns in the psychotherapy of personality disorders: empirical investigation

BACKGROUND

The concept of transference has broadened to a recognition that patients often express enduring relational patterns in the therapeutic relationship.

AIMS

To examine the structure of patient relational patterns in psychotherapy and their relation with DSM-IV personality disorder symptoms.

METHOD

A random sample of psychologists and psychiatrists (n=181) completed a battery of instruments on a randomly selected patient in their care.

RESULTS

Exploratory factor analysis identified five transference dimensions: angry/entitled, anxious/preoccupied, avoidant/counterdependent, secure/engaged and sexualised. These were associated in predictable ways with Axis II pathology; four mapped on to adult attachment styles. An aggregated portrait of transference patterns in narcissistic patients provided a clinically rich, empirically based description of transference processes that strongly resembled clinical theories.

CONCLUSIONS

The ways patients interact with their therapists can provide important data about their personality, attachment patterns and interpersonal functioning. These processes can be measured in clinically sophisticated and psychometrically sound ways. Such processes are relatively independent of clinicians' theoretical orientation.

[An observational study of effectiveness in community residential facilities]

AIMS

Evaluation of community residential facilities effectiveness in the Department of Mental Health of Desio (Milan).

METHOD

Outcomes in symptoms, disability, family burden and quality of life were evaluated during one year through a longitudinal study, using a pre-test and post test design without control group.

RESULTS

Residential care is effective in reducing disability and symptoms, while it is not effective towards family burden. Quality of life is improved in some domains, but not in others (e.g. social and family relationships).

CONCLUSIONS

Outcome assessment is feasible in residential facilities, following a multiaxial and multifactorial model. We need to clarify the goals of residential care, focussing on active components of the residential treatment.

Intervention fidelity

BACKGROUND

Although an assumption of consistency in intervention delivery is key to the conclusion that the findings are valid, attention to both interventionist adherence and competence has not become a standard in nursing intervention research.

OBJECTIVES

The purpose of this article is to describe the technology model of intervention fidelity as it applies to psychosocial interventions, and to propose its application in nursing intervention research as a way to support accurate conclusions about validity.

METHODS

The features and methods of the Technology Model of intervention fidelity were reviewed. The review included development of a manual, training and supervision of interventionists, measurement of intervention fidelity, and inclusion of intervention fidelity as an independent variable in the analysis.

RESULTS

The Technology Model and the associated methodologies that have been used to monitor psychosocial intervention fidelity in psychiatry can be adapted and applied by nursing. Monitoring costs can be managed through use of a general system that can be applied across programmatic or related studies.

CONCLUSIONS

Application of the Technology Model of intervention fidelity could advance the maturation of nursing intervention research and evidence-based clinical practice.

Sooner or later? Issues in the early diagnosis of dementia in general practice: a qualitative study

OBJECTIVE

The aim of this study was to explore the perspectives of primary care practitioners on the early diagnosis of dementia.

METHODS

A total of 247 GPs, 146 community nurses, 36 practice nurses, 79 community mental health nurses and others working in a range of hospital, residential and community settings attended 24 one-day workshops in 21 cities and towns in the UK. A nominal group approach was used relating to the early diagnosis of dementia in the community.

RESULTS

Groups agreed on the benefits and risks of early diagnosis of dementia; disagreed about screening for dementia, and about professional resistance to making the diagnosis; constructed comprehensive guidelines on diagnosis, but without much reference to resource implications; yet described actual local resource limitations in detail; and avoided dilemmas about dementia care by framing it as a specialist activity.

CONCLUSION

Practitioners situate dementia in a family context but do not yet use a disablement model of dementia which might reduce tensions about early diagnosis and the disclosure of the diagnosis. The term diagnosis could usefully be replaced by recognition, to aid this shift in model. Service gaps may emerge or widen if earlier diagnosis of dementia is pursued as a policy objective.

Practice development in psychological interventions: mental health nurse involvement in the Conversational Model of psychotherapy

This paper describes a mental health nurse led practice development initiative in psychotherapy. Four mental health nurses have been trained to deliver the Conversational Model of psychotherapy (also known as psychodynamic-interpersonal (PI) psychotherapy) a non-cognitive behavioural therapy (CBT) with a robust evidence base. We report on the robust range of both processes and outcome measures being used to evaluate this initiative. We conclude that good quality evidence-based practice requires careful planning and preparation, adequate financial resources from Trusts, as well as commitment and motivation from the staff expected to be involved in such initiatives.

Evidence-based medicine in the psychological treatment of schizophrenia

Treatments should be based on reviews of the best available evidence. However, there are different views about the types of evidence that may be included. Evidence-based medicine (EBM) has its own clear hierarchy with systematic reviews of randomized controlled trials (RCTs) at the top level and expert and user opinion at the lowest level of evidence. Other paradigms, such as qualitative approaches, are difficult to place in such a hierarchy. This article examines some of the assumptions behind evidence-based practice, particularly when applied to complex "real life" clinical situations. A case example is used to demonstrate some of the complexity of applying evidence to practice. A complementary paradigm of practice-based evidence is described.

Day treatment of patients with personality disorders: experiences from a Norwegian treatment research network

This study sought to investigate the following research questions: Are time-limited day treatment programs for patients with personality disorder (PD) effective outside resourceful university settings, and what are the overall treatment results when the program is implemented on a larger scale? Do all categories and subtypes of PDs respond favorably to such treatment? How intensive should such day treatment programs be? All patients (n = 1,244) were consecutively admitted to eight different treatment programs in the Norwegian Network of Psychotherapeutic Day Hospitals from 1993 to 2000. Altogether, 1,010 patients were diagnosed with PD. Avoidant, borderline, not otherwise specified (NOS), and paranoid PD were the most common conditions. SCID-II and MINI were used as diagnostic instruments. Outcome measures included GAF Global Assessment of Functioning, (GAF; American Psychiatric Association, 1994), SCL-90R, CIP, Quality of Life, work functioning and parasuicidal behavior, measured at admittance, discharge and 1-year follow up. The attrition rate was 24%. The number of dropouts did not improve over time. As a group, completers with PD improved significantly on all outcome variables from admittance to discharge and improvement was maintained or increased at follow up. Treatment results were best for borderline PD, cluster C patients, PD NOS and No PD, and poorer for cluster A patients. Units with a high treatment dosage did not experience better outcomes than those with a low treatment dosage (10 hours per week). Results from the University unit were not better than those from units at local hospitals or mental health centers.

Outcomes research in mental health. Systematic review

BACKGROUND

Outcomes research involves the secondary analysis of data collected routinely by clinical services, in order to judge the effectiveness of interventions and policy initiatives. It permits the study of large databases of patients who are representative of 'real world' practice. However, there are potential problems with this observational design.

AIMS

To establish the strengths and limitations of outcomes research when applied in mental health.

METHOD

A systematic review was made of the application of outcomes research in mental health services research.

RESULTS

Nine examples of outcomes research in mental health services were found. Those that used insurance claims data have information on large numbers of patients but use surrogate outcomes that are of questionable value to clinicians and patients. Problems arise when attempting to adjust for important confounding variables using routinely collected claims data, making results difficult to interpret.

CONCLUSIONS

Outcomes research is unlikely to be a quick or cheap means of establishing evidence for the effectiveness of mental health practice and policy.

Towards a standardised brief outcome measure: psychometric properties and utility of the CORE-OM

BACKGROUND

An acceptable, standardised outcome measure to assess efficacy and effectiveness is needed across multiple disciplines offering psychological therapies.

AIMS

To present psychometric data on reliability, validity and sensitivity to change for the CORE-OM (Clinical Outcomes in Routine Evaluation--Outcome Measure).

METHOD

A 34-item self-report instrument was-developed, with domains of subjective well-being, symptoms, function and risk. Analysis includes internal reliability, test-retest reliability, socio-demographic differences, exploratory principal-component analysis, correlations with other instruments, differences between clinical and non-clinical samples and assessment of change within a clinical group.

RESULTS

Internal and test-retest reliability were good (0.75-0.95), as was convergent validity with seven other instruments, with large differences between clinical and non-clinical samples and good sensitivity to change.

CONCLUSIONS

The CORE-OM is a reliable and valid instrument with good sensitivity to change. It is acceptable in a wide range of practice settings.

EBM in practice: psychiatry

Psychiatry was one of the first medical specialties to use the tools of evidence-based medicine (EBM)--randomised controlled trials (RCTs), systematic reviews with meta-analyses--and as many treatment decisions in psychiatry are evidence-based as in general medicine. Psychiatrists have some reservations about the EBM approach because of perceived limitations in methodology of RCTs and systematic reviews; gaps in the evidence base; problems interpreting the available evidence; and neglect of individual patient uniqueness in quantitative research based on groups or populations. Research supports the value of psychotherapy and there are now a number of empirically validated efficacious psychotherapies for a range of disorders.

Service profiling and outcomes benchmarking using the CORE-OM: toward practice-based evidence in the psychological therapies. Clinical Outcomes in Routine Evaluation-Outcome Measures

To complement the evidence-based practice paradigm, the authors argued for a core outcome measure to provide practice-based evidence for the psychological therapies. Utility requires instruments that are acceptable scientifically, as well as to service users, and a coordinated implementation of the measure at a national level. The development of the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) is summarized. Data are presented across 39 secondary-care services (n = 2,710) and within an intensively evaluated single service (n = 1,455). Results suggest that the CORE-OM is a valid and reliable measure for multiple settings and is acceptable to users and clinicians as well as policy makers. Baseline data levels of patient presenting problem severity, including risk, are reported in addition to outcome benchmarks that use the concept of reliable and clinically significant change. Basic quality improvement in outcomes for a single service is considered.