Implementation of a statewide outcome monitoring system: lessons learned from substance abuse treatment provider staff

Psychotherapists' Experience with In-Session Use of Routine Outcome Monitoring: A Qualitative Meta-analysis

Routine outcome monitoring (ROM) has become an increasingly utilized tool in therapeutic practice that has the potential to improve therapy outcomes. This study aimed to synthesize the findings of existing qualitative studies investigating how clinicians use ROM in their work with clients. A systematic search of qualitative studies on clinicians' experience with the use of ROM in mental health services was conducted via PsycInfo, PsycArticles, Medline, Web of Science, and Scopus databases. Qualitative meta-analysis was used to synthesize the finding of the primary studies. Forty-seven studies met the inclusion criteria. The analysis resulted in 21 meta-categories organized into six clusters, namely (1) obtaining clinically relevant information, (2) adapting treatment, (3) facilitating communication, (4) enhancing the therapeutic relationship, (5) facilitating change in clients, and (6) personalized usage of ROM. The meta-analysis revealed that clinicians utilized ROM in diverse ways, including both informational and communicational functions. From the clinicians' perspective, ROM was an element that, on the one hand, introduced additional structure and standardization in treatment and, on the other hand, allowed for greater flexibility and tailoring of treatment.

Producing “Progress” Through the Implementation of Outcome Monitoring in Alcohol and Other Drug Treatment

Outcome monitoring—a process in which clinicians use standardized tools to routinely measure client “progress” on predefined outcomes of interest over time—is increasingly being implemented in alcohol and other drug (AOD) treatment services as a way of demonstrating quality of care. However, relatively little is known about the implications and unintended consequences of implementing outcome monitoring in clinical and social practices. In this paper we draw on qualitative data emerging from focus groups with clinicians who piloted an outcome monitoring tool in Melbourne, Australia, using conceptual tools drawn from science and technology studies. Rather than acting as a stable empirical object, we argue that realities of progress are enacted multiply in relation to preexisting treatment discourses and policy, organizational practices of data collection and management, reporting tools, and clinician attitudes and practices. In particular, we trace how the tool orders the “problem” of drugs differently to the qualitative mode of ordering; how different modes of ordering progress hang together and the tensions and coordination strategies that are involved. And finally, we highlight an unintended consequence of the outcome monitoring process—the enactment of vulnerability and its distribution across the clinical relationship. We suggest that researchers, policy makers, and clinicians need to think reflectively and critically about the ways in which we and our tools and interventions are influential in producing AOD problems, what constitutes progress, and ultimately what the focus of treatment should be.

Implementing measurement-based care (iMBC) for depression in community mental health: a dynamic cluster randomized trial study protocol

Background

Measurement-based care is an evidence-based practice for depression that efficiently identifies treatment non-responders and those who might otherwise deteriorate [1]. However, measurement-based care is underutilized in community mental health with data suggesting fewer than 20 % of behavioral health providers using this practice to inform treatment. It remains unclear whether standardized or tailored approaches to implementation are needed to optimize measurement-based care fidelity and penetration. Moreover, there is some suggestion that prospectively tailored interventions that are designed to fit the dynamic context may optimize public health impact, though no randomized trials have yet tested this notion [2]. This study will address the following three aims: (1) To compare the effect of standardized versus tailored MBC implementation on clinician-level and client-level outcomes; (2) To identify contextual mediators of MBC fidelity; and (3) To explore the impact of MBC fidelity on client outcomes.

Methods/design

This study is a dynamic cluster randomized trial of standardized versus tailored measurement-based care implementation in Centerstone, the largest provider of community-based mental health services in the USA. This prospective, mixed methods implementation-effectiveness hybrid design allows for evaluation of the two conditions on both clinician-level (e.g., MBC fidelity) and client-level (depression symptom change) outcomes. Central to this investigation is the focus on identifying contextual factors (e.g., attitudes, resources, process, etc.) that mediate MBC fidelity and optimize client outcomes.

Discussion

This study will contribute generalizable and practical strategies for implementing systematic symptom monitoring to inform and enhance behavioral healthcare.

Trial registration

Clinicaltrials.gov NCT02266134.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-015-0313-2) contains supplementary material, which is available to authorized users.

Validation and implementation of the Australian Treatment Outcomes Profile in specialist drug and alcohol settings

INTRODUCTION AND AIMS

The measurement of clinical outcomes is an important, but lacking, component of drug and alcohol treatment in Australia. This study aimed to psychometrically validate the Treatment Outcomes Profile under Australian conditions, examining implementation and feasibility issues in three public opioid treatment program clinics in NSW.

DESIGN AND METHODS

The Treatment Outcomes Profile was modified to reflect Australian conditions and re-named the Australian Treatment Outcomes Profile (ATOP). The ATOP was introduced into the participating clinics and administered by clinic staff at 3-month intervals as part of routine clinical practice. Participants completed a research interview, consisting of the ATOP and a suite of 'gold standard' instruments assessing substance use and related health and welfare domains, in the 72 h following completion of a routine clinical ATOP. The researcher- and clinician-administered ATOPs were compared to assess interrater reliability, and the researcher-administered ATOP and 'gold standard' instruments were compared to assess concurrent validity. Implementation and feasibility issues were assessed using questionnaires and focus groups with clinician and clients.

RESULTS

The ATOP demonstrated acceptable concurrent validity and interrater reliability. It was well received by clients and clinicians, particularly for its ease of use, applicability and brevity.

CONCLUSIONS

The ATOP is a psychometrically valid instrument for the measurement of treatment outcomes in Australian opioid treatment populations and can feasibly be implemented as part of routine clinical practice in specialist opioid treatment program clinics. The role of the ATOP to measure outcomes in other drug and alcohol treatment modalities requires exploration.

Development of the Treatment Outcomes Profile

AIM

To develop the Treatment Outcomes Profile (TOP), a new instrument for monitoring substance misuse treatment.

DESIGN

Prospective cohort, psychometric evaluation with 7-day retest and 1-month follow-up to assess inter-rater reliability, concurrent, discriminant and construct validity, and change sensitivity.

PARTICIPANTS

A sample of 1021 service users, aged 16-62 years. Recruitment from 63 treatment agencies in England, collectively providing opioid substitution treatment, psychosocial interventions, in-patient detoxification and residential rehabilitation.

MEASUREMENTS

Thirty-eight frequency, rating scale and period prevalence measures, with 28-day recall, across substance use, health, crime and social functioning domains, administered as personal interview by 163 treatment keyworkers.

FINDINGS

Twenty outcome measures met inter-rater reliability criteria: days used alcohol, opioids, crack cocaine, cocaine powder, amphetamines, cannabis and one other named substance; days injected and period prevalence of direct or indirect needle/syringe sharing; subjective rating of physical and psychological health; days committed shop theft and drug selling, period prevalence of vehicle, property, fraud/forgery and assault/violence offences; rating of quality of life; days worked and attended for education/training; and period prevalence of acute housing problems and risk of eviction. Intraclass correlation coefficients for scale measures and Cohen's kappa for dichotomous measures reached or exceeded 0.75 and 0.61, respectively. There were satisfactory validity assessments and change sensitivity of scale items judged by effect size and smallest detectable difference. The TOP clinical tool contains an additional 10 items for individual treatment planning and review.

CONCLUSIONS

The TOP is a reliable and valid 20-item instrument for treatment outcomes monitoring.

Using administrative data for longitudinal substance abuse research

The utilization of administrative data in substance abuse research has become more widespread than ever. This selective review synthesizes recent extant research from 31 articles to consider what has been learned from using administrative data to conduct longitudinal substance abuse research in four overlapping areas: (1) service access and utilization, (2) underrepresented populations, (3) treatment outcomes, and (4) cost analysis. Despite several notable limitations, administrative data contribute valuable information, particularly in the investigation of service system interactions and outcomes among substance abusers as they unfold and influence each other over the long term. This critical assessment of the advantages and disadvantages of using existing administrative data within a longitudinal framework should stimulate innovative thinking regarding future applications of administrative data for longitudinal substance abuse research purposes.