The effectiveness of clinician feedback in the treatment of depression in the community mental health system

Gender, Race/Ethnicity, and Patient-Therapist Matching on Gender and Race/Ethnicity: Predictors/Moderators of the Effectiveness of Trust/Respect Feedback

A plethora of studies has exhibited the effectiveness of using measurement-based care feedback systems within mental health services to improve treatment outcomes; however, patient gender/race and patient-therapist matching on gender and race remain relatively unexplored as predictors/moderators in feedback studies. We conducted predictor/moderator analyses focusing on the relation of gender, race/ethnicity, and patient-therapist gender and race/ethnicity matching on two outcomes: patient self-reported levels of (1) functioning and (2) trust/respect within the therapeutic relationship. We used data from a randomized controlled trial studying the effectiveness of a feedback system comparing patient-reported levels of trust and respect towards their provider (together with symptom feedback) in comparison to symptom only feedback. We found that men improved in functioning more than women when their therapists received trust/respect feedback compared to symptom only feedback (F[1, 902] = 9.79, p = .002, d = 0.21). We also found that dyads matched on race/ethnicity but not gender, and those matched on gender but not race/ethnicity, improved in functioning over time more than dyads mismatched on both gender and race/ethnicity and those matched on both gender and race/ethnicity (F[1, 897] = 8.63, p = .0034, d = 0.20). On trust/respect outcomes, we found a gender difference over time (F[1, 759] = 6.61, p = .01, d = 0.19), a gender matching difference by feedback condition interaction (F[1, 757] = 5.25, p = .02, d = 0.17), and a racial/ethnic matching difference on trust/respect scores over time (F[1, 785] = 3.89, p = .049, d = 0.14). Male patients showed an initial decrease followed by a steady increase in trust/respect over time while female patients showed an initial increase followed by a steady decrease. Gender-matched therapeutic dyads showed higher levels of trust/respect compared to mismatched dyads when therapists received symptom only feedback, but this difference was not apparent when trust/respect feedback was provided. Dyads mismatched on race/ethnicity improved steadily in trust/respect over time, but matched dyads decreased in trust/respect after an initial increase. Future research should focus on the use of feedback systems to enhance outcomes for patients with specific gender and racial/ethnic identities.

Process and Implementation Elements of Measurement Feedback Systems: A Systematic Review

Measurement feedback systems (MFS) can help guide treatment and improve clinical outcomes. Studies of MFS are heterogeneous both in execution and results, and the effects of MFS seem restricted by limited attention to process and implementation elements and by limited adoption by health professionals. The current systematic review mapped the use of process and implementation elements in MFS studies. An overview of therapists' use of and attitudes toward MFS is provided. Three-level meta-analyses were used to test theoretically informed process and implementation elements as moderators of the effects of MFS. Hypotheses and general propositions from Clinical Performance Feedback Intervention Theory (CP-FIT) were used to organize the elements of the studies and were used as moderator variables. Previous studies on MFS interventions have had a limited focus on implementation efforts and process elements that may increase the effects of MFS and their use among therapists. Efforts have sparsely been made to reduce barriers to MFS use, and several studies have reported limited engagement with MFS among therapists. Therapists' attitudes toward MFS, feedback, or standardized measures were heterogeneously reported, making data synthesis challenging. Identified process and implementation elements were not significantly associated with effect sizes in the studies and the results did not support the propositions of CP-FIT. The lack of statistically significant associations may be due to limited reporting of details about process and implementation aspects. More research designed to test hypotheses regarding process and implementation elements is needed to improve the use and effects of MFS. Future studies should aspire to report findings in a manner that allows for an understanding of the implementation process and therapists' adoption of these systems.

A systematic review on the effect of routine outcome monitoring and feedback on client outcomes in alcohol and other drug treatment

ISSUES

Routine outcome monitoring (ROM) involves regularly measuring clients' outcomes during treatment, which can then be fed back to clinicians and/or clients. In the mental health field, ROM and feedback have been shown to improve client outcomes; however, no systematic reviews have examined whether improvement is also seen in alcohol and other drug (AOD) treatment outcomes. This review examines whether feedback to clients and/or clinicians of ROM data in AOD treatment improves future client outcomes.

APPROACH

This systematic review of papers identified in Medline, PsycInfo and Scopus examines the effect on client outcomes of feeding back ROM data to clinicians and/or clients in AOD treatment settings. Key client outcomes included substance use, treatment attendance and wellbeing measures.

KEY FINDINGS

Ten studies were included-five randomised controlled trials and five pre-post within-subjects designs. Six studies were deemed good- or fair-quality. Of these six, three provided feedback to clinicians only, one to clients only, and two to both clients and clinicians. Only one of the six found feedback was associated with significant reductions in substance use and only among off-track clients. Four of the six found feedback improved other outcomes, including treatment retention, global functioning, therapeutic alliance and mood symptoms.

CONCLUSIONS

There may be some positive effects for clients of providing feedback to clients and/or clinicians; however, the small number of randomised trials and the heterogeneity of methods, outcome measures and findings, mean that firm conclusions cannot be drawn about the efficacy of feedback until larger randomised studies are conducted.

Repeated assessments of depressive symptoms in randomized psychosocial intervention trials: best practice for analyzing symptom change over time

OBJECTIVE

Psychotherapy randomized trials rarely have tested for the best fitting model for time effects. We examined the fit of different statistical models for examining time when repeated assessments of depressive symptoms are the primary outcome.

METHOD

We used data from three studies comparing psychotherapy treatments for major depressive disorder. Outcome measures were self-report ratings for Study 1 (N = 237) and Study 2 (N = 100) and clinician ratings for Study 3 (N = 120) of depressive symptoms measured at every session (Studies 1 and 2) or monthly (Study 3). We examined the fit of the following time patterns: linear, quadratic, cubic, log transformation of time, piece-wise linear, and unstructured.

RESULTS

In Study 1, a log-linear model had the best fit (Δ Akaike information criterion [AICc] = 7.5). In Study 2, all models had essentially no support (Δ AICcs > 10) in comparison to the best fitting model, which was the unstructured model. In Study 3, the cubic model had the best fit, but it was not significantly better than a log-linear (Δ AICc = 3.5) or unstructured model (Δ AICc = 2.5).

CONCLUSIONS

Trials should routinely compare different time models, including an unstructured model, when repeated measures of depressive symptoms are the primary outcome.

Initial Development of Pragmatic Behavioral Activation Fidelity Assessments

PURPOSE

Our goal was to develop brief pragmatic assessments of Behavioral Activation (BA) fidelity to support its dissemination in low-resource settings.

METHODS

We used qualitative and quantitative methods across three investigations to develop pragmatic assessments rated from the perspective of therapists, patients, and observers: (1) we developed an initial comprehensive pool of 119 items and adapted/refined the item pool to 32 items through stakeholder focus groups and cognitive interviews; (2) independent blind judges rated each of items in the refined item pool on an early session of BA for 64 patients to support the selection of items based on predictive validity; and (3) we conducted a preliminary evaluation of the acceptability and feasibility of the assessments of BA fidelity from the perspective of therapists and patients.

RESULTS

The internal consistency reliability for the 10-item total score was .83 rated from the perspective of independent observers. The assessment was completed by patients following 90% of sessions and by clinicians following 93% of sessions. Items were rated high on overall satisfaction by both therapists (M = 4.6, SD = 0.89) and patients (M = 4.8, SD = 0.41).

CONCLUSION

Our findings suggest that these brief assessments of BA fidelity are reliable, feasible, and acceptable to community stakeholders.

Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis

Progress feedback is an intervention aimed at enhancing patient outcomes in routine clinical practice. This study reports a comprehensive multilevel meta-analysis on the effectiveness of progress feedback in psychological treatments in curative care. The short- and long-term effects of feedback on symptom reduction were investigated using 58 (randomized and non-randomized) studies, analyzing 110 effect sizes in a total of 21,699 patients. Effects of feedback on dropout rate, percentage of deteriorated cases, and treatment duration were also examined. Moderation analyses were conducted for study and feedback characteristics. A small significant effect of progress feedback on symptom reduction (d = 0.15, 95% CI: [0.10, 0.20]) was found, compared to control groups. This was also true for not-on-track cases (d = 0.17, 95% CI: [0.11, 0.22]). In addition, feedback had a small favorable effect on dropout rates (OR = 1.19, 95% CI: [1.03, 1.38]). The moderation analyses identified several potentially interesting variables for further research, including feedback instrument, outcome instrument, type of feedback, feedback frequency, treatment intensity, and country in which the study was conducted. Future studies should report on these variables more consistently so that we can obtain a better understanding of when and why feedback improves outcomes.

A Feasibility Study of Behavioral Activation for Major Depressive Disorder in a Community Mental Health Setting

Behavioral activation (BA) is a well-supported treatment approach, but little research has been conducted on the effectiveness of this treatment within publicly funded community mental health settings. We examined the feasibility of conducting a randomized trial examining the effectiveness of nine sessions of BA as a treatment for major depressive disorder (MDD) in a community mental health clinic (CMHC) setting. Following adaptation of a BA manual and training of BA therapists, 80 patients seeking treatment at a CMHC were randomized, with a 3:1 randomization rate of BA to nine sessions of treatment as usual (TAU). Feasibility assessments indicated that only one eligible patient refused randomization and, of patients who attended at least one session, the median number of sessions was six for the BA group and eight for the TAU group. Of three postbaseline monthly assessments, 71.3% (171/240) were successfully obtained. On average, patients in the BA condition completed homework assignments 83.9% of the time. Treatment fidelity ratings indicated that substantially more BA techniques were delivered in the BA group compared to the TAU group (d = 2.11). Measures of BA mechanisms improved significantly over time and these changes were significantly associated with change in depressive symptoms. These results indicate that it is feasible to conduct a randomized study of BA for MDD in a CMHC setting. In addition, the study reconfirmed the potential importance of theory-relevant BA mechanism variables. Following these findings, further investigation into the effectiveness of BA in this setting is needed.

The predictive validity of the progress assessment, a clinician administered instrument for use in measurement-based care for substance use disorders

We tested the predictive validity of the Progress Assessment (PA), a brief counselor administered tool for use in measurement-based care for substance use disorders. The PA includes 5 items assessing relapse risk and 5 items assessing factors protective against relapse. Data were drawn from a completed study of continuing care for cocaine dependence (McKay et al., 2013) and includes 12 months of follow-up on158 participants (76% male) who received brief telephone or face-to-face sessions. Each session began with the administration of the PA, followed by cognitive-behavioral counseling tied to the results of the PA and anticipated risky situations. Outcome was assessed via urine toxicology every 3 months. As administered in an effectiveness trial, average PA risk and protective scales within each 3-month segment of the study predicted urine toxicology results at the end of that period, with higher risk scores and lower protective scores predicting greater rates of cocaine positive urine drug screens. PA scores did not predict dropout from continuing care participation. The 10-item PA shows promise as a pragmatic clinical tool for ongoing monitoring during continuing care for substance dependence.

Improving Mental Health Services: A 50-Year Journey from Randomized Experiments to Artificial Intelligence and Precision Mental Health

This conceptual paper describes the current state of mental health services, identifies critical problems, and suggests how to solve them. I focus on the potential contributions of artificial intelligence and precision mental health to improving mental health services. Toward that end, I draw upon my own research, which has changed over the last half century, to highlight the need to transform the way we conduct mental health services research. I identify exemplars from the emerging literature on artificial intelligence and precision approaches to treatment in which there is an attempt to personalize or fit the treatment to the client in order to produce more effective interventions.

A qualitative evaluation of Veterans Health Administration's implementation of measurement-based care in behavioral health

Measurement-based care (MBC) in behavioral health involves the repeated collection of patient-reported data that is used to track progress, inform care, and engage patients in shared decision making about their treatment. Research suggests that MBC increases the quality and effectiveness of mental health care. However, there can be challenges to implementing MBC, such as time burden, lack of resources to support MBC, and clinician attitudes. The Veterans Health Administration (VHA) is currently undertaking a multiphase MBC roll-out, the first phase of which included 59 sites across the country. The present study examined implementation of this initiative in an effort to learn more about the process of implementation, including best practices, challenges, and innovations. Semistructured interviews were conducted with 20 MBC site champions and 60 staff members from 25 VHA medical centers across the country. Qualitative data analysis was conducted to identify key themes related to MBC implementation. Results were described for 3 components of MBC implementation: preparing for implementation, administering measures, and using and sharing data. Training and staff buy-in were key to the preparation phase. Staff members reported a variety of methods and frequencies for the collection of MBC data, with many staff members identifying a need to streamline the collection process. Staff members reported using data to track progress and adjust treatment with patients. Efforts to use data on a programmatic level were identified as a next step. Innovative solutions across clinics and sites are described in an effort to inform future MBC implementation, both within and outside of VHA. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The Therapy Process Questionnaire - Factor analysis and psychometric properties of a multidimensional self-rating scale for high-frequency monitoring of psychotherapeutic processes

Many outcome measures and session‐related questionnaires in psychotherapy are designed for weekly or biweekly administration. Yet, today, technical developments allow for higher frequency assessments to monitor human change dynamics more closely by daily assessments. For this purpose, the Therapy Process Questionnaire (TPQ) was developed, with a specific focus on inpatient psychotherapy. In this article, we present an explorative and confirmative factor analysis of the TPQ on the basis of the time series data of 150 patients collected during their hospital stay (mean time series length: 69.1 measurement points). A seven‐factor solution was identified, which explains 68.7% of variance and associates 43 items onto the factors, which are “well‐being and positive emotions,” “relationship with fellow patients,” “therapeutic relationship and clinical setting,” “emotional and problem intensity,” “insight/confidence/therapeutic progress,” “motivation for change,” and “mindfulness/self‐care.” The internal consistency (Cronbach's α), the inter‐item correlations of the subscales, and the discriminative power of the items are excellent. The TPQ can be applied in practice and research for creating time series with equidistant measurement points and time series lengths, which are appropriate for the application of nonlinear analysis methods. Especially in clinical practice, it is important to identify precursors of phase transitions, changing synchronization patterns, and critical or instable periods of a process, which now is possible by internet‐ or app‐based applications of this multidimensional questionnaire.

Predictors of treatment attendance in cognitive and dynamic therapies for major depressive disorder delivered in a community mental health setting

OBJECTIVE

Our goal was to evaluate treatment attendance patterns, including both treatment completion and premature termination from treatment, for 2 evidence-based psychotherapies for major depressive disorder (MDD) delivered in a community mental health setting. We explored rates of premature termination across the course of treatment as well as the factors that predicted and moderated premature termination and treatment completion.

METHOD

This investigation included 237 patients with MDD who participated in a noninferiority trial comparing short-term dynamic psychotherapy (DT) to cognitive therapy (CT). Patients in both conditions were offered 16 sessions of treatment and had up to 5 months to complete treatment. All patients completed an extensive self-report battery at treatment baseline as well as measures of the therapeutic alliance and opinions about treatment following Session 2.

RESULTS

Premature termination from both treatments was high with 27% of patients discontinuing treatment very early after only an intake session or a single treatment session. Patients in CT were significantly more likely to terminate treatment prematurely, χ²(3) = 14.35, p = .002. Baseline physical health functioning, subthreshold psychotic symptoms, Session 2 ratings of agreement on tasks, and Session 2 ratings of treatment sensibility all independently predicted premature termination of services. Trauma history significantly moderated very early termination of treatment, χ²(3) = 10.26, p = .017, with patients with high trauma histories more likely to complete DT but terminate prematurely from CT.

CONCLUSIONS

Very early termination from services was higher in CT compared with DT. Including techniques to improve engagement in both therapies and matching patients to treatment based on predictors/moderators may be effective ways to optimize treatment engagement. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Testing a mediation model of psychotherapy process and outcome in psychodynamic psychotherapy: Previous client distress, psychodynamic techniques, dyadic working alliance, and current client distress

To test a sequential model of psychotherapy process and outcome, we included previous client distress, therapist psychodynamic techniques, dyadic working alliance, and current client distress. For 114 sets of eight-session segments in 40 cases of psychodynamic psychotherapy, clients completed the Outcome Questionnaire-45 and Inventory of Interpersonal Problems-32 after the first and final session, judges reliably coded one middle sessions on the Psychodynamic subscale of the Multitheoretical List of Therapeutic Interventions, and clients and therapists completed the Working Alliance Inventory after every session. Results indicated that higher use of psychodynamic techniques was associated with higher levels of the working alliance, which in turn was associated decreased client distress; and working alliance was higher later in psychotherapy. There was a significant indirect effect of psychodynamic techniques on decreases in distress mediated by the working alliance. Implications for theory, practice, and research are provided. Clinical or methodological significance of this article: Conducted a longitudinal, latent variable examination of the relationships of psychodynamic techniques and working alliance on client distress. Psychodynamic techniques have an indirect effect on decreases in client distress through the dyadic working alliance.

A conceptual model for generating and validating in-session clinical judgments

OBJECTIVE

Little attention has been paid to the nuanced and complex decisions made in the clinical session context and how these decisions influence therapy effectiveness. Despite decades of research on the dual-processing systems, it remains unclear when and how intuitive and analytical reasoning influence the direction of the clinical session.

METHOD

This paper puts forth a testable conceptual model, guided by an interdisciplinary integration of the literature, that posits that the clinical session context moderates the use of intuitive versus analytical reasoning.

RESULTS

A synthesis of studies examining professional best practices in clinical decision-making, empirical evidence from clinical judgment research, and the application of decision science theories indicate that intuitive and analytical reasoning may have profoundly different impacts on clinical practice and outcomes.

CONCLUSIONS

The proposed model is discussed with respect to its implications for clinical practice and future research.

Development and Preliminary Feasibility Testing of a Decision Support Tool for Childhood Anxiety Treatment

In this practical application, we describe the steps to build a decision-support tool using GeNIe 2.1 software. The method incorporates principles of decision analyses and allows for a systematic strategy to balance treatment efficacy data with patient preferences. We illustrate the utility for helping clinicians and patients choose between two or more efficacious treatment options (CBT, medication, or their combination). Preliminary pilot data from families (n = 5) seeking services at a specialty clinic for childhood anxiety disorders support the usability of the tool and high patient satisfaction. We use case examples and sample graphical output to illustrate how the decision-support system can be used to integrate data on, 1) baseline symptom severity 2) the relative effectiveness of two or more treatment options, and 3) patient preferences and values, to arrive at a personalized treatment recommendation. The decision-support tool enabled child and parent preferences to be explicitly stated and facilitated discussions about how best to incorporate their preferences into an evidenced-based treatment strategy.

Bridging the Gap Between Research and Practice in Mental Health Service Settings: An Overview of Developments in Implementation Theory and Research

Twenty years after Sobell's (1996) influential call to integrate advances in behavior therapy into clinical settings, significant progress has been made in implementation science. In this narrative review, we provide an overview of implementation research findings and highlight recent findings that can inform efforts to bridge the gap between research and practice in mental health service settings. Key findings are summarized, organized according to levels of influence described in two implementation frameworks: The Exploration, Planning, Implementation, and Sustainment Framework (EPIS; Aarons et al., 2011) and the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009). Important levels of influence to consider when implementing new treatments include the outer context, inner context, characteristics of the individual, and characteristics of the innovation. Research on strategies to prepare clinicians to deliver evidence-based psychosocial treatments (EBPTs) and to address contextual barriers to implementation at each level is described, with discussion of implications for the implementation of EBPTs and next steps for research.

Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults

BACKGROUND

Routine outcome monitoring of common mental health disorders (CMHDs), using patient reported outcome measures (PROMs), has been promoted across primary care, psychological therapy and multidisciplinary mental health care settings, but is likely to be costly, given the high prevalence of CMHDs. There has been no systematic review of the use of PROMs in routine outcome monitoring of CMHDs across these three settings.

OBJECTIVES

To assess the effects of routine measurement and feedback of the results of PROMs during the management of CMHDs in 1) improving the outcome of CMHDs; and 2) in changing the management of CMHDs.

SEARCH METHODS

We searched the Cochrane Depression Anxiety and Neurosis group specialised controlled trials register (CCDANCTR-Studies and CCDANCTR-References), the Oxford University PROMS Bibliography (2002-5), Ovid PsycINFO, Web of Science, The Cochrane Library, and International trial registries, initially to 30 May 2014, and updated to 18 May 2015.

SELECTION CRITERIA

We selected cluster and individually randomised controlled trials (RCTs) including participants with CMHDs aged 18 years and over, in which the results of PROMs were fed back to treating clinicians, or both clinicians and patients. We excluded RCTs in child and adolescent treatment settings, and those in which more than 10% of participants had diagnoses of eating disorders, psychoses, substance use disorders, learning disorders or dementia.

DATA COLLECTION AND ANALYSIS

At least two authors independently identified eligible trials, assessed trial quality, and extracted data. We conducted meta-analysis across studies, pooling outcome measures which were sufficiently similar to each other to justify pooling.

MAIN RESULTS

We included 17 studies involving 8787 participants: nine in multidisciplinary mental health care, six in psychological therapy settings, and two in primary care. Pooling of outcome data to provide a summary estimate of effect across studies was possible only for those studies using the compound Outcome Questionnaire (OQ-45) or Outcome Rating System (ORS) PROMs, which were all conducted in multidisciplinary mental health care or psychological therapy settings, because both primary care studies identified used single symptom outcome measures, which were not directly comparable to the OQ-45 or ORS.Meta-analysis of 12 studies including 3696 participants using these PROMs found no evidence of a difference in outcome in terms of symptoms, between feedback and no-feedback groups (standardised mean difference (SMD) -0.07, 95% confidence interval (CI) -0.16 to 0.01; P value = 0.10). The evidence for this comparison was graded as low quality however, as all included studies were considered at high risk of bias, in most cases due to inadequate blinding of assessors and significant attrition at follow-up.Quality of life was reported in only two studies, social functioning in one, and costs in none. Information on adverse events (thoughts of self-harm or suicide) was collected in one study, but differences between arms were not reported.It was not possible to pool data on changes in drug treatment or referrals as only two studies reported these. Meta-analysis of seven studies including 2608 participants found no evidence of a difference in management of CMHDs between feedback and no-feedback groups, in terms of the number of treatment sessions received (mean difference (MD) -0.02 sessions, 95% CI -0.42 to 0.39; P value = 0.93). However, the evidence for this comparison was also graded as low quality.

AUTHORS' CONCLUSIONS

We found insufficient evidence to support the use of routine outcome monitoring using PROMs in the treatment of CMHDs, in terms of improving patient outcomes or in improving management. The findings are subject to considerable uncertainty however, due to the high risk of bias in the large majority of trials meeting the inclusion criteria, which means further research is very likely to have an important impact on the estimate of effect and is likely to change the estimate. More research of better quality is therefore required, particularly in primary care where most CMHDs are treated.Future research should address issues of blinding of assessors and attrition, and measure a range of relevant symptom outcomes, as well as possible harmful effects of monitoring, health-related quality of life, social functioning, and costs. Studies should include people treated with drugs as well as psychological therapies, and should follow them up for longer than six months.

Increasing psychotherapists' adoption and implementation of the evidence-based practice of progress monitoring

Evidence-based practices (EBPs) reach consumers slowly because practitioners are slow to adopt and implement them. We hypothesized that giving psychotherapists a tool + training intervention that was designed to help the therapist integrate the EBP of progress monitoring into his or her usual way of working would be associated with adoption and sustained implementation of the particular progress monitoring tool we trained them to use (the Depression Anxiety Stress Scales on our Online Progress Tracking tool) and would generalize to all types of progress monitoring measures. To test these hypotheses, we developed an online progress monitoring tool and a course that trained psychotherapists to use it, and we assessed progress monitoring behavior in 26 psychotherapists before, during, immediately after, and 12 months after they received the tool and training. Immediately after receiving the tool + training intervention, participants showed statistically significant increases in use of the online tool and of all types of progress monitoring measures. Twelve months later, participants showed sustained use of any type of progress monitoring measure but not the online tool.