The Difficult Task of Assessing and Interpreting Treatment Deterioration: An Evidence-Based Case Study

How do we know whether treatment has failed? Paradoxical outcomes in counseling with young people

Background

In both routine practice contexts and research studies, evidence from standardized self-report symptom measures, administered pre- and post-treatment, is predominantly used to determine whether psychotherapy has been successful. Understanding the nature of unsuccessful psychotherapy requires an ability to evaluate the credibility of outcome data generated by such techniques. An important body of research has identified discrepancies between outcomes assessed through symptom measures and those obtained from other sources. However, not enough is known about the extent to which such paradoxical outcomes exist.

Objective

This study analyzes the relationship between outcomes, as assessed by a standardized self-report measure, and as assessed by ratings of young people's descriptions of change at post-counseling interviews.

Methods

Participants were 50 young people (13-16 years old) who had taken part in a trial of up to 10 weeks of school-based humanistic counseling. Our primary standardized measure was the Young Person's CORE (YP-CORE). To assess young people's experiences of counseling change, three independent raters scrutinized transcripts of post-counseling interviews, and scored levels of helpfulness on a 1 (Not at all helpful) to 10 (Extremely helpful) scale. Inter-rater reliabilities were 0.94 (Cronbach's Alpha) and 0.96 (McDonald's Omega). Sensitivity analyses were conducted to explore relationships between helpfulness ratings and other outcome measures, i.e., satisfaction with counseling (ESQ) and the Goal-Based-Outcome Tool (GBO), and process measures, i.e., the Working Alliance Inventory (WAI-S) and the Barret Lennard Relationship Inventory (BLRI).

Results

Multilevel analysis indicated that helpfulness ratings were not significantly associated with changes in YP-CORE scores. Analyzed categorically, 38% of those showing reliable improvement on the standardized measure were below the median for self-described helpfulness, and 47% of those not showing reliable change were at or above the median for self-described helpfulness. Sensitivity analyses demonstrated closer correlations between helpfulness ratings and other outcome measures (ESQ and GBO), and between helpfulness ratings and process measures (WAI-S and BLRI).

Discussion

Our results raise questions about reliance on symptom change outcome measures for defining treatment success and failure, given their disparity with clients' own descriptions of the helpfulness of therapy. Implications for practice and research are discussed.

Mental Health Specialist Video Consultations Versus Treatment-as-Usual for Patients With Depression or Anxiety Disorders in Primary Care: Randomized Controlled Feasibility Trial

BACKGROUND

Most people affected by depression or anxiety disorders are treated solely by their primary care physician. Access to specialized mental health care is impeded by patients' comorbidity and immobility in aging societies and long waiting times at the providers' end. Video-based integrated care models may leverage limited resources more efficiently and provide timely specialized care in primary care settings.

OBJECTIVE

The study aims to evaluate the feasibility of mental health specialist video consultations with primary care patients with depression or anxiety disorders.

METHODS

Participants were recruited by their primary care physicians during regular practice visits. Patients who had experienced at least moderate symptoms of depression and/or anxiety disorders were considered eligible for the study. Patients were randomized into 2 groups receiving either treatment-as-usual as provided by their general practitioner or up to 5 video consultations conducted by a mental health specialist. Video consultations focused on systematic diagnosis and proactive monitoring using validated clinical rating scales, the establishment of an effective working alliance, and a stepped-care algorithm within integrated care adjusting treatments based on clinical outcomes. Feasibility outcomes were recruitment, rate of loss to follow-up, acceptability of treatment, and attendance at sessions. Effectiveness outcomes included depression (Patient Health Questionnaire-9), anxiety (Generalized Anxiety Disorder-7), burden of specific somatic complaints (Somatic Symptom Disorder-B Criteria Scale-12), recovery (Recovery Assessment Scale-German [RAS-G]), and perception of chronic illness care (Patient Assessment of Chronic Illness Care), which were measured at baseline and 16 weeks postallocation by assessors blinded to the group allocation.

RESULTS

A total of 50 patients with depression and/or anxiety disorders were randomized, 23 in the intervention group and 27 in the treatment-as-usual group. The recruitment yield (number randomized per number screened) and the consent rate (number randomized per number eligible) were 69% (50/73) and 86% (50/58), respectively. Regarding acceptability, 87% (20/23) of the participants in the intervention group completed the intervention. Of the 108 planned video consultations, 102 (94.4%) were delivered. Follow-up rates were 96% (22/23) and 85% (23/27) for the intervention and control groups, respectively. The change from baseline scores at postmeasurement for the No Domination by Symptoms domain of recovery (RAS-G) was somewhat higher in the intervention group than in the control group (Mann-Whitney U test: rank-biserial r=0.19; 95% CI -0.09 to 0.46; P=.18). We did not detect any notable differences between the intervention and control groups in terms of other effectiveness outcomes. We did not observe any serious adverse events related to the trial.

CONCLUSIONS

The intervention and study procedures were found to be feasible for patients, primary care practice staff, and mental health specialists. A sufficiently powered pragmatic trial on mental health specialist video consultations should be conducted to investigate their effectiveness in routine care.

TRIAL REGISTRATION

German Clinical Trials Register DRKS00015812; https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00015812.