Feedback and therapist effects in the context of treatment outcome and treatment length

Disentangling the Therapist Effect: Clustering Therapists by Using Different Treatment Outcomes

Previous studies have shown that therapists' performance varies, known as therapist effects, and have indicated that therapists who excel in one treatment outcome may not necessarily be effective in other outcomes. This observational naturalistic study aimed to enhance our understanding of therapist effects and the assessment of therapists' performance in different areas. The study included 68 therapists and 5,582 clients from a large mental health facility. Information about their learning activities was available for a subsample of 49 therapists. Separate multilevel analyses were conducted for treatment outcomes, including case mix-corrected OQ-45 change scores, dropout rates, referrals to other facilities, treatment duration, and client satisfaction ratings. A hierarchical cluster analysis was performed to identify groups of therapists based on their performance across various treatment outcomes. Additionally, differences in therapist characteristics among the clusters identified were examined. Therapist effects varied across different outcomes, ranging from small (2.6% for OQ-45 change) to moderate (6.5% for number of sessions). The cluster analysis revealed four distinct clusters of therapists with specific profiles. They had performance differences in certain areas but not in others. This exploratory study supports the notion that therapists exhibit diverse profiles regarding treatment outcomes. These findings are significant for future investigations of therapist effects that aim to identify the characteristics of effective therapists and in the context of personalizing treatment for clients.

Love yourself as a therapist, doubt yourself as an institution? Therapist and institution effects on outcome, treatment length, and dropout

OBJECTIVE

Research suggests that some therapists achieve better outcomes than others. However, an overlooked area of study is how institution differences impact patient outcomes independent of therapist variance. This study aimed to examine the role of institution and therapist differences in adult outpatient psychotherapy.

METHOD

The study included 1428 patients who were treated by 196 therapists at 10 clinics. Two- and three-level hierarchical linear regression models were employed to investigate the effects of therapists and institutions on three dependent patient variables: (1) symptom change, (2) treatment duration, and (3) dropout. Level three explanatory variables were tested.

RESULTS

The results showed that therapist effects (TE) were significant for all three types of treatment outcome (7.8%-18.2%). When a third level (institution) was added to the model, the differences between therapists decreased, and significant institution effects (IE) were found: 6.3% for symptom change, 10.6% for treatment duration, and 6.5% for dropout. The exploratory analyses found no predictors able to explain the systematic variation at the institution level.

DISCUSSION

TE on psychotherapy outcomes remain a relevant factor but may have been overestimated in previous studies due to not properly distinguishing them from differences at the institution level.

Using Progress Feedback to Enhance Treatment Outcomes: A Narrative Review

We face increasing demand for greater access to effective routine mental health services, including telehealth. However, treatment outcomes in routine clinical practice are only about half the size of those reported in controlled trials. Progress feedback, defined as the ongoing monitoring of patients' treatment response with standardized measures, is an evidence-based practice that continues to be under-utilized in routine care. The aim of the current review is to provide a summary of the current evidence base for the use of progress feedback, its mechanisms of action and considerations for successful implementation. We reviewed ten available meta-analyses, which report small to medium overall effect sizes. The results suggest that adding feedback to a wide range of psychological and psychiatric interventions (ranging from primary care to hospitalization and crisis care) tends to enhance the effectiveness of these interventions. The strongest evidence is for patients with common mental health problems compared to those with very severe disorders. Effect sizes for not-on-track cases, a subgroup of cases that are not progressing well, are found to be somewhat stronger, especially when clinical support tools are added to the feedback. Systematic reviews and recent studies suggest potential mechanisms of action for progress feedback include focusing the clinician's attention, altering clinician expectations, providing new information, and enhancing patient-centered communication. Promising approaches to strengthen progress feedback interventions include advanced systems with signaling technology, clinical problem-solving tools, and a broader spectrum of outcome and progress measures. An overview of methodological and implementation challenges is provided, as well as suggestions for addressing these issues in future studies. We conclude that while feedback has modest effects, it is a small and affordable intervention that can potentially improve outcomes in psychological interventions. Further research into mechanisms of action and effective implementation strategies is needed.

Routine Outcome Monitoring and Clinical Feedback in Psychotherapy: Recent Advances and Future Directions

In the past decade, there has been an increase in research related to the routine collection and active use of standardized patient data in psychotherapy. Research has increasingly focused on personalization of care to patients, clinical skills and interventions that modulate treatment outcomes, and implementation strategies, all of which appear to enhance the beneficial effects of ROM and feedback. In this article, we summarize trends and recent advances in the research on this topic and identify several essential directions for the field in the short to medium term. We anticipate a broadening of research from the focus on average effects to greater specificity around what kinds of feedback, provided at what time, to which individuals, in what settings, are most beneficial. We also propose that the field needs to focus on issues of health equity, ensuring that ROM can be a vehicle for increased wellbeing for those who need it most. The complexity of mental healthcare systems means that there may be multiple viable measurement solutions with varying costs and benefits to diverse stakeholders in different treatment contexts, and research is needed to identify the most influential components in each of these contexts.

Bridging the feedback implementation gap: A comparison of empirical and rational decision rules in naturalistic psychotherapy

Objective: Previous research indicates positive effects of feedback based on rational or empirical decision rules in psychotherapy. The implementation of these usually session-to-session-based feedback systems into clinical practice, however, remains challenging. This study aims to evaluate decision rules based on routine outcome monitoring with reduced assessment frequency. Method: Data routinely collected every 5-20 sessions of N = 3758 patients treated with CBT in an outpatient clinic (Msessions = 42.8, SD = 15.4) were used to develop feedback decision rules based on the expected treatment response and nearest neighbors approach, the reliable change index, and method of percental improvement. The detection of patients at risk of treatment failure served as primary endpoint. Results: Significantly lower reliable improvement, higher reliable deterioration rates, and smaller effect sizes were found for patients identified at risk of treatment failure by all rules. The nearest neighbors-based approach showed the highest sensitivity regarding the detection of reliably deteriorated cases. Conclusion: Consistent with previous research, the empirical models outperformed the rational rules. Still, the first-time used percental improvement-based rule also showed satisfactory results. Overall, the results point to the potential of basic feedback systems that might be easier to implement in practice than session-to-session based systems.

To Use or Not to Use: Exploring Therapists' Experiences with Pre-Treatment EMA-Based Personalized Feedback in the TheraNet Project

BACKGROUND

Using idiographic network models in psychotherapy has been a growing area of interest. However, little is known about the perceived clinical utility of network models. The present study aims to explore therapists' experiences with network model-based feedback within the context of the TheraNet Project.

METHODS

In total, 18 therapists who had received network-based feedback for at least 1 patient at least 2 months prior were invited to retrospective focus groups. The focus group questions related to how participation in the study influenced the therapeutic relationship, how the networks were used, and what might improve their clinical utility. The transcribed focus groups were analyzed descriptively using qualitative content analysis.

RESULTS

Most therapists mentioned using the feedback to support their existingtheir case concept, while fewer therapists discussed the feedback directly with the patients. Several barriers to using the feedback were discussed, as well as various suggestions for how to make it more clinically useful. Many therapists reported skepticism with regards to research in the outpatient training center in general, though they were also all pleasantly surprised by being involved, having their opinions heard, and showing a readiness to adapt research to their needs/abilities.

CONCLUSIONS

This study highlights the gap between researchers' and therapists' perceptions about what useful feedback should look like. The TheraNet therapists' interest in adapting the feedback and building more informative feedback systems signals a general openness to the implementation of clinically relevant research. We provide suggestions for future implementations of network-based feedback systems in the outpatient clinical training center setting.

Therapeutic alliance in two forms of guided self-help for binge eating

The role of therapeutic alliance within psychological treatments for eating disorders (EDs), including those delivered remotely, is well established. However, few studies have investigated alliance in guided self-help, a widely recommended first-line treatment for EDs characterised by regular binge eating. Using data from a randomised controlled trial, the current study examined both facilitator and patient assessments of alliance within e-mail-assisted and face-to-face guided self-help and looked at associations between alliance, ED symptoms and ED-related impairment. One hundred thirteen patients and 11 facilitators completed measures of alliance during and following a course of guided self-help. Whilst ratings were reliable across patients and facilitators, alliance scores were higher both in the patient sample and in the face-to-face condition. Ratings of alliance showed no correlations with ED symptoms at post-treatment, and early alliance was not significantly associated with outcome, which could inform how early symptom change is encouraged in guided self-help.

[How Cooperation Instead of Coexistence in Psychotherapy Research can Improve Science, Practice and Continuing Education]

Die kontinuierliche Erhebung psychometrischer Daten vor, während und nach einer psychotherapeutischen Behandlung kann als Qualitätssicherungsmaßnahme Therapeut:innen in ihrer klinischen Arbeit unterstützen und zugleich eine belastbare Datengrundlage für die Psychotherapieforschung schaffen. Im Rahmen der Qualitätssicherung können die erhobenen Daten als zusätzliche Informationsquelle den klinischen Eindruck der Therapeut:innen erweitern und zur Evaluation der Behandlung am Einzelfall aber auch auf der Ebene des Versorgungssystems herangezogen werden. Darüber hinaus können prognostische Vorhersagen von Therapieergebnissen und Abbruchwahrscheinlichkeiten, Behandlungsempfehlungen sowie adaptive Behandlungsanpassungen während der Behandlung auf ihnen aufgebaut werden, die Therapeut:innen in ihren klinischen Entscheidungen unterstützen 1. Eine solche daten-gestützte und evidenzbasierte psychologische Psychotherapie kann die wissenschaftliche Fundierung der therapeutischen Herangehensweise und die Wirksamkeit der Behandlung verbessern. Dazu bedarf es umfangreicher Datenerhebungen, die verlässliche und aussagekräftige Forschungsbefunde ermöglichen 2.

Sociodemographic Factors as Predictors of the Duration of Long-term Psychotherapy: Evidence from a Finnish Nationwide Register Study

OBJECTIVE

The role of sociodemographic factors in determining psychotherapy duration has been largely unexplored despite their known association with treatment use. We examined the association between sociodemographic factors and rehabilitative psychotherapy treatment duration, as well as any changes in duration over time.

METHOD

We used three register-based nationally representative cohorts. Participants included employed Finnish individuals (n = 5572, 77% women, mean age = 37) who started psychotherapy treatment in 2011, 2013 or 2016 and were followed until 2019. We used negative binomial regression to examine the association between sociodemographic factors (age, gender, education, occupational status, income, geographical area of residence, and onset year of treatment) with treatment duration.

RESULTS

The mean treatment duration was 27 months (with a standard deviation of 12 months). Several sociodemographic factors were associated with treatment duration. Gender and education were found to have the largest impact on treatment duration, with females having a longer duration (IRR 1.08, 95% CI 1.04-1.11) and those with low education having a shorter duration (IRR 0.91, 95% CI 0.85-0.97), resulting in a difference of 2-3 months. Treatment duration also increased in later years, which suggests potentially increasing differences in treatment implementation. At largest, the combined effect of all factors corresponded to a 10-month difference in treatment duration.

CONCLUSIONS

The duration of long-term psychotherapy varied across the sociodemographic groups and increased in all studied groups in the 2010s.

Process and Implementation Elements of Measurement Feedback Systems: A Systematic Review

Measurement feedback systems (MFS) can help guide treatment and improve clinical outcomes. Studies of MFS are heterogeneous both in execution and results, and the effects of MFS seem restricted by limited attention to process and implementation elements and by limited adoption by health professionals. The current systematic review mapped the use of process and implementation elements in MFS studies. An overview of therapists' use of and attitudes toward MFS is provided. Three-level meta-analyses were used to test theoretically informed process and implementation elements as moderators of the effects of MFS. Hypotheses and general propositions from Clinical Performance Feedback Intervention Theory (CP-FIT) were used to organize the elements of the studies and were used as moderator variables. Previous studies on MFS interventions have had a limited focus on implementation efforts and process elements that may increase the effects of MFS and their use among therapists. Efforts have sparsely been made to reduce barriers to MFS use, and several studies have reported limited engagement with MFS among therapists. Therapists' attitudes toward MFS, feedback, or standardized measures were heterogeneously reported, making data synthesis challenging. Identified process and implementation elements were not significantly associated with effect sizes in the studies and the results did not support the propositions of CP-FIT. The lack of statistically significant associations may be due to limited reporting of details about process and implementation aspects. More research designed to test hypotheses regarding process and implementation elements is needed to improve the use and effects of MFS. Future studies should aspire to report findings in a manner that allows for an understanding of the implementation process and therapists' adoption of these systems.

The effect of therapist characteristics on the use and outcome of systematic client feedback in outpatient mental healthcare

OBJECTIVE

Therapist characteristics are known to affect treatment outcome in general and could also influence the use of systematic client feedback (SCF). The current study explores the effect of feedback orientation, regulatory focus, self-efficacy, attitude towards feedback resources and perceived feedback validity on the use and outcome of SCF in outpatient mental healthcare.

METHOD

The data of therapists (n = 12) and patients (n = 504) of two outpatient centres offering brief psychological treatment were analysed when SCF, based on the Partners for Change Outcome Management System (PCOMS), was added to treatment as usual. The data of therapists were obtained through a therapist questionnaire composed of relevant characteristics from feedback studies in social and organizational psychology. The effect on the use of SCF was analysed using logistic regression; whereas, the effect on outcome was assessed using a two-level multilevel analysis. Regular use of SCF and the Outcome Questionnaire (OQ-45) were used as outcome variables. DSM-classification, sex and age of each patient were included as covariates.

RESULTS

High perceived feedback validity significantly increased the use of SCF. No significant therapist characteristics effects were found on outcome, but high promotion focus was associated with treating more complex patients.

CONCLUSIONS

The perceived feedback validity of SCF is likely to have an influence on its use and is probably affected by the changes in the organizational climate.

Routine Outcome Monitoring (ROM) and Feedback: Research Review and Recommendations

OBJECTIVE

To provide a research review of the components and outcomes of routine outcome monitoring (ROM) and recommendations for research and therapeutic practice.

METHOD

A narrative review of the three phases of ROM - data collection, feeding back data, and adapting therapy - and an overview of patient outcomes from 11 meta-analytic studies.

RESULTS

Patients support ROM when its purpose is clear and integrated within therapy. Greater frequency of data collection is more important for shorter-term therapies, and use of graphs, greater specificity of feedback, and alerts are helpful. Overall effects on patient outcomes are statistically significant (g ≈ 0.15) and increase when clinical support tools (CSTs) are used for not-on-track cases (g ≈ 0.36-0.53). Effects are additive to standard effects of psychological therapies. Organizational, personnel, and resource issues remain the greatest obstacles to the successful adoption of ROM.

CONCLUSION

ROM offers a low-cost method for enhancing patient outcomes, on average resulting in an ≈ 8% advantage (success rate difference; SRD) over standard care. CSTs are particularly effective for not-on-track patients (SRD between ≈ 20% and 29%), but ROM does not work for all patients and successful implementation is a major challenge, along with securing appropriate cultural adaptations.

Assessment of modifications to evidence-based psychotherapies using administrative and chart note data from the US department of veterans affairs health care system

Background

The US Department of Veterans Affairs (VA) has over 15 years of experience in delivery of evidence-based psychotherapies (EBPs). This paper describes strategies for using clinical documentation and administrative data to understand adherence and modifications to EBPs for Posttraumatic Stress Disorder (PTSD).

Methods

This study focused on two EBPs for PTSD, Cognitive Processing Therapy (CPT) and Prolonged Exposure (PE). The sample included VA therapists from across the US who provided CPT and PE and the patients they treated over a 1-year period. The data sources for this study were templated EBP chart notes and VA administrative data. We used a manual review of note content and administrative data rules to code therapy adherence and modifications in 7,297 EBP sessions for 1,257 patients seen by 182 therapists. Two trained coders rated each therapy note and resolved discrepancies through consensus. To contextualize and explain variation in adherence and modifications, we conducted brief 30-45-min semi-structured interviews with a purposive subsample of these therapists (n = 32).

Findings

Combining manual chart review and administrative data allowed for identification of 11 types of modifications. Raters disagreed on adherence for 30% of notes. The disagreement stemmed from the presence of therapy modifications that were not clearly documented, necessitating the development of decision rules and strategies for modification coding. Both therapists and patients contributed to the variance in the extent to which different modifications occurred. Therapist interviews demonstrated therapist awareness of modifying the protocols in the ways identified through chart review.

Conclusion

Healthcare systems can use data collected as part of routine care to understand how and when EBPs are modified but need to develop scalable strategies to document adaptations and modifications to EBPs in routine care.

Predicting patients who will drop out of out-patient psychotherapy using machine learning algorithms

BACKGROUND

About 30% of patients drop out of cognitive-behavioural therapy (CBT), which has implications for psychiatric and psychological treatment. Findings concerning drop out remain heterogeneous.

AIMS

This paper aims to compare different machine-learning algorithms using nested cross-validation, evaluate their benefit in naturalistic settings, and identify the best model as well as the most important variables.

METHOD

The data-set consisted of 2543 out-patients treated with CBT. Assessment took place before session one. Twenty-one algorithms and ensembles were compared. Two parameters (Brier score, area under the curve (AUC)) were used for evaluation.

RESULTS

The best model was an ensemble that used Random Forest and nearest-neighbour modelling. During the training process, it was significantly better than generalised linear modelling (GLM) (Brier score: d = -2.93, 95% CI (-3.95, -1.90)); AUC: d = 0.59, 95% CI (0.11 to 1.06)). In the holdout sample, the ensemble was able to correctly identify 63.4% of cases of patients, whereas the GLM only identified 46.2% correctly. The most important predictors were lower education, lower scores on the Personality Style and Disorder Inventory (PSSI) compulsive scale, younger age, higher scores on the PSSI negativistic and PSSI antisocial scale as well as on the Brief Symptom Inventory (BSI) additional scale (mean of the four additional items) and BSI overall scale.

CONCLUSIONS

Machine learning improves drop-out predictions. However, not all algorithms are suited to naturalistic data-sets and binary events. Tree-based and boosted algorithms including a variable selection process seem well-suited, whereas more advanced algorithms such as neural networks do not.

[Effectivity of Inpatient and Day-clinic Psychosomatic-Psychotherapeutic Treatment]

AIM

(Partial) inpatient psychotherapy is well implemented in Germany. To better understand efficacy factors and effects, efficacy studies are necessary. This naturalistic study investigates the effectiveness of inpatient and day clinic psychotherapy as well as patient-and disorder-related factors influencing individual symptom improvements.

METHODS

Patients at a psychosomatic-psychotherapeutic university hospital treated from 2015 to 2019 who completed the BSI-18 either at admission and discharge (n=1366) or at admission and three-month catamnesis (n=497) were included in the analysis.

RESULTS

Improvements in global symptom severity showed moderate effect sizes. Descriptively, these were larger for day clinic patients than for those receiving treatment as inpatients-especially in the follow-up comparison (immediately after discharge: dinpatient=0.401, dday clinic=0.482; three months after discharge: dinpatient=0.403, dday clinical=0.807). Day hospitalized patients differed significantly from inpatients-slightly in age, employment status, ability to work and initial symptom burden, moderately in the number of mental comorbidities and strongly in their main diagnoses. Socio-demographic factors showed no positive influence on symptom improvement, initial symptom severity a moderate positive influence and the number of mental comorbidities a complex influence.

DISCUSSION

In general, this study confirms the effectiveness of (partial) inpatient psychosomatic therapy. The relevance of day clinic offers is emphasised in the context of cost efficiency and good integration into everyday life, but under consideration of individual treatment indications.

Working Towards the Development and Implementation of Precision Mental Healthcare: An Example

Leonard Bickman’s (2020) Festschrift paper in the special issue “The Future of Children’s Mental Health Services” on improving mental health services is an impressive reflection of his career, highlighting his major insights and the development of mental health services research as a whole. Five major difficulties in this field’s current research and practice are attentively delineated: poor diagnostics, measurement problems, disadvantages of randomized controlled trials (RCTs), lack of feedback and personalized treatments. Dr. Bickman recommends possible solutions based on his extensive experience and empirical findings. We agree with his thoughts and illustrate how we, challenged with the same problems, have attempted to develop clinically informed research and evidence-based clinical practice. A comprehensive feedback system that deals with the aforementioned problems is briefly described. It includes pre-treatment recommendations for treatment strategies and an empirically informed dropout prediction based on a variety of data sources. In addition to treatment recommendations, continuous feedback as well as individualized treatment adaptation tools are provided during ongoing therapy. New projects are being implemented to further improve the system by including new data assessment strategies and sources, e.g., ecological momentary assessment (EMA) and automated video analysis.

Are some therapists better at facilitating and consolidating sudden gains than others?

BACKGROUND

Changes during psychotherapy often include sudden symptom improvements, called sudden gains (SGs), which have been identified as being superior to gradual symptom change with regard to treatment success. This study investigates the role of therapists in initiating and/or consolidating SGs.

METHODS

The analyses are based on a sample of patients (N = 1937) who were seen by 155 therapists and received individual psychotherapy at a university outpatient clinic. First, the therapist effect (TE) on SG was investigated using multilevel modeling (MLM). Second, MLM was used to explore the relative importance of patient and therapist variability in SGs as they relate to outcome.

RESULTS

The TE on SGs accounted for 1.8% of variance, meaning that therapists are accountable for inter-individual differences in their patients' likelihood to experience SGs. Furthermore, results revealed a significant effect of SGs on outcome for both levels, while therapist differences regarding the consolidation of SGs were not significant.

CONCLUSIONS

The analyses indicated that some therapists are better in facilitating and initiating SGs. The process of triggering SGs seems to be a therapist skill or competence, which opens up an additional pathway to positive outcomes that could be used to improve clinical training.

Interpersonal clarification effects in Cognitive-Behavioral Therapy for depression and how they are moderated by the therapeutic alliance

BACKGROUND

Although a wide body of research links depression to interpersonal deficits, Cognitive-Behavioral Therapy (CBT), considered the gold standard in the treatment of this condition, has not been developed to specifically address interpersonal difficulties. However, cognitive changes on a relational level occurring during CBT might play an important role in the treatment of depression. Interpersonal clarification refers to the process of better understanding the nature of one's interpersonal patterns during therapy. The aim of this study is to analyze the effects of interpersonal clarification in CBT and how they are moderated by the therapeutic alliance.

METHODS

A sample of 621 patients diagnosed with depression were treated with CBT by 126 therapists in a university outpatient clinic. Patients completed measures of interpersonal problems and depression severity at baseline, measures of symptomatic evolution before each session and process measures (assessing interpersonal clarification and alliance) after each session. Multilevel models separating between-patient (BP) and within-patient (WP) effects of interpersonal clarification, and including BP and WP alliance effects as covariates and moderators of the interpersonal clarification effects were conducted.

RESULTS

Analyses showed both significant BP and WP effects interpersonal clarification, even when adjusting for alliance effects. Furthermore, significant interactive effects were found between outcome of WP interpersonal clarification with both BP alliance and WP alliance.

LIMITATIONS

Interpersonal clarification was measured with one single-item and adherence to CBT was not explicitly measured.

CONCLUSIONS

The results present preliminary evidence for considering interpersonal clarification a meaningful change process in CBT for depression, especially in the context of a stronger therapeutic alliance.

Why do patients go off track? Examining potential influencing factors for being at risk of psychotherapy treatment failure

BACKGROUND

Routine outcome monitoring can support clinicians to detect patients who deteriorate [not-on-track (NOT)] early in psychotherapy. Implemented Clinical Support Tools can direct clinicians' attention towards potential obstacles to a positive treatment outcome and provide suggestions for suitable interventions. However, few studies have compared NOT patients to patients showing expected progress [on-track (OT)] regarding such obstacles. This study aimed to identify domains that have predictive value for NOT trajectories and to compare OT and NOT patients regarding these domains and the items of the underlying scales.

METHODS

During treatment, 413 outpatients filled in the Hopkins-Symptom-Checklist-11 (depressive and anxious symptom distress) before every therapy session as a routine outcome measure. Further, the Assessment for Signal Clients, Affective Style Questionnaire, and Outcome Questionnaire-30 were applied every fifth session. These questionnaires measure the following domains, which were investigated as potential obstacles to treatment success: risk/suicidality, therapeutic alliance, motivation, social support and life events, as well as emotion regulation. Two groups (OT and NOT patients) were formed by defining a cut-off (failure boundary) as the 90% confidence interval (upper bound) of the respective patients' expected recovery curves. In order to differentiate group membership based on the respective problem areas, multilevel logistic regression analyses were performed. Further, OT and NOT patients were compared with regard to the domains' and items' cut-offs by performing Pearson chi-square tests and independent samples t-tests.

RESULTS

The life events and motivation scale as well as the risk/suicidality scale proved to be significant predictors of being not-on-track. NOT patients also crossed the cut-off significantly more often on the domains risk/suicidality, social support, and life events. For both OT and NOT patients, the emotion regulation domain's cut-off was most commonly exceeded.

CONCLUSION

Life events, motivation, and risk/suicidality seem to be directly linked to treatment failure and should be further investigated for the use in clinical support tools.

Introducing a Measurement Feedback System for Youth Mental Health: Predictors and Impact of Implementation in a Community Agency

Measurement feedback systems (MFSs) that routinely collect and report client progress to mental health therapists have demonstrated beneficial impact on outcomes in numerous studies, with evidence that there is a dose-response relationship related to the implementation of the MFS. The current study examined the impact of MFS implementation (Implementation Index) on youth symptom outcomes separately by caregiver and youth self-report. Additionally, we tested the extent to which Implementation Index rates varied by individual therapists and clients, and whether therapist and client characteristics predicted MFS implementation. Methods: Administrative data (client charts, youth- and caregiver-reported Youth Outcome Questionnaires) for 229 youth (52.83% Latinx, 42.79% girls, M age = 10.33) treated during a 1-year period at a community mental health organization in Central Texas were analyzed using multi-level modeling. Caregiver-reported symptoms decreased faster for those with a higher MFS Implementation Index. Between-group differences among therapists accounted for a significant proportion of variance in the Implementation Index for caregiver report, whereas client differences accounted for most of the variance in the Implementation Index for youth self-report. Therapist trainee status predicted a significant increase in the Implementation Index for caregiver-report data. Youth symptom improvement as reported by caregivers varied with the extent of MFS implementation fidelity, and MFS implementation fidelity was higher for clients treated by trainees relative to staff therapists for caregiver report of symptoms.

Using technology to understand how therapist variables are associated with clinical outcomes in IAPT

The UK’s Improving Access to Psychological Therapy Programme (IAPT) has improved transparency of primary mental health care in relation to the mandatory reporting of clinical outcomes. However, the data reveal a significant variance in outcomes. These findings have led to a growing body of research investigating to what extent therapist variables account for the difference in clinical outcomes. Previous studies have not had access to sufficient recordings or transcripts of therapy sessions in order to fully address this question. The purpose of this study was to use therapy transcripts derived from internet enabled cognitive behavioural therapy (CBT) treatment sessions in order to investigate whether and how therapist variables are associated with clinical outcome. A hierarchical log-linear analysis examined the relationship between therapist/patient variables and clinical outcome. Therapist fidelity to the CBT model and associated adherence to an evidence-based protocol were significantly related to clinical outcome. A graphical representation of the statistical model suggests that patient recovery is directly linked with fidelity and indirectly with adherence, after adjusting for patient attributes of age, gender and clinical presentation. Corroborating previous research, therapist competence and adherence to an evidence-based treatment protocol appear to be important in improving outcomes. These findings have implications for the continuing professional development of qualified therapists, potentially reinforcing the importance of reducing therapist drift.

Key learning aims

(1)

Clinical dilemmas of routine outcome monitoring and clinical feedback: A qualitative study of patient experiences

ABSTRACT Purpose: Routine outcome monitoring (ROM) and clinical feedback systems (CFS) are becoming prevalent in mental health services, but there are several challenges to successful implementation. ROM/CFS seem to be helpful for some patients, but not for others. To investigate this, we explored patients' experiences with ROM/CFS as an interpersonal and psychotherapeutic process, in naturalistic settings. Method: We used video-assisted interpersonal process recall interviews to investigate the experiences of 12 patients using ROM/CFS in a Norwegian mental health outpatient clinic. Data were analyzed through systematic text condensation. Results: Our analysis resulted in three pairs of experiences with ROM/CFS: (1) Explicit vs. implicit use of information, (2) Directing focus towards or away from therapeutic topics, and (3) Giving and receiving feedback. These experiences could be helpful or hindering, depending on participants' needs and preferences. All participants needed to know that the CFS was used in a meaningful way. If not, it could be detrimental to the therapeutic process. Conclusion: In order to be helpful for patients, ROM/CFS should be used in a way that is flexible, meaningful to patients, and sensitive to individual needs and preferences. Future research should further explore this how-to aspect of ROM/CFS with different CFS and populations.

Feedback in Couple and Family Therapy: A Randomized Clinical Trial

Routine Outcome Monitoring (ROM) is recommended as a psychotherapy procedure to serve as clinical feedback in order to improve client treatment outcomes. ROM can work as a warning signal to the therapist if the client shows signs of no change or deterioration. This study has investigated whether any difference in outcome could be detected between those clients in couple and family therapy who used the Systemic Therapy Inventory of Change (STIC) feedback system (ROM condition) versus those who were offered treatment without the use of STIC ("treatment as usual" or TAU condition). A sample of 328 adults seeking couple and family therapy in Norway was randomly assigned to ROM versus TAU conditions. Outcome measures were The Outcome Questionnaire-45 and The Revised Dyadic Adjustment Scale. The results demonstrated no significant differences in outcomes between the ROM and TAU. Possible explanations of this result related to design and implementation issues are discussed.

Trajectories of change of youth depressive symptoms in routine care: shape, predictors, and service-use implications

Depression is one of the main reasons for youth accessing mental health services, yet we know little about how symptoms change once youth are in routine care. This study used multilevel modeling to examine the average trajectory of change and the factors associated with change in depressive symptoms in a large sample of youth seen in routine mental health care services in England. Participants were 2336 youth aged 8-18 (mean age 14.52; 77% females; 88% white ethnic background) who tracked depressive symptoms over a period of up to 32 weeks while in contact with mental health services. Explanatory variables were age, gender, whether the case was closed, total length of contact with services, and baseline severity in depression scores. Faster rates of improvement were found in older adolescents, males, those with shorter time in contact with services, closed cases, and those with more severe symptoms at baseline. This study demonstrates that when youth self-report their depressive symptoms during psychotherapy, symptoms decrease in a linear trajectory. Attention should be paid to younger people, females, and those with lower than average baseline scores, as their symptoms decrease at a slower pace compared to others.

Perspektiven einer evidenzbasierten und personalisierten Psychotherapie

Zusammenfassung. Theoretischer Hintergrund: Der Diskurs um eine evidenz-basierte und personalisierte (bzw. „Precision“) Medizin sowie zur Umsetzung von Evaluation und Qualitätssicherung hat in den letzten Jahren auch Einfluss auf die Psychotherapieforschung genommen. Dies gilt in Bezug auf die patientenspezifische Auswahl von Behandlungen (u. a. personalisierte Vorhersagen) als auch für die dynamische Anpassung von Interventionen im Therapieverlauf (adaptive Indikation, Feedback, Problemlösetools). Fragestellung und Methode: Im Bereich der differentiellen Indikation sind mittlerweile unterschiedliche Algorithmen („machine learning“) und Netzwerkmodelle zur Vorhersage erprobt worden. Für eine empirisch gestützte adaptive Indikation bilden insbesondere die Studien zum psychometrischen Feedback sowie die Entwicklung von Problemlösetools für Risikopatient_innen die Grundlage. Ergebnisse: Diese Grundlagenforschung war die Basis für die Entwicklung eines Entscheidungssystems (Trierer Therapie Navigator, TTN) zur Vorhersage der optimalen Behandlungsstrategie und des Abbruchrisikos. Darüber hinaus enthält der TTN ein adaptives Modellierungselement des Behandlungsverlaufs. Es können damit Risikopatienten für einen Behandlungsmisserfolg identifiziert und Behandlungsoptimierungen über Problemlösetools unterstützt werden. Schlussfolgerungen: In vorliegender Arbeit werden zentrale neue Ansätze einer evidenz-basierten und personalisierten Psychotherapie zusammenfassend dargestellt sowie die Anwendung in der klinischen Praxis diskutiert.

Imagery-based treatment for test anxiety: A multiple-baseline open trial

BACKGROUND

Many students are affected by test anxiety (TA), which involves considerable distress and can lead to reduced performance. Despite its prevalence, TA has been under-studied; specifically, few effective and brief interventions have been identified to date. In our work, we explore the adaptation of imagery, an emotion-focused treatment technique used widely in the treatment of other anxiety disorders, for addressing TA.

METHODS

In a two-site concurrent multiple-baseline pilot study (n = 31), we examined the effectiveness of a targeted six-session protocol developed for the treatment of TA, which integrates traditional cognitive behavioral techniques with imagery work.

RESULTS

The protocol was well-accepted by clients. We found that students' test anxiety levels did not drop between the recruitment and pre-intervention assessments, but did drop significantly from recruitment or baseline to the delayed follow-up (Cohen's d = 0.75 and 0.84, respectively). We also found evidence for session-level processes tying the quality of the imagery work with session efficacy.

LIMITATIONS

The study involved a relatively small sample size, leading to weaker power to detect treatment effects. Moreover, some clients did not have any exams scheduled before the delayed follow-up assessment. Due to the scheduling intensity of the sessions, some clients had insufficient time to implement or practice skills and to complete tasks discussed in the sessions.

CONCLUSIONS

This study provides preliminary evidence for the utility of integrating imagery work with traditional cognitive-behavioral techniques for treating test anxiety.

The effect of patients' feedback on treatment outcome in a child and adolescent psychiatric sample: a randomized controlled trial

The systematic use of feedback from patients on treatment progress and treatment satisfaction is a promising method to increase treatment effectiveness. The extent to which this also applies to the treatment of children with severe psychiatric problems is not clear. We conducted a Randomized Controlled Trial (RCT) to study the effect of adding Feedback Informed Treatment (FIT) to care as usual in a child psychiatric sample. Quality of Life (QoL) was used as the primary outcome measure and symptom severity as the second. Fifty-one therapists from eight Autism Care Teams in a multi-center facility for Child and Adolescent Psychiatry (Karakter) participated and were cluster randomized to the FIT condition (n = 4 teams) or the Care as Usual (CAU) condition (n = 4 teams). Children aged 6-18 years, mainly with an Autism Spectrum Disorder (ASD) and treated in one of the Autism Care Teams were allocated to the FIT condition (n = 86) or the CAU condition (n = 80). Results indicated that adding FIT leads to an increased QoL [F (2,165) = 3.16, p = 0.045]. No additional effects were observed for symptom severity decrease [F (2,158) = 0.19, p = 0.825]. No interaction with time was found for QoL nor symptom severity. Adding FIT in a child psychiatric setting may increase QoL, but does not appear to decrease symptom severity as compared with CAU. It is suggested that FIT positively changes parents' expectations. Results should be replicated in other child psychiatric samples and with an extended theoretical model.

Mindfulness and progressive muscle relaxation as standardized session-introduction in individual therapy: A randomized controlled trial

OBJECTIVE

There is scarce research on the effects of mindfulness in individual therapy. As many practitioners integrate mindfulness exercises into individual therapy, empirical evidence is of high clinical relevance.

METHOD

We investigated the effects of a session-introducing intervention with mindfulness elements (SIIME) in a randomized, controlled design. The effects of SIIME on therapeutic alliance and symptomatic outcome were compared with progressive muscle relaxation (PMR) and treatment-as-usual (TAU) control conditions. The sample comprised 162 patients with anxiety and depression.

RESULTS

Multilevel modeling revealed a significant symptom reduction and significant increase of alliance over the course of therapy. There were no significant time-condition interactions on outcome and alliance, indicating the comparable efficiency of all three treatment conditions.

CONCLUSIONS

We found no advantage of SIIME versus PMR and TAU. Add-on mindfulness might not improve individual therapy related to alliance and outcome.

What happens when the therapist leaves? The impact of therapy transfer on the therapeutic alliance and symptoms

BACKGROUND

The therapeutic alliance is an important factor in psychotherapy, affecting both therapy processes and outcome. Therapy transfers may impair the quality of the therapeutic alliance and increase symptom severity. The aim of this study is to investigate the impact of patient transfers in cognitive behavioural therapy on alliance and symptoms in the sessions after the transfer.

METHOD

Patient- and therapist-rated therapeutic alliance and patient-reported symptom severity were measured session-to-session. Differences in the levels of alliance and symptom severity before (i.e., with the original therapist) and after (i.e., with the new therapist) the transfer session were analysed. The development of alliance and symptom severity was explored using multilevel growth models.

RESULTS

A significant drop in the alliance was found after the transfer, whereas no differences were found with regard to symptom severity. After an average of 2.93 sessions, the therapeutic alliance as rated by patients reached pretransfer levels, whereas it took an average of 5.05 sessions for therapist-rated alliance levels to be at a similar level as before the transfer. Inter-individual differences were found with regard to the development of the therapeutic alliance over time.

CONCLUSIONS

Therapy transfers have no long lasting negative effects on either symptom impairment or the therapeutic alliance.

The therapeutic alliance in different mental disorders: A comparison of patients with depression, somatoform, and eating disorders

OBJECTIVE

The therapeutic alliance is intensively investigated in psychotherapy research. However, there is scarce research on the role of the specific diagnosis of the patient in the formation of the therapeutic alliance. Hence, the aim of this study was to address this research gap by comparing the alliance in different mental disorders.

DESIGN

Our sample comprised 348 patients (mean age = 40 years; 68% female; 133 patients with depression, 122 patients with somatoform disorders, and 93 patients with eating disorders).

METHODS

Patients completed the Working Alliance Inventory and measures of therapeutic outcome in early, middle, and late stages of inpatient psychotherapy. We applied multivariate multilevel models to address the nested data structure.

RESULTS

All three disorder groups experienced positive alliances that increased across the course of therapy and showed similar alliance-outcome relations that were of comparable strengths as in current meta-analyses. However, we found perspective incongruence of alliance ratings from patient and therapist in the three disorder groups.

CONCLUSIONS

Our results generally indicate that the working alliance is of importance irrespective of the specific mental disorder. Perspective incongruence feedback of working alliance experiences could help to strengthen coordination between patient and therapist and thereby improve the therapeutic process. Further implications of these findings are discussed.

PRACTITIONER POINTS

We found no differences in the strengths of alliance ratings and alliance-outcome associations in depressive, somatoform, and eating disorder patients. This indicates that the working alliance is of general clinical importance irrespective of the disorder group and should be a central target in all therapies. We found perspective incongruence in alliance ratings between patient and therapist in all three disorder groups. Perspective incongruence feedback of working alliance experiences could help to strengthen coordination between patient and therapist and thereby improve the therapeutic process.

Randomized controlled trial to evaluate the effects of personalized prediction and adaptation tools on treatment outcome in outpatient psychotherapy: study protocol

BACKGROUND

Psychotherapy is successful for the majority of patients, but not for every patient. Hence, further knowledge is needed on how treatments should be adapted for those who do not profit or deteriorate. In the last years prediction tools as well as feedback interventions were part of a trend to more personalized approaches in psychotherapy. Research on psychometric prediction and feedback into ongoing treatment has the potential to enhance treatment outcomes, especially for patients with an increased risk of treatment failure or drop-out.

METHODS/DESIGN

The research project investigates in a randomized controlled trial the effectiveness as well as moderating and mediating factors of psychometric feedback to therapists. In the intended study a total of 423 patients, who applied for a cognitive-behavioral therapy at the psychotherapy clinic of the University Trier and suffer from a depressive and/or an anxiety disorder (SCID interviews), will be included. The patients will be randomly assigned either to one therapist as well as to one of two intervention groups (CG, IG2). An additional intervention group (IG1) will be generated from an existing archival data set via propensity score matching. Patients of the control group (CG; n = 85) will be monitored concerning psychological impairment but therapists will not be provided with any feedback about the patients assessments. In both intervention groups (IG1: n = 169; IG2: n = 169) the therapists are provided with feedback about the patients self-evaluation in a computerized feedback portal. Therapists of the IG2 will additionally be provided with clinical support tools, which will be developed in this project, on the basis of existing systems. Therapists will also be provided with a personalized treatment recommendation based on similar patients (Nearest Neighbors) at the beginning of treatment. Besides the general effectiveness of feedback and the clinical support tools for negatively developing patients, further mediating and moderating variables on this feedback effect should be examined: treatment length, frequency of feedback use, therapist effects, therapist's experience, attitude towards feedback as well as congruence of therapist's and patient's evaluation concerning the progress. Additional procedures will be implemented to assess treatment adherence as well as the reliability of diagnosis and to include it into the analyses.

DISCUSSION

The current trial tests a comprehensive feedback system which combines precision mental health predictions with routine outcome monitoring and feedback tools in routine outpatient psychotherapy. It also adds to previous feedback research a stricter design by investigating another repeated measurement CG as well as a stricter control of treatment integrity. It also includes a structured clinical interview (SCID) and controls for comorbidity (within depression and anxiety). This study also investigates moderators (attitudes towards, use of the feedback system, diagnoses) and mediators (therapists' awareness of negative change and treatment length) in one study.

TRIAL REGISTRATION

Current Controlled Trials NCT03107845 . Registered 30 March 2017.

Service-level variation, patient-level factors, and treatment outcome in those seen by child mental health services

Service comparison is a policy priority but is not without controversy. This paper aims to investigate the amount of service-level variation in outcomes in child mental health, whether it differed when examining outcomes unadjusted vs. adjusted for expected change over time, and which patient-level characteristics were associated with the difference observed between services. Multilevel regressions were used on N = 3256 young people (53% male, mean age 11.33 years) from 13 child mental health services. Outcome was measured using the parent-reported Strengths and Difficulties Questionnaire. The results showed there was 4-5% service-level variation in outcomes. Findings were broadly consistent across unadjusted vs. adjusted outcomes. Young people with autism or infrequent case characteristics (e.g., substance misuse) had greater risk of poor outcomes. Comparison of services with high proportions of young people with autism or infrequent case characteristics requiring specialist input needs particular caution as these young people may be at greater risk of poor outcomes.

Nutzung von psychometrischem Feedback als empirische Unterstützung des Supervisionsprozesses bei Ausbildungstherapien

Zusammenfassung. Theoretischer Hintergrund: Trotz der gut belegten generellen Wirksamkeit von Psychotherapie wird die Zahl der Patienten, die nicht auf eine Therapie ansprechen oder sich sogar im Verlauf verschlechtern, auf etwa ein Drittel geschätzt. Da Therapeuten im Vergleich zu empirischen Algorithmen weniger gut in der Lage sind negative Entwicklungen zu entdecken oder zu prognostizieren, brauchen sie zusätzliche Unterstützung in Form von kontinuierlichen Rückmeldungen über den Fortschritt ihrer Patienten. Solche Feedbackinterventionen können als empirisch basierte Unterstützung des Supervisions- oder Intervisionsprozesses verstanden werden. Diese Interventionen haben ihre Wirksamkeit zur Reduktion therapeutischer Misserfolge in zahlreichen Einzelstudien und Meta-Analysen wiederholt zeigen können. Fragestellung: Wie nutzen Therapeuten ein solches Feedback in ihrer praktischen Arbeit und in welchem Ausmaß spielen Therapeutenfaktoren dabei eine Rolle? Methode: In der vorliegenden Studie wurden 72 Therapeuten für 648 ihrer Patienten unmittelbar nach der Therapie dazu befragt, wie sie das psychometrische Feedback genutzt haben. Therapeutenunterschiede wurden mittels Mehrebenenmodellen ermittelt. Ergebnisse: Es konnte gezeigt werden, dass Therapeuten für einen Großteil ihrer Patienten das Feedback verwendeten. In etwa einem Drittel der Fälle gab es den Therapeuten den Anstoß zusätzliche Hilfen (z. B. Supervision/Intervision) zu beanspruchen. Ähnlich einer personengestützten Supervision wurde für über die Hälfte der Patienten das Feedback genutzt um therapeutische Interventionen anzupassen. Ob und in welcher Form es genutzt wurde hing jedoch stark von dem Therapeuten ab. Je nach Verwendungsart konnten Therapeutenunterschiede zwischen 27 % und 52 % der Feedbacknutzung erklären. Die Wahrscheinlichkeit, dass Therapeuten die Nutzungsmöglichkeiten des Feedbacks anwendeten war größer, wenn diese angaben im Durchschnitt zufriedener mit den grafischen Rückmeldungen zu sein. Schlussfolgerungen: Diese Ergebnisse machen deutlich, dass es zu einem großen Teil nicht nur vom Therapieverlauf oder den Charakteristika der jeweiligen Patienten abhängt wie Therapeuten das Feedback nutzen, sondern auch von Variablen, die den Therapeuten betreffen.

When is Sessional Monitoring More Likely in Child and Adolescent Mental Health Services?

Sessional monitoring of patient progress or experience of therapy is an evidence-based intervention recommended by healthcare systems internationally. It is being rolled out across child and adolescent mental health services (CAMHS) in England to inform clinical practice and service evaluation. We explored whether patient demographic and case characteristics were associated with the likelihood of using sessional monitoring. Multilevel regressions were conducted on N = 2609 youths from a routinely collected dataset from 10 CAMHS. Girls (odds ratio, OR 1.26), older youths (OR 1.10), White youths (OR 1.35), and youths presenting with mood (OR 1.46) or anxiety problems (OR 1.59) were more likely to have sessional monitoring. In contrast, youths under state care (OR 0.20) or in need of social service input (OR 0.39) were less likely to have sessional monitoring. Findings of the present research may suggest that sessional monitoring is more likely with common problems such as mood and anxiety problems but less likely with more complex cases, such as those involving youths under state care or those in need of social service input.

Daily affect dynamics predict early response in CBT: Feasibility and predictive validity of EMA for outpatient psychotherapy

BACKGROUND

Previous studies have shown that individual differences in affect dynamics during depressed patients' everyday lives allow the prediction of treatment outcome and of symptom reoccurrence in remitted patients. In this study, we analyze whether understanding patients' affective states and their fluctuation patterns helps predict early treatment response (until session 5).

METHODS

Ecological Momentary Assessment (EMA) strategies allow in-depth analyses of real-time affective states and of their dynamics. Repeated assessments were made four times a day during a two-week period to capture real-life affective states (positive affect, PA and negative affect, NA) and dynamics (fluctuations in NA and PA) before the start of outpatient treatment of 39 patients. Due to the nested structure of the data, hierarchical linear models were conducted.

RESULTS

PA/NA ratios, as well as fluctuations in NA predicted early treatment response, even when adjusting for initial impairment. In contrast, mean levels of NA or PA, as well as fluctuations in PA did not predict treatment response.

LIMITATIONS

The time between the EMA assessment and treatment onset varied between patients. However, this variation was not associated with early change.

CONCLUSIONS

The results suggest that pre-treatment affect dynamics could provide valuable information for predicting treatment response independent of initial impairment levels. Better predictions of early treatment response help to improve treatment choices early in the treatment progress.

Therapist Effects on and Predictors of Non-Consensual Dropout in Psychotherapy

BACKGROUND

Whereas therapist effects on outcome have been a research topic for several years, the influence of therapists on premature treatment termination (dropout) has hardly been investigated. Since dropout is common during psychological treatment, and its occurrence has important implications for both the individual patient and the healthcare system, it is important to identify the factors associated with it.

METHOD

Participants included 707 patients in outpatient psychotherapy treated by 66 therapists. Multilevel logistic regression models for dichotomous data were used to estimate the impact of therapists on patient dropout. Additionally, sociodemographic variables, symptoms, personality style and treatment expectations were investigated as potential predictors.

RESULTS

It was found that 5.7% of variance in dropout could be attributed to therapists. The therapist's effect remained significant after controlling for patient's initial impairment. Furthermore, initial impairment was a predictor of premature termination. Other significant predictors of dropout on a patient level were male sex, lower education status, more histrionic and less compulsive personality style and negative treatment expectations.

CONCLUSIONS

The findings indicate that differences between therapists influence the likelihood of dropout in outpatient psychotherapy. Further research should focus on variables, which have the potential to explain these inter-individual differences between therapists (e.g., therapist's experience or self-efficacy). Copyright © 2016 John Wiley & Sons, Ltd.

KEY PRACTITIONER MESSAGES

There are substantial differences between therapists concerning their average dropout rates. At the patient level, higher initial impairment, male sex, lower education, less compulsive personality style, more histrionic personality style and low treatment expectations seem to be risk factors of non-consensual treatment termination. Psychometric feedback during the course of treatment should be used to identify patients who are at risk for dropout.

Patient-focused and feedback research in psychotherapy: Where are we and where do we want to go?

In the last 15 years feedback interventions have had a significant impact on the field of psychotherapy research and have demonstrated their potential to enhance treatment outcomes, especially for patients with an increased risk of treatment failure. This article serves as an introduction to the special issue on "Patient-focused and feedback research in psychotherapy: Where are we and where do we want to go?" Current investigations on feedback research are concerned with potential moderators and mediators of these effects, as well as the design and the implementation of feedback into routine care. This introduction summarizes the current state of feedback research and provides an overview of the three main research topics in this issue: (1) How to implement feedback systems into routine practice and how do therapist and patient attitudes influence its effects?, (2) How to design feedback reports and decision support tools?, and (3) What are the reasons for patients to become at risk of treatment failure and how should therapists intervene with these patients? We believe that the studies included in this special issue reflect the current state of feedback research and provide promising pathways for future endeavors that will enhance our understanding of feedback effects.