Enhancing the effect of psychotherapy through systematic client feedback in outpatient mental healthcare: A cluster randomized trial

Process and Implementation Elements of Measurement Feedback Systems: A Systematic Review

Measurement feedback systems (MFS) can help guide treatment and improve clinical outcomes. Studies of MFS are heterogeneous both in execution and results, and the effects of MFS seem restricted by limited attention to process and implementation elements and by limited adoption by health professionals. The current systematic review mapped the use of process and implementation elements in MFS studies. An overview of therapists' use of and attitudes toward MFS is provided. Three-level meta-analyses were used to test theoretically informed process and implementation elements as moderators of the effects of MFS. Hypotheses and general propositions from Clinical Performance Feedback Intervention Theory (CP-FIT) were used to organize the elements of the studies and were used as moderator variables. Previous studies on MFS interventions have had a limited focus on implementation efforts and process elements that may increase the effects of MFS and their use among therapists. Efforts have sparsely been made to reduce barriers to MFS use, and several studies have reported limited engagement with MFS among therapists. Therapists' attitudes toward MFS, feedback, or standardized measures were heterogeneously reported, making data synthesis challenging. Identified process and implementation elements were not significantly associated with effect sizes in the studies and the results did not support the propositions of CP-FIT. The lack of statistically significant associations may be due to limited reporting of details about process and implementation aspects. More research designed to test hypotheses regarding process and implementation elements is needed to improve the use and effects of MFS. Future studies should aspire to report findings in a manner that allows for an understanding of the implementation process and therapists' adoption of these systems.

Using Progress Feedback to Enhance Treatment Outcomes: A Narrative Review

We face increasing demand for greater access to effective routine mental health services, including telehealth. However, treatment outcomes in routine clinical practice are only about half the size of those reported in controlled trials. Progress feedback, defined as the ongoing monitoring of patients' treatment response with standardized measures, is an evidence-based practice that continues to be under-utilized in routine care. The aim of the current review is to provide a summary of the current evidence base for the use of progress feedback, its mechanisms of action and considerations for successful implementation. We reviewed ten available meta-analyses, which report small to medium overall effect sizes. The results suggest that adding feedback to a wide range of psychological and psychiatric interventions (ranging from primary care to hospitalization and crisis care) tends to enhance the effectiveness of these interventions. The strongest evidence is for patients with common mental health problems compared to those with very severe disorders. Effect sizes for not-on-track cases, a subgroup of cases that are not progressing well, are found to be somewhat stronger, especially when clinical support tools are added to the feedback. Systematic reviews and recent studies suggest potential mechanisms of action for progress feedback include focusing the clinician's attention, altering clinician expectations, providing new information, and enhancing patient-centered communication. Promising approaches to strengthen progress feedback interventions include advanced systems with signaling technology, clinical problem-solving tools, and a broader spectrum of outcome and progress measures. An overview of methodological and implementation challenges is provided, as well as suggestions for addressing these issues in future studies. We conclude that while feedback has modest effects, it is a small and affordable intervention that can potentially improve outcomes in psychological interventions. Further research into mechanisms of action and effective implementation strategies is needed.

Motives of Therapists for Using Routine Outcome Monitoring (ROM) and How it is Used by Them in Clinical Practice: Two Qualitative Studies

Despite its demonstrated value, many mental health institutions struggle to implement progress feedback effectively. There is also insufficient information about how therapists utilize progress feedback. To gain more insight, two qualitative studies were conducted. The first study compared the attitudes and motives of therapists who used and those who did not use progress feedback. The second study examined how psychologists incorporated progress feedback into their practice. In total, 23 therapists were interviewed, and the data were analyzed using thematic analysis. The first study found that almost all the therapists had a positive attitude about progress feedback. Those who did not use it indicated reasons such as a heavy workload and patient-related factors, and they also lacked sufficient information about the potential benefits of progress feedback. The second study revealed four major ways in which the therapists utilized feedback, namely: supporting actions to discuss progress feedback, discussing progress feedback with patients, modifications in the ongoing treatment, and peer consultation. However, discussions during peer consultations about using progress feedback for patients who were not benefitting from treatment and how the treatment might be adjusted accordingly were often lacking. In conclusion, it is crucial that training and education be provided to therapists on how to use progress feedback effectively. Having regular discussions about progress feedback during peer consultations could also facilitate its integration into clinical practice.

Staff and client preferences for the design and delivery of an outcomes monitoring system in a mental health service

BACKGROUND

Outcome monitoring can support the delivery of quality service that meets the needs of clients, clinicians and services. However, few studies have examined client or staff perspectives on the design and implementation of outcomes monitoring within a service. Implementation of outcomes monitoring requires understanding the preferences and expectations of relevant stakeholders.

OBJECTIVE

Informed by the Consolidated Framework for Implementation Research, this study aimed to investigate the preferences and priorities of relevant staff, clients and family members to inform the design and implementation of an effective outcomes monitoring system in the context of a mental health service for military veterans and their families.

METHOD

Twenty-nine staff participated across five online focus groups, including clinical staff, peer workers, policy staff and supervisors. Ten clients participated in online or telephone semi-structured interviews. Thematic analysis was used to develop themes from the data.

RESULTS

Clients and staff agreed that outcomes monitoring should cover more than symptoms, particularly by incorporating functional outcomes. Assessing mental health over time was considered a valuable tool for supporting treatment processes and providing actionable information. Challenges identified by clients and staff included the need for efficient processes, ensuring measures are relevant and acceptable, and maintaining client privacy. The ability to personalise data collection and have a streamlined, responsive system were key attributes of a quality outcomes monitoring framework.

CONCLUSIONS

Findings suggest that an effective outcome monitoring framework should be client-led, tailored to the individual's needs, and provide feedback on progress. Outcomes monitoring should also be efficient, accessible and allow for safe information sharing.

Effectiveness of a Novel Web-Based Intervention to Enhance Therapeutic Relationships and Treatment Outcomes in Adult Individual Psychotherapy: Randomized Controlled Trial and Analysis of Predictors of Dropouts

BACKGROUND

Routine process and outcome monitoring interventions added to psychotherapy are known to improve treatment outcomes, although they vary in format and effectiveness.

OBJECTIVE

This study aimed to evaluate whether a therapist-independent, internet-based routine process monitoring and feedback system could significantly reduce psychological distress and enhance the quality of the therapeutic relationship compared with a treatment-as-usual control group among individuals already engaged in individual psychotherapy.

METHODS

We randomized 475 participants into either the intervention group, which received access to an internet-based routine process monitoring and feedback system in addition to psychotherapy, or the control group, which received only psychotherapy. The trial lasted for 10 weeks. Follow-up assessments at 5 weeks and 10 weeks used the Clinical Outcomes in Routine Evaluation-Outcome Measure as the primary outcome, with the Working Alliance Inventory-Short Revised and the Real Relationship Inventory-Client form as secondary outcomes.

RESULTS

Per-protocol analyses (n=166) showed that psychological distress decreased in both groups, but there was no significant advantage for the intervention group. The intervention group experienced a decline in the genuineness dimension score of the real relationship, with an effect size of d=-0.27, compared with d=0.01 in the control group. In the intervention group (but not in the control group), dropouts showed significantly lower real relationship levels (P=.002), working alliance quality (P=.051), and emotional disclosure (P=.01) compared with those who completed the study. Additionally, logistic regression revealed distinct predictors of dropout within the control group and intervention group.

CONCLUSIONS

The findings do not provide conclusive evidence for the efficacy of the new internet-based intervention in enhancing self-monitoring and prompting reflection on patients' emotional responses to their therapists. However, the intervention appears to influence patients' perceptions of the genuineness dimension in the therapeutic relationship, warranting further investigation. We hypothesize that this alteration in the genuineness dimension could be attributed to the intervention facilitating a more realistic and accurate perception of the therapeutic relationship among participants.

TRIAL REGISTRATION

ClinicalTrials.gov NCT06038747; https://clinicaltrials.gov/study/NCT06038747.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/55369.

Evaluation of completely online psychotherapy with app-support versus therapy as usual for clients with depression or anxiety disorder: A retrospective matched cohort study investigating the effectiveness, efficiency, client satisfaction, and costs

INTRODUCTION

Depressive and anxiety disorders are common mental disorders ranking among the leading causes of global disease burden. Not all clients currently benefit from therapy and clients are looking for modern ways of therapy. Online psychotherapy is a promising option for better meeting clients' needs. Recently, a new psychotherapy concept has emerged that combines videoconferencing sessions with support through a mobile application. The latter allows for ecological momentary assessments and interventions, facilitates communication between patients and therapists in between sessions through chat, and allows for incorporating feedback-informed treatment principles.

MATERIAL AND METHODS

The study was a retrospective observational matched cohort study, comparing online psychotherapy with Therapy As Usual (TAU) for clients with depressive or anxiety disorders. Data were obtained via questionnaires, which are part of standard clinical care. Primary outcomes included general mental functioning, and symptoms of depression and anxiety. Secondary outcomes were efficiency, client satisfaction, and therapy costs. Primary endpoints were analyzed using linear mixed models analysis, with an interaction term between time and group. Secondary outcomes were analyzed using linear regression.

RESULTS

Larger improvements were observed in the online compared to the TAU group for general mental functioning and depressive disorder (i.e., General mental functioning: B = -8.50, 95 CI: -15.01 - -1.97, p = 0.011; Depressive disorder: B = - 3.66, 95 % CI: -5.79 - -1.54p < 0.01). No significant differences in change over time between the two groups were observed for anxiety disorder (B = -3.64, 95 % CI: (-13.10 - 5.82) p = 0.447). The total number of sessions was significantly higher in the online psychotherapy group than in TAU (B = 3.71, p < 0.01), although clients were matched on treatment time in weeks. Treatment session duration in minutes was comparable across the groups.

DISCUSSION

Online psychotherapy with app support showed to be a promising alternative to TAU for depressive and anxiety disorders. More research is needed to evaluate the effectiveness, cost-effectiveness and client satisfaction of online psychotherapy compared to TAU, such as randomized controlled trials or studies multiple baseline series designs, and in-depth qualitative research.

Patient perspectives on systematic client feedback in Dutch outpatient mental healthcare, a qualitative case reports study

OBJECTIVE

The added value of systematic client feedback (SCF) to psychotherapy can be affected by patient perspectives, both in a positive and negative way, and is influenced by cultural factors as well. Current study explores patients' perspectives on use and optimization of SCF in Dutch outpatient mental healthcare. Primary aim of present study is to generate implications for daily practice and optimize SCF implementation, particularly for the Netherlands.

METHOD

Four patients suffering from mild to moderate psychological disorders were in-depth interviewed on their perspective on the use of SCF, when the Partners for Change Outcome Management System (PCOMS, high frequent), the Outcome Questionnaire (OQ-45, low frequent) and the Mental Health Continuum Short Form (MHC-SF, low frequent) was added to treatment as usual in two Dutch outpatient mental healthcare centers offering brief psychological treatment. Interview topics were (a) SCF in general; (b) type of questionnaires; (c) frequency of use; (d) effect of SCF on therapy; and (e) perceived added value of SCF. A SCF expert team analyzed the data through reflexive Thematic Analysis.

RESULTS

We identified three themes, all of which have two sub-themes: (a) Requirements to use SCF, with sub-themes (a1) Balance between effort to complete SCF and perceived validity, and (a2) Balance between used frequency and perceived validity; (b) Modifiers for test-taking attitude, with subthemes (b1) SCF as an embedded part of therapy, and (b2) Quality of Therapist-Client alignment; and (c) Effects on therapeutic process, with subthemes (c1) Focus on task and goals, and shared responsibility, and (c2) Effects on outcome and satisfaction.

CONCLUSIONS

Adding SCF to therapy can be perceived as helpful by patients in psychotherapy if two conditions are met: (1) Creating a right balance between effort and yield for SCF to be used at all; and (2) embedding SCF as an integral part of therapy, through therapist-client alignment. Throughout the progression of therapy, it might be useful to perceive SCF and therapy as communicating vessels; according to patients it is not only necessary to adjust therapy based on SCF, but also to adjust SCF based on the course of therapy.

TRIAL REGISTRATION

This trial was registered on September 30, 2015 in the Dutch Trial Register NTR5466. The Medical Ethics Committee of the University of Twente (Enschede) approved this study (registration number: K15-11, METC Twente).

The Patient-Perceived Helpfulness of Measures Scale: Development and Validation of a Scale to Assess the Helpfulness of Using Measures in Psychological Treatment

In response to the increase in Routine Outcome Monitoring and Clinical Feedback, the Patient-Perceived Helpfulness of Measures Scale (ppHMS) was developed to assess the helpfulness-as perceived by patients-of using measures in psychological treatment. Study 1: The construct of patient-perceived helpfulness of measures was explored using thematic analysis with 15 patients. Six helpful and three unhelpful themes were identified and informed item development. Study 2: 28 items were formulated and rated by experts. Ten items were taken forward for psychometric shortening in a sample of 76 patients. Confirmatory factor analysis (CFA) led to an adequately fitting six-item model with excellent internal consistency, and convergence with the Delighted-Terrible single item of product satisfaction and a single item of measure helpfulness. Study 3: In a stratified online sample of 514 U.K. psychotherapy patients, a five-item model constituted the best fit. The final ppHMS had excellent internal consistency (McDonald's ω = .90), convergent validity with psychotherapy satisfaction (r = .5; p < .001), divergence from social desirability (r = .1), and metric and scalar invariance across measures. Study 4: Analyses were replicated and confirmed in a stratified U.S. sample (n = 602). The ppHMS is a reliable and valid scale that can be used to assess and compare patients' perceptions of the helpfulness of different measures as part of their psychological treatment.

Do routine outcome monitoring measures tell the same story? Evaluating outcome trajectories and clinical outcomes for the outcome rating scale and outcome questionnaire - 45

Routine outcome monitoring (ROM) of psychotherapy improves client outcomes and reduces premature terminations. Two ROM systems with the most empirical support are the OQ System (Lambert, 2010) and the Partners for Change Outcome Management System (PCOMS; Duncan, 2012; Miller et al., 2005). We evaluated if the global distress measures, the Outcome Rating Scale (ORS; PCOMS) and the Outcome Questionnaire-45.2 (OQ-45; OQ System) show similar outcome trajectories and rates of reliable and clinically significant change when completed by the same clients.

Participants (N = 290) were clients from archived data collected in a training clinic for a psychology doctoral program in the western United States, who completed both the ORS and OQ-45 across treatment.

Bi-variate correlations between the ORS and OQ-45 were highly correlated at the first session (r = -.73), and for later sessions linear and quadratic estimates were only moderately related (rs = -.31 to -.57). Statistically significantly more clients indicated both reliable change and clinically significant change on the ORS compared to the OQ-45.

The results indicate that these measures are correlated but not interchangeable in relation to the conclusions they generate about treatment outcomes. Measurement tools should be carefully considered given the potential for different conclusions to be drawn regarding treatment progress and outcomes.

A Clinical Leadership Lens on Implementing Progress Feedback in Three Countries: Development of a Multidimensional Qualitative Coding Scheme

BACKGROUND

Progress feedback, also known as measurement-based care (MBC), is the routine collection of patient-reported measures to monitor treatment progress and inform clinical decision-making. Although a key ingredient to improving mental health care, sustained use of progress feedback is poor. Integration into everyday workflow is challenging, impacted by a complex interrelated set of factors across patient, clinician, organizational, and health system levels. This study describes the development of a qualitative coding scheme for progress feedback implementation that accounts for the dynamic nature of barriers and facilitators across multiple levels of use in mental health settings. Such a coding scheme may help promote a common language for researchers and implementers to better identify barriers that need to be addressed, as well as facilitators that could be supported in different settings and contexts.

METHODS

Clinical staff, managers, and leaders from two Dutch, three Norwegian, and four mental health organizations in the USA participated in semi-structured interviews on how intra- and extra-organizational characteristics interact to influence the use of progress feedback in clinical practice, supervision, and program improvement. Interviews were conducted in the local language, then translated to English prior to qualitative coding.

RESULTS

A team-based consensus coding approach was used to refine an a priori expert-informed and literature-based qualitative scheme to incorporate new understandings and constructs as they emerged. First, this hermeneutic approach resulted in a multi-level coding scheme with nine superordinate categories and 30 subcategories. Second-order axial coding established contextually sensitive categories for barriers and facilitators.

CONCLUSIONS

The primary outcome is an empirically derived multi-level qualitative coding scheme that can be used in progress feedback implementation research and development. It can be applied across contexts and settings, with expectations for ongoing refinement. Suggestions for future research and application in practice settings are provided. Supplementary materials include the coding scheme and a detailed playbook.

Features of Self-Organization during the Process of Mindfulness-Based Stress Reduction: A Single Case Study

Compared to the extensive evidence of the effectiveness of mindfulness-based interventions, there is only a limited understanding of their mechanisms of change. The three aims of this study are (1) to identify features of self-organization during the process (e.g., pattern transitions), (2) to obtain an impression of the effects of continuous self-assessments and feedback sessions on mindfulness-related stress reduction, and (3) to test the feasibility of high-frequency process monitoring and process feedback. Concerning aim (1), the specific hypothesis is that change will occur as a cascade of discontinuous pattern transitions emerging spontaneously in the sense of not being a reaction to external input. This single case study describes changing patterns of multiple time series that were produced by app-based daily self-assessments during and after an 8-week mindfulness-based stress reduction program. After this MBSR program, the participant (a female nurse) continued the self-assessment and the mindfulness practice for a further 10 months. The results confirm findings on the positive effects of mindfulness programs for healthcare professionals, especially on coping with work-related stress. The analysis of the time series data supports the hypothesis of self-organization as a possible mechanism of change manifesting as a cascade of phase transitions in the dynamics of a biopsychosocial system. At the end of the year, the participant reported a beneficial impact of daily monitoring and systematic feedback on the change process. The results underline the feasibility and usefulness of continuous high-frequency monitoring during and after mindfulness interventions.

Routine Outcome Monitoring (ROM) and Feedback: Research Review and Recommendations

OBJECTIVE

To provide a research review of the components and outcomes of routine outcome monitoring (ROM) and recommendations for research and therapeutic practice.

METHOD

A narrative review of the three phases of ROM - data collection, feeding back data, and adapting therapy - and an overview of patient outcomes from 11 meta-analytic studies.

RESULTS

Patients support ROM when its purpose is clear and integrated within therapy. Greater frequency of data collection is more important for shorter-term therapies, and use of graphs, greater specificity of feedback, and alerts are helpful. Overall effects on patient outcomes are statistically significant (g ≈ 0.15) and increase when clinical support tools (CSTs) are used for not-on-track cases (g ≈ 0.36-0.53). Effects are additive to standard effects of psychological therapies. Organizational, personnel, and resource issues remain the greatest obstacles to the successful adoption of ROM.

CONCLUSION

ROM offers a low-cost method for enhancing patient outcomes, on average resulting in an ≈ 8% advantage (success rate difference; SRD) over standard care. CSTs are particularly effective for not-on-track patients (SRD between ≈ 20% and 29%), but ROM does not work for all patients and successful implementation is a major challenge, along with securing appropriate cultural adaptations.

A Systematic Review and Meta-Analysis of Measurement Feedback Systems in Treatment for Common Mental Health Disorders

To investigate the effects of measurement feedback systems (MFSs) in therapy on mental health outcomes through a literature review and meta-analysis. Using a three-level modeling approach, we conducted a meta-analysis of all effect sizes from randomized controlled studies of MFSs used in the treatment of common mental health disorders. Eighty-two effect sizes were extracted from the thirty-one included studies. Analyses were performed to consider the post-treatment effects of the MFS-assisted treatment compared to treatment as usual. A separate analysis was done for the subgroup "not-on-track" patients as it is theorized that MFSs will be clinically useful because they make therapists aware of patients who fail to progress. MFSs had a significant effect on mental health outcomes (d = 0.14, 95% CI [0.082-0.206], p < .001). Further analysis found a larger effect in patients identified as less respondent to therapy, the "not-on-track" group (d = 0.29, 95% CI [0.114, 0.464], p = .003). Moderation analyses indicated that the type of outcome measurement and type of feedback system used, and whether it was used for a child and youth or adult population, influenced effect sizes. MFSs seem to have a small positive effect on treatment outcomes. The effects seem to be larger for "not-on-track" patients, the group of patients that would usually not benefit much from treatment.

Organization-level variation in therapists' attitudes toward and use of measurement-based care

PURPOSE

Despite significant interest in improving behavioral health therapists' implementation of measurement-based care (MBC)-and widespread acknowledgment of the potential importance of organization-level determinants-little is known about the extent to which therapists' use of, and attitudes toward, MBC vary across and within provider organizations or the multilevel factors that predict this variation.

METHODS

Data were collected from 177 therapists delivering psychotherapy to youth in 21 specialty outpatient clinics in the USA. Primary outcomes were use of MBC for progress monitoring and treatment modification, measured by the nationally-normed Current Assessment of Practice Evaluation-Revised. Secondary outcomes were therapist attitudes towards MBC. Linear multilevel regression models tested the association of theory-informed clinic and therapist characteristics with these outcomes.

RESULTS

Use of MBC varied significantly across clinics, with means on progress monitoring ranging from values at the 25th to 93rd percentiles and means on treatment modification ranging from the 18th to 71st percentiles. At the clinic level, the most robust predictor of both outcomes was clinic climate for evidence-based practice implementation; at the therapist level, the most robust predictors were: attitudes regarding practicality, exposure to MBC in graduate training, and prior experience with MBC. Attitudes were most consistently related to clinic climate for evidence-based practice implementation, exposure to MBC in graduate training, and prior experience with MBC.

CONCLUSIONS

There is important variation in therapists' attitudes toward and use of MBC across clinics. Implementation strategies that target clinic climate for evidence-based practice implementation, graduate training, and practicality may enhance MBC implementation in behavioral health.

Grief Impairment Scale: A biopsychosocial measure of grief-related functional impairment

Though most mourners are resilient, a minority of the bereaved experience disabling grief accompanied by clinically significant impairment in important areas of functioning. Although impairment measures exist, they have notable limitations in the context of bereavement. Therefore, the objective of this study was to develop and evaluate the Grief Impairment Scale (GIS), which was designed to identify the impact of grief on biopsychosocial functioning. The resulting 5-item scale, which was based on 363 adults who are struggling with grief, demonstrated solid reliability, factorial validity, and convergent and divergent validity with correlations with measures of impairment, prolonged grief, and psychological distress. The GIS also measures grief-related functioning equivalently across demographic groups and satisfactorily discriminates between persons with and without impairment using an optimized cut score of ≥ 9 (79% sensitivity and 74% specificity). These results provide preliminary support for the GIS as a flexible tool for clinical research and practice.

Therapy from my point of view: A case illustration of routine outcome monitoring and feedback in psychotherapeutic interventions

Lately, the use of patient-reported outcome measures (PROM) to adapt and improve ongoing psychotherapeutic treatments has become more widespread. Their main purpose is to support data-informed, collaborative treatment decisions which include the patient's point of view on their progress. In case of nonresponse or deterioration, these systems are able to warn clinicians and guide the process "back on track" in treatment. In this case illustration, the Greifswald Psychotherapy Navigator System (GPNS) detected the deterioration of 19-year-old Sarah during the first eight sessions of cognitive-behavioral therapy for social anxiety and depression. Here, the GPNS helped the therapist gain insight as to how Sarah's social anxiety affected their treatment and adjust her strategy accordingly. Using the symptom curves and progress scales of the GPNS, the therapist was able to then address her patient's struggles in detail during their sessions and with her supervisor. After adapting her therapeutic approach, the patient's deterioration could be averted while simultaneously strengthening their communication in the process. Clinical implications and the benefits of using PROM systems for evidence-based personalization of psychotherapy are presented.