The challenges of making clinical feedback in psychotherapy benefit all users: A qualitative study

What if we wait? Using synthetic waiting lists to estimate treatment effects in routine outcome data

Objective: Due to the lack of randomization, pre-post routine outcome data precludes causal conclusions. We propose the "synthetic waiting list" (SWL) control group to overcome this limitation.

Method: First, a step-by-step introduction illustrates this novel approach. Then, this approach is demonstrated using an empirical example with data from an outpatient cognitive-behavioral therapy (CBT) clinic (N = 139). We trained an ensemble machine learning model ("Super Learner") on a data set of patients waiting for treatment (N = 311) to make counterfactual predictions of symptom change during this hypothetical period.

Results: The between-group treatment effect was estimated to be d = 0.42. Of the patients who received CBT, 43.88% achieved reliable and clinically significant change, while this probability was estimated to be 14.54% in the SWL group. Counterfactual estimates suggest a clear net benefit of psychotherapy for 41% of patients. In 32%, the benefit was unclear, and 27% would have improved similarly without receiving CBT.

Conclusions: The SWL is a viable new approach that provides between-group outcome estimates similar to those reported in the literature comparing psychotherapy with high-intensity control interventions. It holds the potential to mitigate common limitations of routine outcome data analysis.

How therapists and patients need to develop a clinical feedback system after 18 months of use in a practice-research network: a qualitative study

BACKGROUND

A personalized computer-adaptive system for clinical feedback and routine outcome monitoring in mental health, Norse Feedback aims to bridge the needs for standardized and idiographic measures in ordinary practice.

METHODS

Item response theory analyses of completed treatment processes (n = 800) informed a qualitative study comprising individual in-depth interviews and focus groups with patients (n = 9) and clinicians (n = 10). The research question was: How do clinicians and patients contribute to developing a clinical feedback system in a continuous process aimed at refining its clinical usefulness?

RESULTS

We conducted thematic analyses and found five themes: 1. Added clinical needs, 2. Needs for re-organizing the clinician report, 3.Needs for differentiation of clinical content, 4. User-interface needs, and 5. Item level suggestions.

CONCLUSION

In this article, we detail resulting needs for continuous adaptation to practice, and discuss implications of the concrete experiences with the Norse action research program for the larger field of ROM/CFS implementation.

Continuous quality improvement in measure development: Lessons from building a novel clinical feedback system

Purpose

While the use of clinical feedback systems has become commonplace in psychological treatment, many of the most common instruments used for this purpose have not changed in decades. This paper describes the first four cycles of a measure development method designed to embrace continuous quality improvement.

Methods

Using techniques and philosophies developed in business management and academia—lean continuous quality improvement, action research, and practice research networks—we iterated through multiple cycles of development with the goal of creating an optimal clinical feedback system. These cycles emphasize building capacity to receive and implement feedback from a variety of stakeholders, especially patients and providers of behavioral health services, while also being responsive to quantitative findings from measure development.

Results

Iterating measure development with stakeholder feedback over the course of 5 years has resulted in a novel measurement system with 19 subscales administered via branching logic, and a supporting practice research network to sustain development.

Conclusion

In developing a new clinical feedback system, the less-frequently-discussed practical aspects of measure development require close attention. Specifically, being willing to embrace change, planning for iteration, and systematically seeking stakeholder feedback are identified as central methods for improving clinical feedback systems.

Complexity and potentials of clinical feedback in mental health: an in-depth study of patient processes

Purpose

Routine outcome monitoring (ROM) and clinical feedback systems (CFS) are becoming increasingly prevalent in mental health services. Their overall efficacy is unclear, but quantitative evidence suggests they can be useful tools for preventing treatment failure and enhancing therapeutic outcomes, especially for patients who are not progressing in therapy. The body of qualitative material, however, is smaller and less refined. We need to know more about how ROM/CFS is used in psychotherapy, and why it is helpful for some patients, but not others.

Methods

We recorded therapy sessions of 12 patients who were using a CFS as part of their therapies at an outpatient clinic in Norway. We then conducted video-assisted interviews and follow-up interviews with patients. Data were analyzed with systematic text condensation.

Results

Analysis revealed three themes: (1) triggering reflections, emotions, and self-awareness, (2) Ambivalent and ambiguous self-presentation, and (3) potential for feeling understood and talking about what matters.

Conclusion

Answering questions in a CFS is an interpretative and intentional process of self-presentation and the results from ROM/CFS must be interpreted and explored in conversation to be clinically useful. When they are, they have potential for enhancing the therapeutic process by stimulating self-awareness, reflexivity, and allowing access to new therapeutic topics. Further research should explore this how-to aspect of ROM/CFS with different CFS and different types of patients. Integrating clinical feedback in therapeutic practice can be conceptualized as a clinical skill, which should be a part of training programs for therapists.

Electronic supplementary material

The online version of this article (10.1007/s11136-020-02550-1) contains supplementary material, which is available to authorized users.

Making way for a clinical feedback system in the narrow space between sessions: navigating competing demands in complex healthcare settings

BACKGROUND

Although substantial empirical research supports the clinical value of routine outcome measures/clinical feedback systems (ROM/CFS), translation into routine practice poses several challenges. The present case study investigated how stakeholders, clinicians, patients and clinical managers related to the implementation of the Norse Feedback (NF) in ordinary practice.

METHODS

We did an in-depth qualitative case study of the implementation of NF in a public mental-health institution. The settings were two outpatient clinics and two in-patient clinics organized under the same health trust. Data were drawn from three sources: archival sources (n = 16), field notes (n = 23), and 43 in-depth interviews with clinicians (n = 19), clinical managers (n = 5) and patients (n = 12). Ten of the participants were interviewed twice. The data were coded inductively and analyzed using a stringent qualitative methodology.

RESULTS

We present our findings under three inter-related domains. First, we describe what followed the clinical feedback implementation. Second, we present the context experienced as being complex and high on work-pressure. Third, we describe the situated rules about the priority between competing tasks.

CONCLUSIONS

The preliminary results complement and contextualize understandings of known barriers to implementing ROM/CFS in clinical settings. We apply a socio-material perspective to discuss clinicians' responses to complexity, implementation, and why some incentivized tasks prevailed over others regardless of therapists' perceived benefits.