Complexity and potentials of clinical feedback in mental health: an in-depth study of patient processes

Response processes for patients providing quantitative self-report data: a qualitative study

OBJECTIVE

To identify factors that influence response processes for patients providing quantitative self-report data. Secondly, due to the lack of integrative and explanatory models in this area, to develop a model of patients' response processes that can guide what to look for when considering validity evidence and interpreting scores on individual items.

METHODS

Participants (n = 13) were recruited from a specialized substance use disorder treatment clinic and interviewed while responding to items from a clinical feedback system implemented for routine outcome monitoring in that setting. The interview approach was based on cognitive interviewing. Data collection and analysis were inspired by a grounded theory approach.

RESULTS

We identified several variables that influenced the participants' response processes. The variables were organized into five categories: context-related variables; item-related variables; response base variables; reasoning strategies; and response selection strategies. We also found that the participants' responses for many items were affected by different aspects of the response process in ways that are relevant to interpretation but not necessarily discernible from the numerical scores alone, and we developed response categories to capture this.

CONCLUSION

The findings suggest that patients providing quantitative self-report data encounter conditions in the response process that challenge and influence their ability to convey meaning and accuracy. This results in responses that for many of the items reflect messages important for interpretation and follow-up, even if it does not appear from the numerical scores alone. The proposed model may be a useful tool when developing items, assessing validity, and interpreting responses.

Routine Outcome Monitoring and Clinical Feedback in Psychotherapy: Recent Advances and Future Directions

In the past decade, there has been an increase in research related to the routine collection and active use of standardized patient data in psychotherapy. Research has increasingly focused on personalization of care to patients, clinical skills and interventions that modulate treatment outcomes, and implementation strategies, all of which appear to enhance the beneficial effects of ROM and feedback. In this article, we summarize trends and recent advances in the research on this topic and identify several essential directions for the field in the short to medium term. We anticipate a broadening of research from the focus on average effects to greater specificity around what kinds of feedback, provided at what time, to which individuals, in what settings, are most beneficial. We also propose that the field needs to focus on issues of health equity, ensuring that ROM can be a vehicle for increased wellbeing for those who need it most. The complexity of mental healthcare systems means that there may be multiple viable measurement solutions with varying costs and benefits to diverse stakeholders in different treatment contexts, and research is needed to identify the most influential components in each of these contexts.

The Connections Matter: Bi-Directional Learning in Program Evaluation and Practice-Oriented Research

Intended for researchers and clinical leaders, this article suggests that embedded program evaluation is a good fit with the desired features of practice-oriented research. The systematic nature of evaluation that is built into the operational workflow of a practice setting may increase the diversity of methods available to explore processes and outcomes of interest. We propose a novel conceptual framework that uses a human-centered systems lens to foster such embedded evaluation in clinical routine. This approach emphasizes the evaluator-practitioner partnership to build confidence in the bi-directional learning of practice-based evidence with evidence-based practice. The iterative cycles inherent to design thinking are aimed at developing better evaluation questions. The attention to structure and context inherent to systems thinking is intended to support meaningful perspectives in the naturally complex world of health care. Importantly, the combined human-centered systems lens can create greater awareness of the influence of individual and systemic biases that exist in any endeavor or institution that involves people. Recommended tools and strategies include systems mapping, program theory development, and visual facilitation using a logic model to represent the complexity of mental health treatment for communication, shared understanding, and connection to the broader evidence base. To illustrate elements of the proposed conceptual framework, two case examples are drawn from routine outcome monitoring (ROM) and progress feedback. We conclude with questions for future collaboration and research that may strengthen the partnership of evaluators and practitioners as a community of learners in service of local and system-level improvement.

Therapists' and patients' experiences of using patients' self-reported data in ongoing psychotherapy processes-A systematic review and meta-analysis of qualitative studies

OBJECTIVE

Using patient-generated quantitative data in psychotherapy (feedback) appears to enhance treatment outcome, but there is variability in its effect. Different ways and reasons to implement routine outcome measurement might explain such variability. The goal of this review is to address the insufficient knowledge on how these data are used by therapists and patients.

METHODS

The present study is a systematic review and meta-analysis of qualitative reports of therapists' and patients' experiences using patient-generated quantitative data during ongoing psychotherapy.

RESULTS

Four main categories of use were identified: (1) uses of patients' self-reported data as nomothetic/objective markers for assessment, process monitoring, and treatment planning; (2) intrapersonal uses that enhance self-awareness, initiate reflection, and influence patients' mood or responses; (3) uses that prompt interactional processes by facilitating communication, supporting exploration, creating ownership in patients, changing treatment focus, enhancing therapeutic alliance, or disturbing the psychotherapy process; and (4) patients responding for specific purposes due to uncertainty and interpersonal motives, or strategic responding to achieve a desired result.

CONCLUSION

These results demonstrate that patient-reported data, when used in active psychotherapy, is very clearly not just an objective measurement of client functioning: the inclusion of patient-data has the potential to influence psychotherapy in numerous ways.

Case report: A dual case study of how clinical feedback can be a communication aide and influence therapeutic work

Background

While routine outcome monitoring and clinical feedback may improve outcomes after psychotherapy, results from efficiency studies have been mixed. Moreover, how clinical feedback is implemented influences how it works for patients and clinicians, and working mechanisms are hitherto not thoroughly explored. Researchers have argued that inviting and using feedback from patients is best conceived of as a clinical skill. In this paper, we use case study methodology to explore and describe feedback's functions within three clinical skill themes: actualizing alliance work, concretizing change and stagnation and verbalizing the non-verbal.

Case presentation

Sonja is a young adult patient with a trauma background. She has a history of serious suicide attempts and distrust in relationships. She attended psychotherapy for eight months. Harald is a middle-aged man with a stable family. Traumatic events in his past has made him conceal own needs and developing depression. He attended psychotherapy for 19 months. Case material include the patient's clinical feedback over a range of life areas, medical health notes and the therapist's process notes.

Conclusion

Clinical feedback can be a positive supplement to the therapeutic work and process. The importance of making this as a joint tool between the client and the therapist is significant.

Routine Outcome Monitoring (ROM) and Feedback: Research Review and Recommendations

OBJECTIVE

To provide a research review of the components and outcomes of routine outcome monitoring (ROM) and recommendations for research and therapeutic practice.

METHOD

A narrative review of the three phases of ROM - data collection, feeding back data, and adapting therapy - and an overview of patient outcomes from 11 meta-analytic studies.

RESULTS

Patients support ROM when its purpose is clear and integrated within therapy. Greater frequency of data collection is more important for shorter-term therapies, and use of graphs, greater specificity of feedback, and alerts are helpful. Overall effects on patient outcomes are statistically significant (g ≈ 0.15) and increase when clinical support tools (CSTs) are used for not-on-track cases (g ≈ 0.36-0.53). Effects are additive to standard effects of psychological therapies. Organizational, personnel, and resource issues remain the greatest obstacles to the successful adoption of ROM.

CONCLUSION

ROM offers a low-cost method for enhancing patient outcomes, on average resulting in an ≈ 8% advantage (success rate difference; SRD) over standard care. CSTs are particularly effective for not-on-track patients (SRD between ≈ 20% and 29%), but ROM does not work for all patients and successful implementation is a major challenge, along with securing appropriate cultural adaptations.

Relationship and Alliance Formation Processes in Psychotherapy: A Dual-Perspective Qualitative Study

Objective

To explore how therapists and clients act dyadically to establish a therapeutic relationship during the first five sessions of psychotherapy. The study aimed to identify both relational facilitative and hindering processes occurring in routine care.

Methods

Using the method ‘interpersonal process recall’ (IPR), we videotaped the third and fifth session of 12 psychotherapy dyads, and conducted video-assisted interviews with each therapist and client separately. In total, the data material consist of 47 IPR interviews. Data were analyzed using a thematic approach.

Results

The analysis process revealed two main groups. The first group consisted of dyads with a positive relational outcome, and the second group consisted of dyads with a troubled or frail relational outcome. During the initial phase of therapy, clients described feeling overwhelmed by fear and shame. Positive relational development occurred when these emotions were successfully accommodated and replaced with a growing sense of safety with the therapist. However, the relationship became troubled when the client experienced an increase in shame and/or fear during the first sessions. When forming a therapeutic relationship, it is vital that the client experience the therapist as genuine and skilled, and that the therapist is able to engage and connect deeply with the client on a person-to-person level. The article further provides a discussion on how these dyadic experiences align with the working alliance and real relationship, and how the two consolidate during the first sessions of psychotherapy.

Conclusion

The current study explored the complex relational processes underlying the formation of the therapeutic relationship. Core aspects of the real relationship are prerequisites to forming a collaborative working alliance in which both therapist and client are actively engaged. Facilitating a positive relationship is crucial in the early phase of psychotherapy, and therapists can actively identify and repair ruptures at this time.

How therapists and patients need to develop a clinical feedback system after 18 months of use in a practice-research network: a qualitative study

BACKGROUND

A personalized computer-adaptive system for clinical feedback and routine outcome monitoring in mental health, Norse Feedback aims to bridge the needs for standardized and idiographic measures in ordinary practice.

METHODS

Item response theory analyses of completed treatment processes (n = 800) informed a qualitative study comprising individual in-depth interviews and focus groups with patients (n = 9) and clinicians (n = 10). The research question was: How do clinicians and patients contribute to developing a clinical feedback system in a continuous process aimed at refining its clinical usefulness?

RESULTS

We conducted thematic analyses and found five themes: 1. Added clinical needs, 2. Needs for re-organizing the clinician report, 3.Needs for differentiation of clinical content, 4. User-interface needs, and 5. Item level suggestions.

CONCLUSION

In this article, we detail resulting needs for continuous adaptation to practice, and discuss implications of the concrete experiences with the Norse action research program for the larger field of ROM/CFS implementation.

Initial quantitative development of the Norse Feedback system: a novel clinical feedback system for routine mental healthcare

Purpose

As routine outcome monitoring has become prevalent in psychological practice, there is need for measurement tools covering diverse symptoms, treatment processes, patient strengths, and risks. Here we describe the development and initial tests of the psychometric properties of a multi-scale system for use in mental healthcare, Norse Feedback.

Methods

In Study 1, we present the item-generation process and structure of the Norse Feedback, a 17-scale digital-first measurement tool for psychopathology and treatment-relevant variables. In Study 2, we present analyses of this initial measure in a nonclinical sample of 794 healthy controls and a sample of 222 mental health patients. In Study 3, we present the analysis of a revised 20-scale system in two separate samples of patients. In each analysis, we investigate item and test information in particular, including analysis of differential item functioning on gender, age, site, and sample differences where applicable.

Results

Scales performed variably. Changes to items and scales are described. Several scales appeared to reliably discriminate individuals entering mental health treatment on severity, and others are less reliable. Marked improvements in scale internal consistency and measurement precision were observed between the first and second implemented versions.

Conclusion

This system includes some scales with reasonable structural validity, though several areas for future development are identified. The system was developed to be iteratively re-evaluated, to strengthen the validity of its scales over time. There are currently a number of limitations on inferences from these scores, which future developments should address.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11136-021-02825-1.

Clinical dilemmas of routine outcome monitoring and clinical feedback: A qualitative study of patient experiences

ABSTRACT Purpose: Routine outcome monitoring (ROM) and clinical feedback systems (CFS) are becoming prevalent in mental health services, but there are several challenges to successful implementation. ROM/CFS seem to be helpful for some patients, but not for others. To investigate this, we explored patients' experiences with ROM/CFS as an interpersonal and psychotherapeutic process, in naturalistic settings. Method: We used video-assisted interpersonal process recall interviews to investigate the experiences of 12 patients using ROM/CFS in a Norwegian mental health outpatient clinic. Data were analyzed through systematic text condensation. Results: Our analysis resulted in three pairs of experiences with ROM/CFS: (1) Explicit vs. implicit use of information, (2) Directing focus towards or away from therapeutic topics, and (3) Giving and receiving feedback. These experiences could be helpful or hindering, depending on participants' needs and preferences. All participants needed to know that the CFS was used in a meaningful way. If not, it could be detrimental to the therapeutic process. Conclusion: In order to be helpful for patients, ROM/CFS should be used in a way that is flexible, meaningful to patients, and sensitive to individual needs and preferences. Future research should further explore this how-to aspect of ROM/CFS with different CFS and populations.