Initial quantitative development of the Norse Feedback system: a novel clinical feedback system for routine mental healthcare

Leveraging Implementation Science to Integrate Digital Mental Health Interventions as part of Routine Care in a Practice Research Network

The supply / demand issue in behavioral health care is a well-established fact, and the mental health toll of the COVID-19 pandemic continues to add challenges to an already taxed system. Existing healthcare models are not set up to adequately address the increasing mental health related needs. As such, innovative models are needed to provide patients with access to appropriate, evidence-based behavioral health care within routine clinical care. This paper introduces Precision Behavioral Health (PBH) as an example of such a model. PBH is an innovative, digital first care delivery model that provides an ecosystem of evidence-based digital mental health interventions to patients as a frontline behavioral health treatment within routine care in a large multispecialty group medical center in the United States. This paper describes the implementation of PBH within a practice research network set-up as part of an integrated behavioral health department. We will present how our team leveraged the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance; "What is RE-AIM?," n.d.) implementation science framework, which emphasizes the design, dissemination, and implementation processes at the individual, staff, and organizational levels, to prioritize key implementation constructs to enhance the successful integration of PBH within routine care. We describe how each of these constructs were operationalized to aid data gathering for rapid evaluation and lessons learned. We discuss the benefits of these types of initiatives across multiple stakeholders including patients, providers, organizations, payers, and digital intervention vendors.

Routine Outcome Monitoring and Clinical Feedback in Psychotherapy: Recent Advances and Future Directions

In the past decade, there has been an increase in research related to the routine collection and active use of standardized patient data in psychotherapy. Research has increasingly focused on personalization of care to patients, clinical skills and interventions that modulate treatment outcomes, and implementation strategies, all of which appear to enhance the beneficial effects of ROM and feedback. In this article, we summarize trends and recent advances in the research on this topic and identify several essential directions for the field in the short to medium term. We anticipate a broadening of research from the focus on average effects to greater specificity around what kinds of feedback, provided at what time, to which individuals, in what settings, are most beneficial. We also propose that the field needs to focus on issues of health equity, ensuring that ROM can be a vehicle for increased wellbeing for those who need it most. The complexity of mental healthcare systems means that there may be multiple viable measurement solutions with varying costs and benefits to diverse stakeholders in different treatment contexts, and research is needed to identify the most influential components in each of these contexts.

The Connections Matter: Bi-Directional Learning in Program Evaluation and Practice-Oriented Research

Intended for researchers and clinical leaders, this article suggests that embedded program evaluation is a good fit with the desired features of practice-oriented research. The systematic nature of evaluation that is built into the operational workflow of a practice setting may increase the diversity of methods available to explore processes and outcomes of interest. We propose a novel conceptual framework that uses a human-centered systems lens to foster such embedded evaluation in clinical routine. This approach emphasizes the evaluator-practitioner partnership to build confidence in the bi-directional learning of practice-based evidence with evidence-based practice. The iterative cycles inherent to design thinking are aimed at developing better evaluation questions. The attention to structure and context inherent to systems thinking is intended to support meaningful perspectives in the naturally complex world of health care. Importantly, the combined human-centered systems lens can create greater awareness of the influence of individual and systemic biases that exist in any endeavor or institution that involves people. Recommended tools and strategies include systems mapping, program theory development, and visual facilitation using a logic model to represent the complexity of mental health treatment for communication, shared understanding, and connection to the broader evidence base. To illustrate elements of the proposed conceptual framework, two case examples are drawn from routine outcome monitoring (ROM) and progress feedback. We conclude with questions for future collaboration and research that may strengthen the partnership of evaluators and practitioners as a community of learners in service of local and system-level improvement.

Case report: A dual case study of how clinical feedback can be a communication aide and influence therapeutic work

Background

While routine outcome monitoring and clinical feedback may improve outcomes after psychotherapy, results from efficiency studies have been mixed. Moreover, how clinical feedback is implemented influences how it works for patients and clinicians, and working mechanisms are hitherto not thoroughly explored. Researchers have argued that inviting and using feedback from patients is best conceived of as a clinical skill. In this paper, we use case study methodology to explore and describe feedback's functions within three clinical skill themes: actualizing alliance work, concretizing change and stagnation and verbalizing the non-verbal.

Case presentation

Sonja is a young adult patient with a trauma background. She has a history of serious suicide attempts and distrust in relationships. She attended psychotherapy for eight months. Harald is a middle-aged man with a stable family. Traumatic events in his past has made him conceal own needs and developing depression. He attended psychotherapy for 19 months. Case material include the patient's clinical feedback over a range of life areas, medical health notes and the therapist's process notes.

Conclusion

Clinical feedback can be a positive supplement to the therapeutic work and process. The importance of making this as a joint tool between the client and the therapist is significant.

Routine Outcome Monitoring (ROM) and Feedback: Research Review and Recommendations

OBJECTIVE

To provide a research review of the components and outcomes of routine outcome monitoring (ROM) and recommendations for research and therapeutic practice.

METHOD

A narrative review of the three phases of ROM - data collection, feeding back data, and adapting therapy - and an overview of patient outcomes from 11 meta-analytic studies.

RESULTS

Patients support ROM when its purpose is clear and integrated within therapy. Greater frequency of data collection is more important for shorter-term therapies, and use of graphs, greater specificity of feedback, and alerts are helpful. Overall effects on patient outcomes are statistically significant (g ≈ 0.15) and increase when clinical support tools (CSTs) are used for not-on-track cases (g ≈ 0.36-0.53). Effects are additive to standard effects of psychological therapies. Organizational, personnel, and resource issues remain the greatest obstacles to the successful adoption of ROM.

CONCLUSION

ROM offers a low-cost method for enhancing patient outcomes, on average resulting in an ≈ 8% advantage (success rate difference; SRD) over standard care. CSTs are particularly effective for not-on-track patients (SRD between ≈ 20% and 29%), but ROM does not work for all patients and successful implementation is a major challenge, along with securing appropriate cultural adaptations.

How to make the most of routine outcome monitoring (ROM): A multitude of clinical decisions and nuances to consider

Routine outcome monitoring (ROM) involves the use of patient-reported standardized outcome measures to monitor progress throughout the course of treatment, followed by feedback of the patient's scores to the therapist. The potential benefits of ROM have been established, however, from our own experiences, we know that the implementation in clinical practice can be challenging. We therefore wanted to explore in more detail exactly how we might be able to apply ROM in difficult clinical contexts. The inspiring case illustrations in this issue of Journal of Clinical Psychology: In Session highlight the heterogeneity in ROM systems, and the way in which ROM can be used in treatment. Just as there are many ways of interpreting a survey data-point, there are also many ways in which ROM may be used to complement the treatment and supervision. Whether or not ROM is implemented may partly be determined by clinic policies and routines, but there remain a multitude of clinical decisions that require careful consideration by the individual therapist. To complement the evidence supporting the benefits of using ROM, further empirical support and clinical guidance is needed on how exactly therapists are to use ROM in their work and how ROM may be used in evidence-based practice. We make suggestions for additional uses of ROM for deliberate practice, and teletherapy practice, and look toward novel ways of assessing progress in the near future.

Measurement-Based and Data-Informed Psychological Therapy

Outcome measurement in the field of psychotherapy has developed considerably in the last decade. This review discusses key issues related to outcome measurement, modeling, and implementation of data-informed and measurement-based psychological therapy. First, an overview is provided, covering the rationale of outcome measurement by acknowledging some of the limitations of clinical judgment. Second, different models of outcome measurement are discussed, including pre-post, session-by-session, and higher-resolution intensive outcome assessments. Third, important concepts related to modeling patterns of change are addressed, including early response, dose-response, and nonlinear change. Furthermore, rational and empirical decision tools are discussed as the foundation for measurement-based therapy. Fourth, examples of clinical applications are presented, which show great promise to support the personalization of therapy and to prevent treatment failure. Finally, we build on continuous outcome measurement as the basis for a broader understanding of clinical concepts and data-driven clinical practice in the future. Expected final online publication date for the Annual Review of Clinical Psychology, Volume 18 is May 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

How therapists and patients need to develop a clinical feedback system after 18 months of use in a practice-research network: a qualitative study

BACKGROUND

A personalized computer-adaptive system for clinical feedback and routine outcome monitoring in mental health, Norse Feedback aims to bridge the needs for standardized and idiographic measures in ordinary practice.

METHODS

Item response theory analyses of completed treatment processes (n = 800) informed a qualitative study comprising individual in-depth interviews and focus groups with patients (n = 9) and clinicians (n = 10). The research question was: How do clinicians and patients contribute to developing a clinical feedback system in a continuous process aimed at refining its clinical usefulness?

RESULTS

We conducted thematic analyses and found five themes: 1. Added clinical needs, 2. Needs for re-organizing the clinician report, 3.Needs for differentiation of clinical content, 4. User-interface needs, and 5. Item level suggestions.

CONCLUSION

In this article, we detail resulting needs for continuous adaptation to practice, and discuss implications of the concrete experiences with the Norse action research program for the larger field of ROM/CFS implementation.