A Novel Approach to the Assessment of Fidelity to a Cognitive Behavioral Therapy for PTSD Using Clinical Worksheets: A Proof of Concept With Cognitive Processing Therapy

Implementation Science and Practice-Oriented Research: Convergence and Complementarity

Implementation science is the scientific study of methods to promote the uptake of research findings and other evidence-based practices in routine care, with the goal of improving the quality and effectiveness of health services (Bauer et al., 2015). In addition to this common goal, practice-oriented psychotherapy research (and researchers) and implementation science (and scientists) share a common focus on the people and the places where treatment happens. Thus, there exists strong potential for combining these two approaches. In this article, we provide a primer on implementation science for psychotherapy researchers and highlight important areas and examples of convergence and complementarity between implementation science and practice-oriented psychotherapy research. Specifically, we (a) define and describe the core features of implementation science; (b) discuss similarities and areas of complementarity between implementation science and practice-oriented psychotherapy research; (c) discuss a case example that exemplifies the integration of implementation science and practice-oriented research; and (d) propose directions for future research and collaborations that leverage both implementation science and practice-oriented research.

Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation

Large language models (LLMs) such as Open AI's GPT-4 (which power ChatGPT) and Google's Gemini, built on artificial intelligence, hold immense potential to support, augment, or even eventually automate psychotherapy. Enthusiasm about such applications is mounting in the field as well as industry. These developments promise to address insufficient mental healthcare system capacity and scale individual access to personalized treatments. However, clinical psychology is an uncommonly high stakes application domain for AI systems, as responsible and evidence-based therapy requires nuanced expertise. This paper provides a roadmap for the ambitious yet responsible application of clinical LLMs in psychotherapy. First, a technical overview of clinical LLMs is presented. Second, the stages of integration of LLMs into psychotherapy are discussed while highlighting parallels to the development of autonomous vehicle technology. Third, potential applications of LLMs in clinical care, training, and research are discussed, highlighting areas of risk given the complex nature of psychotherapy. Fourth, recommendations for the responsible development and evaluation of clinical LLMs are provided, which include centering clinical science, involving robust interdisciplinary collaboration, and attending to issues like assessment, risk detection, transparency, and bias. Lastly, a vision is outlined for how LLMs might enable a new generation of studies of evidence-based interventions at scale, and how these studies may challenge assumptions about psychotherapy.

Getting More Out of Clinical Documentation: Can Clinical Dashboards Yield Clinically Useful Information?

This study investigated coded data retrieved from clinical dashboards, which are decision-support tools that include a graphical display of clinical progress and clinical activities. Data were extracted from clinical dashboards representing 256 youth (M age = 11.9) from 128 practitioners who were trained in the Managing and Adapting Practice (MAP) system (Chorpita & Daleiden in BF Chorpita EL Daleiden 2014 Structuring the collaboration of science and service in pursuit of a shared vision. 43(2):323 338. 2014, Chorpita & Daleiden in BF Chorpita EL Daleiden 2018 Coordinated strategic action: Aspiring to wisdom in mental health service systems. 25(4):e12264. 2018) in 55 agencies across 5 regional mental health systems. Practitioners labeled up to 35 fields (i.e., descriptions of clinical activities), with the options of drawing from a controlled vocabulary or writing in a client-specific activity. Practitioners then noted when certain activities occurred during the episode of care. Fields from the extracted data were coded and reliability was assessed for Field Type, Practice Element Type, Target Area, and Audience (e.g., Caregiver Psychoeducation: Anxiety would be coded as Field Type = Practice Element; Practice Element Type = Psychoeducation; Target Area = Anxiety; Audience = Caregiver). Coders demonstrated moderate to almost perfect interrater reliability. On average, practitioners recorded two activities per session, and clients had 10 unique activities across all their sessions. Results from multilevel models showed that clinical activity characteristics and sessions accounted for the most variance in the occurrence, recurrence, and co-occurrence of clinical activities, with relatively less variance accounted for by practitioners, clients, and regional systems. Findings are consistent with patterns of practice reported in other studies and suggest that clinical dashboards may be a useful source of clinical information. More generally, the use of a controlled vocabulary for clinical activities appears to increase the retrievability and actionability of healthcare information and thus sets the stage for advancing the utility of clinical documentation.

Development and Evaluation of a Pragmatic Measure of Adherence to Dialectical Behavior Therapy: The DBT Adherence Checklist for Individual Therapy

This paper presents two studies conducted to develop and evaluate a new pragmatic measure of therapist adherence to Dialectical Behavior Therapy (DBT): the DBT Adherence Checklist for Individual Therapy (DBT AC-I). Study 1 used item response analysis to select items from the gold standard DBT Adherence Coding Scale (DBT ACS) using archival data from 1271 DBT sessions. Items were then iteratively refined based on feedback from 33 target end-users to ensure relevance, usability, and understandability. Study 2 examined the psychometric properties of the DBT AC-I as a therapist self-report and observer-rated measure in 100 sessions from 50 therapist-client dyads, while also evaluating predictors of therapist accuracy in self-rated adherence. When used as a therapist self-report measure, concordance between therapist and observer ratings was at least moderate (AC1 ≥ 0.41) for all DBT AC-I items but overall concordance (ICC = 0.09) as well as convergent (r = 0.05) and criterion validity (AUC = 0.54) with the DBT ACS were poor. Higher therapist accuracy was predicted by greater DBT knowledge and adherence as well as more severe client suicidal ideation. When used by trained observers, the DBT AC-I had excellent interrater reliability (ICC = 0.93), convergent validity (r = 0.90), and criterion validity (AUC = 0.94). While therapists' self-rated adherence on the DBT AC-I should not be assumed to reflect their actual adherence, some therapists may self-rate accurately. The DBT AC-I offers an effective and relatively efficient method of evaluating adherence to DBT when used by trained observers.

Measuring the active elements of cognitive-behavioral therapies

Understanding how and for whom cognitive-behavioral therapies work is central to the development and improvement of mental health interventions. Suboptimal quantification of the active elements of cognitive-behavioral therapies has hampered progress in elucidating mechanisms of change. To advance process research on cognitive-behavioral therapies, we describe a theoretical measurement framework that focuses on the delivery, receipt, and application of the active elements of these interventions. We then provide recommendations for measuring the active elements of cognitive-behavioral therapies aligned with this framework. Finally, to support measurement harmonization and improve study comparability, we propose the development of a publicly available repository of assessment tools: the Active Elements of Cognitive-Behavioral Therapies Measurement Kit.

What is the Status of Multi-Informant Treatment Fidelity Research?

OBJECTIVE

The precise measurement of treatment fidelity (quantity and quality in the delivery of treatment strategies in an intervention) is essential for intervention development, evaluation, and implementation. Various informants are used in fidelity assessment (e.g., observers, practitioners [clinicians, teachers], clients), but these informants often do not agree on ratings. This scoping review aims to ascertain the state of science around multi-informant assessment of treatment fidelity.

METHOD

A literature search of articles published through December 2021 identified 673 articles. Screening reduced the number of articles to 44, and the final study set included 35 articles.

RESULTS

There was substantial variability across studies regarding study design, how fidelity was operationalized, and how reliability was defined and assessed. Most studies evaluated the agreement between independent observers and practitioner-report, though several other informant pairs were assessed. Overall, findings suggest that concordance across fidelity informants was low to moderate, with a few key exceptions.

CONCLUSIONS

It is difficult to draw clear conclusions about the degree to which single versus multiple informant assessment is needed to produce an accurate and complete picture of treatment fidelity. The field needs to take steps to determine how to leverage multi-informant assessment to accurately assess treatment fidelity.

Implementing Evidence-Based Mental-Health Treatments: Attending to Training, Fidelity, Adaptation, and Context

This article provides a very high-level overview of some key considerations in the field of implementation science as it relates to clinical psychology. The article reviews recent findings regarding treatment fidelity, adaptation of treatments, and clinical outcomes. It then details some recent findings on training therapists to provide evidence-based therapies as they were designed to be delivered, with sufficient skill and adaptations if needed to ensure better fit for clients who receive treatments in different treatment settings. Finally, the article considers implications for implementing and sustaining new treatments and supporting their fidelity and adaptation, as well as key directions for future research.

Launching a Competency-Based Training Program in Evidence-Based Treatments for PTSD: Supporting Veteran-Serving Mental Health Providers in Texas

Community mental health providers play an essential role in delivering services to veterans who either have limited access to U.S. Department of Veterans Affairs (VA) facilities or who prefer to seek care outside of the VA. However, there are limited training opportunities in evidence-based treatments for posttraumatic stress disorder (PTSD) outside of the VA. In 2017, the STRONG STAR Training Initiative was established to develop competency-based training in two evidence-based therapies for PTSD and to provide that training for mental health providers serving veterans and their families in community settings in Texas. This article describes the program's development and implementation, baseline characteristics of participating clinicians, and lessons learned toward the scale-up and extension of this competency-based training effort to include other interventions and locations.