A Statewide Common Elements Initiative for Children's Mental Health

Identification of treatment elements for adolescents with callous unemotional traits: a systematic narrative review

Adolescents with callous unemotional (CU) traits are at risk for poor quality-of-life outcomes such as incarceration, suicide, and psychopathy. It is currently unknown which treatments are implemented with CU adolescents specifically and which elements make up these interventions. A narrative systematic review was used to identify the treatments and common elements used with CU adolescents (12-18 years). Eligible studies were randomized controlled trials and quasi-experimental studies evaluating psychosocial interventions, delivered within a clinical context, and directed towards the adolescent or their family. Eight studies with 1291 participants were included. Significant decreases in CU traits were demonstrated only in a minority of studies. The most utilized practice elements were set goals for treatment, practice interpersonal/communication skills, prepare for termination, and teach parents skills and strategies; the most utilized process elements were formal therapy, practice exercises, important others, and flexible/adaptive. A majority of the studies did not report an implementation element. While theoretical frameworks for CU children are employed with CU adolescents, there is still uncertainty regarding their suitability for this age group. Given the limited empirical evidence, there is a critical need for further exploration.

Translating research evidence into youth behavioral health policy and action: using a community-engaged storyboard approach

Introduction

There is nationwide shortage of child and adolescent behavioral health providers. Lack of diversity in the mental health care profession compounds workforce capacity issues, contributing to greater disparities in treatment access and engagement for youth from historically disenfranchised communities. Strategies are needed to foster cross-sector alignment to inform policy which can improve mental health care access and reduce disparities. This current case study details a specific research-practice-policy partnership strategy, storyboarding, as a method to engage community partners in Washington State to deliberate on information drawn from research on non-specialist models of child and adolescent mental health care to support the behavioral workforce expansion.

Method

Research evidence from a scoping literature review on non-specialist models of child and adolescent mental health care was shared via storyboards with community partners to inform policy efforts around the behavioral health workforce expansion. In Phase 1, community members with lived experience and clinical expertise contributed to the storyboard design process. In Phase 2, a broader community partner group shared their perspectives on the models of care presented in the storyboards via Qualtrics survey with open-ended questions. Listening sessions were also held with non-English speaking refugee and immigrant communities to elicit feedback on whether these models of care would meet their needs. Qualitative data was coded to explore emerging themes using a rapid deductive approach.

Results

Community partners shared mixed responses to models of care presented from the research literature. Immigrant and refugee communities explicitly stated these existing models would not fit their context. Regarding partnership strategy success, the smaller community partner group was engaged in the storyboard design process. The broader community interacted with and provided detailed responses to the models of care presented in the storyboards. Success was also reflected in community partners' continued participation in the next stage of the project.

Discussion

Findings demonstrate how storyboarding can be effectively used to translate research evidence into accessible information to promote community partner engagement and capture community voice in policy processes. More work is needed exploring how such methods can be used to increase the use of research evidence in policy and practice spaces.

CBT+ Training Initiative in Washington State Community Mental Health: An Evaluation of Child Clinical Outcomes

The Washington State CBT+  Initiative offers a flexible training and consultation approach for community mental health providers in evidence-based practices for four child mental health targets: cognitive behavioral therapy for depression, anxiety, trauma, and behavioral difficulties. As part of consultation, clinicians used an online system to track delivery of treatment components and clinical outcomes using standardized symptom measures. The current study used these clinician-input data to examine symptom change for children using paired sample t-tests. Additionally, we explored if time elapsed or number of sessions between measurements related to symptom change using simple linear regression. Children had significant symptom reduction across all four targets. For most measures, children did not show greater improvements with increased length of time or increased number of sessions between assessment measures. Findings suggest that children treated by a CBT+  trained clinician may demonstrate symptom reduction for their primary clinical problem. Findings add to support for flexible training approaches for community mental health clinicians.

Supervisory Alliance as a Moderator of the Effects of Behavioral Rehearsal on TF-CBT Fidelity: Results from a Randomized Trial of Supervision Strategies

Exposure is an important element of treatment for many evidence-based treatments but can be challenging to implement. Supervision strategies to support exposure delivery may be an important tool to facilitate the use of exposure techniques; however, they must be considered and used in the context of the supervisory alliance. The present study examined relations between supervisory alliance and fidelity to the trauma narrative (TN; i.e., imaginal exposure) component of Trauma-Focused Cognitive Behavioral Therapy. We also examined how supervisory alliance moderated the effect of behavioral rehearsal use in supervision on TN fidelity. We analyzed data from a randomized controlled trial, in which forty-two supervisors and their clinicians (N = 124) from 28 Washington State community-based mental health offices participated. Clinicians were randomized to receive one of two supervision conditions-symptom and fidelity monitoring (SFM) or SFM with behavioral rehearsal (SFM + BR). Supervisory alliance alone did not predict delivery (i.e., occurrence) or extensiveness of delivery of the trauma narrative. Client-focused supervisory alliance moderated the effectiveness of behavioral rehearsal-as client-focused alliance increased, the odds of delivering the TN also increased significantly. Future research should further investigate how to appropriately match supervision techniques with supervisory dyads and explore the interplay of alliance with supervision techniques a supervisor might employ.

Study protocol: Novel Methods for Implementing Measurement-Based Care with youth in Low-Resource Environments (NIMBLE)

BACKGROUND

For youth receiving care in community mental health centers, comorbidities are the rule rather than the exception. Using measurement-based care (MBC), or the routine evaluation of symptoms to inform care decisions, as the foundation of treatment for youth with comorbid problems significantly improves the impact of psychotherapy by focusing care and building engagement and alliance. MBC increases the rate of symptom improvement, detects clients who would otherwise deteriorate, and alerts clinicians to non-responders. Despite its demonstrated utility, MBC is rarely implemented with fidelity; less than 15% of providers report using MBC per recommendations. Previous efforts to support MBC implementation have yielded suboptimal outcomes, in part, due to organizations' challenges with identifying and prioritizing barriers and selecting and developing strategies to overcome them. New methods are needed for identifying and prioritizing barriers, and matching strategies to barriers to optimize MBC implementation and treatment quality to improve youth mental health outcomes in community settings.

METHODS

Pragmatic implementation methods will be piloted in four diverse community mental health centers. Methods include (a) rapid evidence synthesis; (b) rapid ethnography; (c) design kits (e.g., kits with disposable cameras, journals, maps); (d) barrier prioritization, and (e) causal pathway diagramming. These activities will generate actionable barriers; subsequently, we will use facilitated group processes to prioritize barriers and develop causal pathway diagrams to match strategies to barriers to create implementation plans that optimize MBC fidelity (Aim 1). We will track strategy deployment for 6 months, then compare MBC fidelity for another 6 months post-implementation with data from 2 years of historical controls (Aim 2). Finally, we will co-design a toolkit for design kit methods with youth and the practice and scientific communities (Aim 3).

DISCUSSION

Optimizing MBC implementation in community mental health centers could transform youth mental health care by ensuring the most pressing symptoms are targeted early in treatment. The discussion section highlights expected challenges and limits to using the five methods, including recruitment and engagement given the high pressure on community mental health settings.

TRIAL REGISTRATION

Clinicaltrials.gov. NCT05644756 . Registered on 18 November 2022. This trial was retrospectively registered.

Policy for Equity: Associations Between Community Mental Health Agency Policies and Clinicians' Cultural Competence

Systemic reform is needed to address racism as a root cause of mental health inequities, such as understanding how community mental health (CMH) agencies' practices and policies may impact care provided to racially minoritized populations. This study described and examined associations between CMH clinicians' multicultural knowledge and awareness and agency practices and policies to improve care for Clients of Color. CMH clinicians (N = 119) across Washington State reported on their multicultural competence and agencies' practices and policies in an online survey. Multicultural competence was assessed with the Multicultural Counseling Knowledge and Awareness Scale (MCKAS), which assesses respondents' knowledge of multicultural counseling frameworks and awareness of multicultural counseling issues. Agency policies were examined with an adapted version of the Multiculturally Competent Service System (MCSS) Assessment Guide, which asked respondents to endorse the degree to which their agencies had taken specific steps to better serve racially and ethnically minoritized populations across 11 domains, including policies, linguistic diversity in services, and quality monitoring and improvement. Multicultural knowledge and awareness were generally high across the sample. Clinicians commonly endorsed that their agencies had mission statements that were committed to cultural competence. Endorsement of concrete steps to improve services for non-English speaking clients was associated with greater multicultural knowledge and awareness, and practices to monitor and improve care provided to Clients of Color were associated with lower scores. Addressing mental health inequities requires multifaceted solutions. Results highlight the potential of examining agency practices and policies as one solution to improve care for Clients of Color.

Evidence-based Treatment for Substance Use Disorders in Community Mental Health Centers: the ACCESS Program

A significant gap remains in the availability and accessibility of evidence-based treatments (EBTs) in community substance use disorder (SUD) treatment. This study describes a 2-year statewide training initiative that sought to address this gap by training community-based therapists in motivational enhancement/cognitive behavioral therapy (MET/CBT). Therapists (N = 93) participated in a 2-day MET/CBT workshop followed by bi-weekly clinical consultation, fidelity monitoring, guided readings, and online resources. Therapists completed pre-training and follow-up assessments measuring knowledge, attitudes, confidence, and implementation barriers. Most therapists attended 10 or more consultation calls. Submission of session recordings for feedback was the least utilized training element. Therapists reported increased confidence in their ability to implement MET/CBT for SUD and demonstrated improvement in MI and CBT knowledge. Therapists reported several implementation barriers, including lack of time and opportunity to treat patients with MET/CBT. Recommendations for future training initiatives and addressing the barriers identified in this study are discussed.

Implementation of Evidence-Based Practices within Treatment-As-Usual and Evidence-Based Practice Initiatives

Publicly funded initiatives are underway to improve implementation of evidence-based practices (EBP) in youth mental health services. However, we know little about the success of these initiatives or about EBP implementation independent of such initiatives. We examined EBP implementation in a treatment as usual (TAU) state and in six states with publicly funded EBP initiatives (EBPIs). In Study 1, we examined providers' use of practices derived from the evidence base (PDEB) and their predictors among 780 providers in a TAU state. In Study 2, we conducted a systematic review of implementation strategies, outcomes, and predictors of EBP use in six state funded EBPIs. Study 1 suggests TAU providers use PDEB alongside practices without consistent research support; provider racial/ethnic minority status, learning theory orientation, and manual use predict greater PDEB use. Study 2 indicates EBPIs employ multiple recommended implementation strategies with variable outcomes across studies and measurement approaches. Predictors of EBP use in EBPIs also varied, though training, setting, and youth age were consistent predictors across studies. While sample differences and inconsistent measurement across studies made direct comparisons somewhat tenuous, rates of PDEB use in the TAU sample appeared similar to those in publicly funded EBPIs. However, two states reported comparisons with TAU samples and found higher EBP implementation under EBPI. Different predictors impacted EBP use in TAU versus EBPIs. Our findings highlight the need for improved evaluation of EBPIs including clear reporting standards for outcomes and more consistent, standardized measurement of EBP use in order to better understand and improve EBPIs.

How low can you go? Examining the effects of brief online training and post-training consultation dose on implementation mechanisms and outcomes for measurement-based care

BACKGROUND

Initial training and ongoing post-training consultation (i.e., ongoing support following training, provided by an expert) are among the most common implementation strategies used to change clinician practice. However, extant research has not experimentally investigated the optimal dosages of consultation necessary to produce desired outcomes. Moreover, the degree to which training and consultation engage theoretical implementation mechanisms-such as provider knowledge, skills, and attitudes-is not well understood. This study examined the effects of a brief online training and varying dosages of post-training consultation (BOLT+PTC) on implementation mechanisms and outcomes for measurement-based care (MBC) practices delivered in the context of education sector mental health services.

METHODS

A national sample of 75 clinicians who provide mental health interventions to children and adolescents in schools were randomly assigned to BOLT+PTC or control (services as usual). Those in BOLT+PTC were further randomized to 2-, 4-, or 8-week consultation conditions. Self-reported MBC knowledge, skills, attitudes, and use (including standardized assessment, individualized assessment, and assessment-informed treatment modification) were collected for 32 weeks. Multilevel models were used to examine main effects of BOLT+PTC versus control on MBC use at the end of consultation and over time, as well as comparisons among PTC dosage conditions and theorized mechanisms (skills, attitudes, knowledge).

RESULTS

There was a significant linear effect of BOLT+PTC over time on standardized assessment use (b = .02, p < .01), and a significant quadratic effect of BOLT+PTC over time on individualized assessment use (b = .04, p < .001), but no significant effect on treatment modification. BOLT + any level of PTC resulted in higher MBC knowledge and larger growth in MBC skill over the intervention period as compared to control. PTC dosage levels were inconsistently predictive of outcomes, providing no clear evidence for added benefit of higher PTC dosage.

CONCLUSIONS

Online training and consultation in MBC had effects on standardized and individualized assessment use among clinicians as compared to services as usual with no consistent benefit detected for increased consultation dosage. Continued research investigating optimal dosages and mechanisms of these established implementation strategies is needed to ensure training and consultation resources are deployed efficiently to impact clinician practices.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05041517 . Retrospectively registered on 10 September 2021.

Measuring Evidence-Based Treatment Strategies in Youth Community Mental Health Care: the Evidence-Based Strategies Scale

While there are established measures for fidelity to specific evidence-based treatments, there is no widely accepted, feasible measure of the use of evidence-based treatment strategies in youth mental health (MH) care. This study examined the factor structure of a provider self-report measure of evidence-based treatment strategy use, the Evidence-Based Strategies Scale (EBSS). MH providers completed the EBSS as part of a larger mailed survey. The factor structure of the EBSS was examined using exploratory factor analysis in a national, multidisciplinary sample of MH providers (N = 1092), and confirmatory factor analysis was subsequently conducted to replicate this factor structure in a state-wide, multidisciplinary sample of Medicaid MH providers (N = 780). Findings indicated a three-factor structure, representing working alliance, youth-focused, and family-focused evidence-based treatment strategies factors. The EBSS offers a potential method for measuring the evidence-based treatment strategies being delivered in youth community MH care, though more research is needed.

Transitioning to Telehealth for COVID-19 and Beyond: Perspectives of Community Mental Health Clinicians

In response to COVID-19, mental health clinics transitioned to telehealth to maintain psychotherapy delivery. Community mental health (CMH) settings, which are often under-resourced, likely experienced many barriers. This study examined CMH clinicians’ experiences transitioning to telehealth. Data came from a state-funded initiative training CMH clinicians in cognitive behavioral therapy. Participants (N = 197) completed pre-training and post-consultation surveys which included questions about their experiences with telehealth. Most clinicians found telehealth beneficial and effective. Clinicians strongly endorsed wanting telehealth as an option even after in-person services resume. CMH clinicians rated “engaging younger children” as the most significant barrier to telehealth. Despite some telehealth barriers, clinicians generally viewed telehealth favorably and prefer having it as a long-term option. Future work should continue to understand when telehealth may be advantageous and for whom in order to improve the accessibility and quality of behavioral health services.

Implementation of the Wolverine Mental Health Program, Part 2: Implementation Phase

To enhance mental health care for youth in a midwestern residential treatment facility, Wolverine Human Services partnered with the Beck Institute (an intermediary) and an implementation research team to implement cognitive-behavioral therapy (CBT). CBT has strong evidence supporting effectiveness for treating youth internalizing and externalizing problems, but it is a complex psychosocial intervention that demands a thoughtful implementation approach. This study outlines the implementation phase (2.5 years) of a 5-year collaborative effort. The implementation phase focused on (a) adapting CBT to fit the complex youth needs and the roles of the multidisciplinary team members resulting in a new comprehensive and coordinated care model, and (b) the strategies utilized to support its competent integration by all team members. Six blended implementation strategies were deployed in this phase: forging implementation teams, installing progress monitoring, adapting CBT, training, providing supervision and consultation, and training the trainers. A components-based approach to CBT yielded six core skills: active listening, problem solving, mood monitoring and intervention mapping, activity scheduling, distress tolerance, and cognitive restructuring. By the end of this phase, all staff had robust exposure to and experience with the adapted form of CBT. The work of our academic-community partnership has both research and clinical implications, with respect to integrating an adapted version of CBT for residential environments (CBT-RE).

The implementation potential of a method to monitor empirically-supported children's mental health treatment through claims data

Background

The delivery of evidence-supported treatments (EST) in children’s mental health could be a valuable measure for monitoring mental healthcare quality; however, efforts to monitor the use of EST in real world systems are hindered by the lack of pragmatic methods. This mixed methods study examined the implementation and agency response rate of a pragmatic, claims-based measure of EST designed to be applied as a universal quality measure for child psychotherapy encounters in a state Medicaid system.

Methods

Implementation potential of the EST measure was assessed with healthcare leader rankings of the reporting method’s acceptability, appropriateness and feasibility (n = 53), and post-implementation ratings of EST rate accuracy. Ability of the healthcare system to monitor EST through claims was measured by examining the agency responsivity in using the claims-based measure across 98 Medicaid-contracted community mental health (CMH) agencies in Washington State.

Results

The analysis found the reporting method had high implementation potential. The method was able to measure the use of an EST for 83% of children covered by Medicaid with 58% CMH agencies reporting > 0 ESTs in one quarter. Qualitative analyses revealed that the most significant barrier to reporting ESTs was the operability of electronic health record systems and agencies’ mixed views regarding the accuracy and benefits of reporting.

Conclusions

Measurement of child mental health ESTs through Medicaid claims reporting has acceptable implementation potential and promising real world responsiveness from CMH agencies in one state. Variation in reporting by agency site and low to moderate perceived value by agency leaders suggests the need for additional implementation supports for wider uptake.

The impact of implementation climate on community mental health clinicians’ attitudes toward exposure: An evaluation of the effects of training and consultation

Background:

Most evidence-based treatments (EBTs) for posttraumatic stress disorder (PTSD) and anxiety disorders include exposure; however, in community settings, the implementation of exposure lags behind other EBT components. Clinician-level determinants have been consistently implicated as barriers to exposure implementation, but few organizational determinants have been studied. The current study examines an organization-level determinant, implementation climate, and clinician-level determinants, clinician demographic and background factors, as predictors of attitudes toward exposure and changes in attitudes following training.

Method:

Clinicians (n = 197) completed a 3-day training with 6 months of twice-monthly consultation. Clinicians were trained in cognitive behavioral therapy (CBT) for anxiety, depression, behavior problems, and trauma-focused CBT (TF-CBT). Demographic and background information, implementation climate, and attitudes toward exposure were assessed in a pre-training survey; attitudes were reassessed at post-consultation. Implementation climate was measured at the aggregated/group-level and clinician-level.

Results:

Attitudes toward exposure significantly improved from pre-training to post-consultation (t(193) = 9.9, p < .001; d = 0.71). Clinician-level implementation climate scores did not predict more positive attitudes at pre-training (p > .05) but did predict more positive attitudes at post-consultation (ß = −2.46; p < .05) and greater changes in those attitudes (ß = 2.28; p < .05). Group-level implementation climate scores did not predict attitudes at pre-training, post-consultation, or changes in attitudes (all ps > .05). Higher frequency of self-reported CBT use was associated with more positive attitudes at pre-training (ß = −0.81; p < .05), but no other clinician demographic or background determinants were associated with attitudes at post-consultation (all p > .05) or with changes in attitudes (all p > .05).

Conclusions:

Clinician perceptions of implementation climate predicted greater improvement of attitudes toward exposure following EBT training and consultation. Findings suggest that organizational determinants outside of training impact changes in clinicians’ attitudes. Training in four EBTs, only two of which include exposure as a component, resulted in positive changes in clinicians’ attitudes toward exposure, which suggests non-specialty trainings can be effective at changing attitudes, which may enable scale-up.

Exposure is highly effective for treating trauma symptoms and anxiety-based disorders, but it is not commonly used in community mental health settings. Clinicians who endorsed higher expectations, support, and rewards for using exposure in their agency had more positive attitudes toward exposure after training and consultation. Additionally, clinicians who endorsed that exposure is expected, supported, and rewarded in their agency showed a greater improvement in attitudes throughout the training process. Organizational culture can affect clinicians’ attitude changes in the training process, and therefore should become a focus of training efforts.

Harnessing Innovative Technologies to Train Nurses in Suicide Safety Planning With Hospitalized Patients: Protocol for Formative and Pilot Feasibility Research

Background

Suicide is the 10th leading cause of death in the United States, with >47,000 deaths in 2019. Most people who died by suicide had contact with the health care system in the year before their death. Health care provider training is a top research priority identified by the National Action Alliance for Suicide Prevention; however, evidence-based approaches that target skill-building are resource intensive and difficult to implement. Advances in artificial intelligence technology hold promise for improving the scalability and sustainability of training methods, as it is now possible for computers to assess the intervention delivery skills of trainees and provide feedback to guide skill improvements. Much remains to be known about how best to integrate these novel technologies into continuing education for health care providers.

Objective

In Project WISE (Workplace Integrated Support and Education), we aim to develop e-learning training in suicide safety planning, enhanced with novel skill-building technologies that can be integrated into the routine workflow of nurses serving patients hospitalized for medical or surgical reasons or traumatic injury. The research aims include identifying strategies for the implementation and workflow integration of both the training and safety planning with patients, adapting 2 existing technologies to enhance general counseling skills for use in suicide safety planning (a conversational agent and an artificial intelligence–based feedback system), observing training acceptability and nurse engagement with the training components, and assessing the feasibility of recruitment, retention, and collection of longitudinal self-report and electronic health record data for patients identified as at risk of suicide.

Methods

Our developmental research includes qualitative and observational methods to explore the implementation context and technology usability, formative evaluation of the training paradigm, and pilot research to assess the feasibility of conducting a future cluster randomized pragmatic trial. The trial will examine whether patients hospitalized for medical or surgical reasons or traumatic injury who are at risk of suicide have better suicide-related postdischarge outcomes when admitted to a unit with nurses trained using the skill-building technology than those admitted to a unit with untrained nurses. The research takes place at a level 1 trauma center, which is also a safety-net hospital and academic medical center.

Results

Project WISE was funded in July 2019. As of September 2021, we have completed focus groups and usability testing with 27 acute care and 3 acute and intensive care nurses. We began data collection for research aims 3 and 4 in November 2021. All research has been approved by the University of Washington institutional review board.

Conclusions

Project WISE aims to further the national agenda to improve suicide prevention in health care settings by training nurses in suicide prevention with medically hospitalized patients using novel e-learning technologies.

International Registered Report Identifier (IRRID)

DERR1-10.2196/33695

Partnering for Success: Factors Impacting Implementation of a Cross-Systems Collaborative model Between Behavioral Health and Child Welfare

Cross-system implementation efforts can support needed mental health (MH) service utilization among children involved in the child welfare (CW) system. The Partnering for Success (PfS) initiative is one such effort that promotes greater collaboration between the CW and MH providers by building capacity within and across each system. Frontline CW providers learn to accurately identify child MH treatment targets, link families to locally-provided evidence-based treatments (EBTs), and monitor treatment progress. Concurrently, local MH providers are trained along with CW workers to utilize Cognitive Behavioral Therapy plus Trauma-Focused CBT (CBT +), a common elements training and consultation approach focusing on typical MH issues for CW-involved children: Anxiety, Depression, Behavioral Problems, and Traumatic Stress. Finally, agency leadership receive support around promoting implementation and sustainment. This paper examines factors identified by participating CW and MH staff which impacted PfS implementation. Twenty-nine frontline, supervisory, and executive CW and MH providers were interviewed via audio-recorded web-based calls in six focus groups and 10 individual interviews. Factors facilitating implementation success included training/consultation, support from supervisors and agency leadership, improved referral processes, high quality relationships and communication between CW and MH frontline staff, PfS tools and resources, opportunities to use PfS, as well as buy-in from providers and families. Implementation barriers included poor communication between CW and MH providers, conflicts over role expectations, workload and turnover challenges, lack of buy-in, as well as provider (e.g., not aligned with CBT +) and client characteristics (e.g., frequent crises).

Application of Evidence-Based Treatment in Community Mental Health Settings: Examining EBT Delivery Duration and Client Discharge

Characterizing community mental health (CMH) treatment duration and discharge is an important step toward understanding how to better meet client needs. This report describes patterns of treatment duration and discharge among clinicians participating in a state-funded evidence-based treatment (EBT) training initiative. After training and consultation, clinicians (N = 376) reported on treatment duration and discharge for their "most complete case." On average, clinicians delivered 12.4 sessions (SD = 5.1) of the treatment. After completing treatment, half of clinicians (58.7%) continued with regularly scheduled therapy, either using EBT elements or switching to supportive therapy. Clinicians who continued with regularly scheduled therapy delivered treatment in approximately the same number of sessions. Results revealed that CMH clinicians often do not discontinue therapy after completing a treatment protocol. These findings suggest it may be essential to better understand clinician decision-making around applying EBTs to their caseloads.

The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development

Background

Developing pragmatic assessment tools to measure clinician use of evidence-based practices is critical to advancing implementation of evidence-based practices in mental health. This case study details our community-partnered process of developing the Therapy Process Observation Coding Scale-Self-Reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY), a pragmatic, clinician-report instrument to measure cognitive behavioral therapy (CBT) delivery.

Approach

We describe a five-step community-partnered development process. Initial goals were to create a self-report instrument that paralleled an existing direct observation measure of clinician delivery of CBT use to facilitate later assessment of measure performance. Cognitive interviews with community clinicians (n = 6) and consultation with CBT experts (n = 6) were used to enhance interpretability and usability as part of an iterative refinement process. The instrument was administered to 247 community clinicians along with an established self-reported measure of clinician delivery of CBT and other treatments to assess preliminary psychometric performance. Preliminary psychometrics were promising.

Conclusion

Our community-partnered development process showed promising success and can guide future development of pragmatic implementation measures both to facilitate measurement of ongoing implementation efforts and future research aimed at building learning mental health systems.

Plain language summary

Developing brief, user-friendly, and accurate tools to measure how therapists deliver cognitive behavioral therapy (CBT) in routine practice is important for advancing the reach of CBT into community settings. To date, developing such "pragmatic" measures has been difficult. There is little known about how researchers can best develop these types of assessment tools so that they (1) are easy for clinicians in practice to use and (2) provide valid and useful information about implementation outcomes. As a result, there are few well-validated measures in existence that measure therapist use of CBT that are feasible for use in community practice. This paper contributes to the literature by describing our community-partnered process for developing a measure of therapist use of CBT (Therapy Process Observation Coding Scale -Self-Reported Therapist Intervention Fidelity for Youth; TPOCS-SeRTIFY). This descriptive case study outlines the community-partnered approach we took to develop this measure. This case study will contribute to future research by serving as a guide to others aiming to develop pragmatic implementation measures. In addition, the TPOCS-SeRTIFY is a pragmatic measure of clinician use of CBT that holds promise for its use by both researchers and clinicians to measure the success of CBT implementation efforts.

Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes

A lack of effective therapist training is a major barrier to evidence-based intervention (EBI) delivery in the community. Systematic reviews published nearly a decade ago suggested that traditional EBI training leads to higher knowledge but not more EBI use, indicating that more work is needed to optimize EBI training and implementation. This systematic review synthesizes the training literature published since 2010 to evaluate how different training models (workshop, workshop with consultation, online training, train-the-trainer, and intensive training) affect therapists' knowledge, beliefs, and behaviors. Results and limitations for each approach are discussed. Findings show that training has advanced beyond provision of manuals and brief workshops; more intensive training models show promise for changing therapist behavior. However, methodological issues persist, limiting conclusions and pointing to important areas for future research.

Common Elements of Practice, Process and Implementation in Out-of-School-Time Academic Interventions for At-risk Children: a Systematic Review

Academic achievement is a strong preventive factor against marginalization. Children at risk of academic failure and drop out can benefit from out-of-school-time academic (OSTA) interventions. Wide-scaled implementation and sustainment of effective interventions remain a struggle across education, welfare, and health. The need for approaches to increase implementability, effectiveness, and efficiency of interventions is pressing. Advancements in the field of education and mental health suggest identifying and studying discrete elements that are common across interventions for the purpose of hypothesis generation, intervention optimization, design improvement, and implementation. This review identified OSTA interventions for primary school children at risk of academic failure. Common elements methodology was used to code practice elements (n = 62), process elements (n = 49), and implementation elements (n = 36) in 30 effective and six ineffective OSTA interventions in matrices. Based on frequency counts, common practice, process, and implementation elements across the interventions were identified, and given frequency count values (FV) reflecting how often elements were included in effective studies as opposed to in ineffective studies. The five common practice elements with the highest FVs were homework support, training in positive parental school involvement, positive reinforcement, structured tutoring, and psychoeducation. The most common process element was regular support to intervention receiver, and the most common implementation element was quality monitoring. Common combinations of elements were also identified and given FVs. Results from this review can inform efforts to design or optimize OSTA interventions, and inform education, implementation, and practice to improve academic achievement for children at risk.

Implementing Mental Health Services for Children and Adolescents: Caregiver Involvement in School-Based Care

OBJECTIVE

This study compared clinician contact with clients' caregivers by service setting, specifically schools, which are the most common service setting for youths.

METHODS

Data were from a state-funded cognitive-behavioral therapy training initiative. Clinicians (N=177) completed pretraining and postconsultation surveys including retrospective reports of caregiver contact and amount of school-based practice.

RESULTS

School-based clinicians were less likely than non-school-based clinicians to report any contact with caregivers. Full-time school-based clinicians were less likely than part-time school-based clinicians to report any contact with caregivers. School-based clinicians also were less likely than clinicians in other settings to have in-person contact with caregivers, and full-time school-based clinicians were less likely than part-time school-based clinicians to report in-person contact with caregivers.

CONCLUSIONS

Given the inherent advantages of school-based treatment, integration of mental health services for youths in schools is increasingly supported by funding and policy. The findings of this study suggest, however, that investing in strategies to engage caregivers in such treatment may be worthwhile.

Advancing the state-level tracking of evidence-based practices: a case study

Background

Despite a sustained focus by policymakers and researchers on improving the standard of clinical care in public mental health services, the use of evidence-based practice remains low. Among other challenges, this reflects the difficulty of translating clinical research into useable policy that can be feasibly funded and monitored by state or large healthcare systems.

Case presentation

In this paper we present a case study of Washington State’s strategy for monitoring the use of clinical elements at the session level for all Medicaid-funded children’s mental health services. The implementation of this strategy reflects policy actions to promote effective practice while also actively influencing multiple other levels of the implementation ecology. The approach is informed by the Policy Ecology Framework, the Consolidated Framework for Implementation Research, the evidence-based policymaking literature, and common ontology and clinical elements models.

Conclusions

We found the strategy developed in Washington State to be a feasible method of collecting session level information about the use of effective clinical mental health practices. In addition, the approach appears to be having influence on multiple layers of the implementation ecology that could be explored through further study.

Efficient Monitoring of Treatment Response during Youth Psychotherapy: The Behavior and Feelings Survey

An emerging trend in youth psychotherapy is measurement-based care (MBC): treatment guided by frequent measurement of client response, with ongoing feedback to the treating clinician. MBC is especially needed for treatment that addresses internalizing and externalizing problems, which are common among treatment-seeking youths. A very brief measure is needed, for frequent administration, generating both youth- and caregiver-reports, meeting psychometric standards, and available at no cost. We developed such a measure to monitor youth response during psychotherapy for internalizing and externalizing problems. Across 4 studies, we used ethnically diverse, clinically relevant samples of caregivers and youths ages 7-15 to develop and test the Behavior and Feelings Survey (BFS). In Study 1, candidate items identified by outpatient youths and their caregivers were examined via an MTurk survey, with item response theory methods used to eliminate misfitting items. Studies 2-4 used separate clinical samples of youths and their caregivers to finalize the 12-item BFS (6 internalizing and 6 externalizing items), examine its psychometric properties, and assess its performance in monitoring progress during psychotherapy. The BFS showed robust factor structure, internal consistency, test-retest reliability, convergent and discriminant validity in relation to three well-established symptom measures, and slopes of change indicating efficacy in monitoring treatment progress during therapy. The BFS is a brief, free youth- and caregiver-report measure of internalizing and externalizing problems, with psychometric evidence supporting its use for MBC in clinical and research contexts.

Implementation Climate and Time Predict Intensity of Supervision Content Related to Evidence Based Treatment

Objective: Children infrequently receive evidence-based treatments (EBTs) for mental health problems due to a science-to-practice implementation gap. Workplace-based clinical supervision, in which supervisors provide oversight, feedback, and training on clinical practice, may be a method to support EBT implementation. Our prior research suggests that the intensity of supervisory focus on EBT (i.e., thoroughness of coverage) during workplace-based supervision varies. This study explores predictors of supervisory EBT intensity.

Methods: Participants were twenty-eight supervisors and 70 clinician supervisees. They completed a baseline survey, and audio recorded supervision sessions over 1 year. Four hundred and thirty eight recordings were coded for supervision content. We chose to explore predictors of two EBT content elements due to their strong evidence for effectiveness and sufficient variance to permit testing. These included a treatment technique (“exposure”) and a method to structure treatment (“assessment”). We also explored predictors of non-EBT content (“other topics”). Mixed-effects models explored predictors at organizational/supervisor, clinician, and session levels.

Results: Positive implementation climate predicted greater intensity of EBT content coverage for assessment (coefficient = 0.82, p = 0.004) and exposure (coefficient = 0.87, p = 0.001). Intensity of exposure coverage was also predicted by more time spent discussing each case (coefficient = 0.04, p < 0.001). Predictors of greater non-EBT content coverage included longer duration of supervision sessions (coefficient = 0.05, p < 0.001) and lower levels of supervisor EBT knowledge (coefficient = −0.17, p = 0.013). No other supervisor- or clinician-level variables were significant predictors in the mixed effects models.

Conclusion: This was the first study to explore multi-level predictors of objectively coded workplace-based supervision content. Results suggest that organizations that expect, support and reward EBT are more likely to have greater intensity of EBT supervision coverage, which in turn may positively impact clinician EBT fidelity and client outcomes. There was evidence that supervisor knowledge of the EBT contributes to greater coverage, although robust supervisor and clinician factors that drive supervision are yet to be identified. Findings highlight the potential effectiveness of implementation strategies that simultaneously address organizational implementation climate and supervisor practices. More research is needed to identify mechanisms that support integration of EBT into supervision.

Implementing Transdiagnostic Cognitive Behavioral Psychotherapy in Adult Public Behavioral Health: A Pilot Evaluation of the Feasibility of the Common Elements Treatment Approach (CETA)

Few evidence-based psychotherapies are provided in adult public behavioral health (PBH), despite the need for such treatments. The common elements treatment approach (CETA) was developed for use by lay providers in low- and middle-income countries and may have relevance in PBH given its unique application with individuals with multiple diagnoses including PTSD, depression, and anxiety. This study utilized data collected as part of the implementation of CETA in 9 PBH agencies in Washington State with 58 providers, including a 2-day workshop and 6 months of consultation. Outcomes included provider-perceived skill in CETA delivery, training and consultation completion rates, and perceived appropriateness of CETA for clients. Thirty-nine (67%) providers completed requirements for training and consultation, and delivered CETA to a total of 56 clients. Perceived competence in delivering CETA improved over time, as well as client symptom scores. CETA shows promise for feasible and effective implementation within US-based PBH systems.

Supervision in Community Mental Health: Understanding Intensity of EBT Focus

The goal of the present study was to examine clinician, supervisor, and organizational factors that are associated with the intensity of evidence-based treatment (EBT) focus in workplace-based clinical supervision of a specific EBT, Trauma-Focused Cognitive Behavioral Therapy (TF-CBT). Supervisors (n = 56) and clinicians (n = 207) from mental health organizations across Washington State completed online self-report questionnaires. Multilevel modeling (MLM) analyses were used to examine the relative influence of nested clinician and supervisor factors on the intensity of EBT focus in supervision. We found that 33% of the variance in clinician report of EBT supervision intensity clustered at the supervisor level and implementation climate was the only significant factor associated with EBT supervision intensity. While individual clinician and supervisor factors may play a role in EBT coverage in supervision, our results suggest that an implementation climate that supports EBT may be the most critical factor for improving intensity of EBT coverage. Thus, implementation efforts that address the extent to which EBTs are expected, rewarded, and supported within an organization may be needed to support greater coverage of EBT during workplace-based supervision.

The Juggling Act of Supervision in Community Mental Health: Implications for Supporting Evidence-Based Treatment

Supervisors are an underutilized resource for supporting evidence-based treatments (EBTs) in community mental health. Little is known about how EBT-trained supervisors use supervision time. Primary aims were to describe supervision (e.g., modality, frequency), examine functions of individual supervision, and examine factors associated with time allocation to supervision functions. Results from 56 supervisors and 207 clinicians from 25 organizations indicate high prevalence of individual supervision, often alongside group and informal supervision. Individual supervision serves a wide range of functions, with substantial variation at the supervisor-level. Implementation climate was the strongest predictor of time allocation to clinical and EBT-relevant functions.

Objective coding of content and techniques in workplace-based supervision of an EBT in public mental health

BACKGROUND

Workplace-based clinical supervision as an implementation strategy to support evidence-based treatment (EBT) in public mental health has received limited research attention. A commonly provided infrastructure support, it may offer a relatively cost-neutral implementation strategy for organizations. However, research has not objectively examined workplace-based supervision of EBT and specifically how it might differ from EBT supervision provided in efficacy and effectiveness trials.

METHODS

Data come from a descriptive study of supervision in the context of a state-funded EBT implementation effort. Verbal interactions from audio recordings of 438 supervision sessions between 28 supervisors and 70 clinicians from 17 public mental health organizations (in 23 offices) were objectively coded for presence and intensity coverage of 29 supervision strategies (16 content and 13 technique items), duration, and temporal focus. Random effects mixed models estimated proportion of variance in content and techniques attributable to the supervisor and clinician levels.

RESULTS

Interrater reliability among coders was excellent. EBT cases averaged 12.4 min of supervision per session. Intensity of coverage for EBT content varied, with some discussed frequently at medium or high intensity (exposure) and others infrequently discussed or discussed only at low intensity (behavior management; assigning/reviewing client homework). Other than fidelity assessment, supervision techniques common in treatment trials (e.g., reviewing actual practice, behavioral rehearsal) were used rarely or primarily at low intensity. In general, EBT content clustered more at the clinician level; different techniques clustered at either the clinician or supervisor level.

CONCLUSIONS

Workplace-based clinical supervision may be a feasible implementation strategy for supporting EBT implementation, yet it differs from supervision in treatment trials. Time allotted per case is limited, compressing time for EBT coverage. Techniques that involve observation of clinician skills are rarely used. Workplace-based supervision content appears to be tailored to individual clinicians and driven to some degree by the individual supervisor. Our findings point to areas for intervention to enhance the potential of workplace-based supervision for implementation effectiveness.

TRIAL REGISTRATION

NCT01800266 , Clinical Trials, Retrospectively Registered (for this descriptive study; registration prior to any intervention [part of phase II RCT, this manuscript is only phase I descriptive results]).

Behavioral Rehearsal for Analogue Fidelity: Feasibility in a State-Funded Children's Mental Health Initiative

A substantial number of evidence-based treatments (EBTs) are available, but are delivered infrequently in public mental health. To improve the quality of care, some states and systems have focused on EBT training; however, these efforts have rarely included objective measurement of clinician fidelity because of feasibility issues. The primary goal of the current study was evaluating the feasibility of the behavioral rehearsal (BR) method to assess "analogue fidelity" in a children's mental health quality improvement initiative. Results indicated low-but representative-clinician participation. Participants demonstrated greatest improvement at post-training with maintenance or decreases at 6-months (post-consultation). Implications for future use of BR are discussed.

Clinician use of standardized assessments following a common elements psychotherapy training and consultation program

Despite increasing emphasis on the implementation of evidence-based treatments in community service settings, little attention has been paid to supporting the use of evidence-based assessment (EBA) methods and processes, a parallel component of evidence-based practice. Standardized assessment (SA) tools represent a key aspect of EBA and are central to data-driven clinical decision making. The current study evaluated the impact of a statewide training and consultation program in a common elements approach to psychotherapy. Practitioner attitudes toward, skill applying, and use of SA tools across four time points (pre-training, post-training, post-consultation, and follow-up) were assessed. Results indicated early increases in positive SA attitudes, with more gradual increases in self-reported SA skill and use. Implications for supporting the sustained use of SA tools are discussed, including the use of measurement feedback systems, reminders, and SA-supportive supervision practices.