Searching for elements of evidence-based practices in children's usual care and examining their impact

Implementation of Evidence-Based Practices within Treatment-As-Usual and Evidence-Based Practice Initiatives

Publicly funded initiatives are underway to improve implementation of evidence-based practices (EBP) in youth mental health services. However, we know little about the success of these initiatives or about EBP implementation independent of such initiatives. We examined EBP implementation in a treatment as usual (TAU) state and in six states with publicly funded EBP initiatives (EBPIs). In Study 1, we examined providers' use of practices derived from the evidence base (PDEB) and their predictors among 780 providers in a TAU state. In Study 2, we conducted a systematic review of implementation strategies, outcomes, and predictors of EBP use in six state funded EBPIs. Study 1 suggests TAU providers use PDEB alongside practices without consistent research support; provider racial/ethnic minority status, learning theory orientation, and manual use predict greater PDEB use. Study 2 indicates EBPIs employ multiple recommended implementation strategies with variable outcomes across studies and measurement approaches. Predictors of EBP use in EBPIs also varied, though training, setting, and youth age were consistent predictors across studies. While sample differences and inconsistent measurement across studies made direct comparisons somewhat tenuous, rates of PDEB use in the TAU sample appeared similar to those in publicly funded EBPIs. However, two states reported comparisons with TAU samples and found higher EBP implementation under EBPI. Different predictors impacted EBP use in TAU versus EBPIs. Our findings highlight the need for improved evaluation of EBPIs including clear reporting standards for outcomes and more consistent, standardized measurement of EBP use in order to better understand and improve EBPIs.

Mental health needs in a large urban school district: Findings from a web-based survey

OBJECTIVE

To explore the mental health needs of students, and the professional development and support needs of teachers and school health professionals, as a way to foster community engagement and help set priorities for a comprehensive school mental health system in the Detroit Public Schools Community District (DPSCD).

SETTING

The study team surveyed all DPSCD staff in June 2019 and all students in grades 8-12 between October and December 2019.

STUDY DESIGN

A descriptive study based on anonymous, web-based surveys focused on student trauma exposure and mental health symptoms, student mental health resource utilization, staff burnout, and professional development needs.

DATA COLLECTION

All students (grades 8-12) and district staff were eligible to participate; the student survey was made available in six languages. Parents/guardians could opt children out; schools could exclude children unable to complete the survey independently. Student surveys were administered in school; staff surveys were sent via email.

PRINCIPAL FINDINGS

Thirty-four percent of DPSCD students reported moderate/severe depression symptoms; 22% had seriously considered suicide in the past year. Rates exceed national averages; 37% of students with severe depression and 34% of those with suicidal ideation had not accessed mental health supports. Staff indicated high levels of burnout and substantial interest in learning about self-care strategies or coping with vicarious trauma. Over 75% of teachers and school mental health professionals expressed interest in learning about best practices for supporting students impacted by trauma or mental illness.

CONCLUSIONS

A large number of DPSCD students are experiencing symptoms of depression and anxiety, and many students who need care are not accessing it. Addressing the mental health needs of students is a high priority for staff, but they need more training and support, as well as support for their own vicarious trauma and high levels of burnout.

Can a Single Measure Estimate Protocol Adherence for Two Psychosocial Treatments for Youth Anxiety Delivered in Community Mental Health Settings?

Treatment adherence measurement can be time and resource-intensive in clinical trials, so the ability to measure protocol adherence for two distinct treatment programs with a single measure may benefit the field. The present study sought to determine if the Therapy Process Observational Coding System - Revised Strategies Scale (TPOCS-RS) could assess protocol adherence to two youth treatment programs. Treatment sessions (N = 796) from 55 youth (M age = 9.89 years, SD = 1.71; range 7-15 years; 55.0% White; 46.0% female) with primary anxiety problems treatment by 39 clinicians (M age = 40.54 years, SD = 9.56; 50.0% White; 80.0% female) were independently scored by coders using observational treatment adherence and alliance measures. The youth received one of three treatments: (a) Standard (i.e., cognitive-behavioral treatment program), (b) Modular (i.e., a program with cognitive-behavioral and parent training components), or (c) Usual Care. Consultants filled out a self-report measure of protocol adherence within the Standard and Modular conditions. Interrater reliability, ICC(2,2) for the various items for the full sample ranged from .17 to .92 (M ICC = .67; SD = .17). Scores from a TPOCS-RS subscale that mapped onto the specific content of the treatment protocols used in the Standard and Modular conditions evidenced convergent validity with the consultant-report adherence measure and discriminant validity with the alliance measure. The model-specific TPOCS-RS subscales also discriminated between the Standard and Modular treatments and Usual Care. This study provides initial evidence that (a) the TPOCS-RS has utility in estimating protocol adherence in different treatment programs and (b) support the score validity of the self-report consultation records.

A Second and Third Look at FIRST: Testing Adaptations of A Principle-Guided Youth Psychotherapy

Objective: We examined the acceptability, integrity, and symptom trajectories associated with FIRST, a principle-guided treatment for youth internalizing and externalizing problems designed to support efficient uptake and implementation.Method: We conducted two open trials of an adapted FIRST, focusing on uptake and implementation by novice trainees in a university-affiliated clinic, limiting treatment duration to six sessions, and benchmarking findings against a 2017 FIRST trial with community therapists. In Study 1, trainees received a two-day training and weekly two-hour supervision (N = 22 youths, ages 7-17, 50% female, 54.54% Caucasian, 4.55% Latinx). In Study 2, trainees received a one-day training and weekly one-hour supervision, delivering the six-session FIRST in a predetermined sequence (N = 26 youths, ages 11-17, 42.31% female, 65.38% Caucasian, 7.69% Latinx). In Study 3, the original study therapists - now practitioners - evaluated FIRST's effectiveness and implementation difficulty, and reported their own post-study FIRST use.Results: Acceptability (treatment completion, session attendance, caregiver participation) and integrity (adherence, competence) were comparable across Study 1, Study 2 and the 2017 trial. Improvement effect sizes across ten outcome measures were in the large range in all three trials: M ES = 1.10 in the 2017 trial, 0.83 in Study 1, and 0.81 in Study 2. Study 3 showed high effectiveness ratings, low difficulty ratings, and continued use of FIRST by a majority of clinicians.Conclusions: Across two open trials and a follow-up survey, FIRST showed evidence of acceptability and integrity, with youth symptom reduction comparable to that in prior research.

Evidence-Based Treatments in Community Mental Health Settings: Use and Congruence With Children's Primary Diagnosis and Comorbidity

Many evidence-based treatments (EBTs) have been identified for specific child mental health disorders, but there is limited research on the use of EBTs in community-based settings. This study used administrative data from a statewide system of care to examine 1) the extent to which EBTs were provided congruent with the child's primary diagnosis, 2) whether there were differences in effectiveness of EBTs that were congruent or incongruent with the child's primary diagnosis, and 3) whether comorbidity moderated the effectiveness of EBTs for children based on congruence with their primary diagnosis. The sample consisted of 23,895 children ages 3-17 with at least one of the most common diagnoses (attention-deficit/hyperactivity disorder, conduct problems, depressive disorders, anxiety disorders, and post-traumatic stress disorder) who received outpatient psychotherapy. Data were collected as part of routine care, including child demographic characteristics, diagnosis, treatment type, and problem severity. Forty-two percent of children received an EBT congruent with their diagnosis, and these children showed greater improvement than the 35% of children who received no EBT (ES = 0.14-0.16) or the 23% who received an EBT incongruent with their diagnosis (ES = 0.06-0.15). For children with comorbid diagnoses, the use of EBTs congruent with the primary diagnosis was also associated with the greatest improvement, especially when compared to no EBT (ES = 0.22-0.24). Results of the current study support the use of EBTs in community-based settings, and suggest that clinicians should select EBTs that match the child's primary diagnosis to optimize treatment outcomes, especially for children with comorbidity.

Practitioners' Use and Evaluation of Transdiagnostic Youth Psychotherapy Years After Training and Consultation Have Ended

We examined practitioners' use of the transdiagnostic Modular Approach to Therapy for Children (MATCH) 7 years after learning MATCH for a clinical trial. The practitioners (N = 29; M_age = 52.10, SD = 12.29, 86% women, 97% white) reported using MATCH with 55% of their caseload; use of the various MATCH modules ranged from 39 to 70%. Use was positively associated with amount of MATCH experience in the trial, perceived effectiveness, and ease of implementation. Patterns of specific module use did not consistently match strength of prior evidence (e.g., exposure was least used of the anxiety modules), suggesting challenges for implementation science.

Pediatric Consultation-Liaison: Patient Characteristics and Considerations for Training in Evidence-Based Practices

Consultation-liaison services are an integral part of many pediatric hospital settings, yet characteristics of this patient population have not been extensively documented. The current study is a retrospective one-year chart review of the consultation-liaison service at a large pediatric hospital in the Southwestern United States. The purpose of this study is twofold: (1) to characterize this hospital's CL population and (2) to use these characteristics to identify preliminary evidence-based practices that should be considered for CL provider training. Identifying evidence-based practice elements that align with the characteristics of consultation-liaison patient populations may inform trainings for consultation-liaison staff. This would help to ensure that youth seen in hospital consultation-liaison services are getting the best available services, which is critical given the shortened time frame available to work with this patient population.

The User-Program Interaction: How Teacher Experience Shapes the Relationship Between Intervention Packaging and Fidelity to a State-Adopted Health Curriculum

Intervention effects observed in efficacy trials are rarely seen when programs are broadly disseminated, underscoring the need to better understand factors influencing fidelity. The Michigan Model for Health™ (MMH) is an evidence-based health curriculum disseminated in schools throughout Michigan that is widely adopted but delivered with limited fidelity. Understanding implementation determinants and how they influence fidelity is essential to achieving desired implementation and behavioral outcomes. The study surveyed health teachers throughout Michigan (n = 171) on MMH implementation, guided by the Consolidated Framework for Implementation Research. We investigated relationships between context, intervention and provider factors and dose delivered (i.e., the proportion of curriculum delivered by teachers), a fidelity dimension. We also examined whether intervention factors were moderated by provider factors to influence fidelity. Our results indicated that program packaging ratings were associated with dose delivered (fidelity). We also found that this relationship was moderated by teacher experience. The strength of this relationship diminished with increasing levels of experience, with no relationship among the most experienced teachers. Intervention adaptability was also associated with dose delivered. We found no association between health education policies (context), provider beliefs, and dose delivered. Intervention factors are important determinants of fidelity. Our results suggest that providers with more experience may need materials tailored to their knowledge and skill level to support materials' continued usefulness and fidelity long-term. Our results also suggest that promoting adaptability may help enhance fidelity. Implementation strategies that focus on systematically adapting evidence-based health programs may be well suited to enhancing the fidelity of the MMH curriculum across levels of teacher experience.

State Strategies for Enhancing Access and Quality in Systems of Care for Youth with Complex Behavioral Health Needs

The investigators conducted a qualitative study with state administrators and policymakers about the financing and policies that structure public systems of care for children with complex behavioral healthcare needs. The objective was to characterize diverse strategies states employed to enhance funding for, access to, and quality and effectiveness of home and community-based services (HCBS) for these youth. States report using a wide variety of creative solutions while navigating the unique needs and constraints of their political and economic environments. Findings can benefit public officials, researchers, and advocates by advancing knowledge-sharing of public policy-making and resourceful problem-solving.

Examining relationships between child skills and potential key components of an evidence-based practice in ASD

BACKGROUND

Research in autism spectrum disorders (ASD) has identified a need to understand key components of complex evidence-based practices (EBP). One approach involves examining the relationship between component use and child behavior.

AIMS

This study provides initial evidence for identifying key components in a specific EBP, Pivotal Response Training (PRT). We examined which components were related to child response and evaluated relationships between provider characteristics, child characteristics and component intensity.

METHODS

Trained coders reviewed archival videos (n = 278) for PRT fidelity and child behavior. We completed multi-level regression and latent profile analysis to examine relationships between intensity of individual or combinations of PRT components and child behavior, and moderators of component use.

RESULTS

Analyses indicated differential relationships between specific components and child behaviors which may support methods of altering intensity of components to individualize intervention. Profile analysis suggested relatively intensive use of most PRT components, especially antecedent strategies, may maximize child responsivity. Providers with postgraduate education trended toward higher intensity component use. Child characteristics did not moderate use.

IMPLICATIONS

Careful examination of key components of ASD interventions may helps clarify the mechanisms of action. Recommendations specific to PRT implementation and use of the methodology for other interventions are discussed.

Clinical Supervision of Mental Health Professionals Serving Youth: Format and Microskills

Clinical supervision is an element of quality assurance in routine mental health care settings serving children; however, there is limited scientific evaluation of its components. This study examines the format and microskills of routine supervision. Supervisors (n = 13) and supervisees (n = 20) reported on 100 supervision sessions, and trained coders completed observational coding on a subset of recorded sessions (n = 57). Results indicate that microskills shown to enhance supervisee competency in effectiveness trials and experiments were largely absent from routine supervision, highlighting potential missed opportunities to impart knowledge to therapists. Findings suggest areas for quality improvement within routine care settings.

Enrolment of children in psychosocial care: problems upon entry, care received, and outcomes achieved

Psychosocial care systems have been designed so that specific problems are treated by specific care types. There is insufficient evidence as to which problem types are actually presented to the various care types. This study assessed types and severity of problems among children and adolescents upon enrolment in psychosocial care, compared to children not enrolled; also outcomes after 3 and 12 months, overall and per care type. We obtained data on a cohort of 1382 Dutch children aged 4-18 years (response rate 56.6%), included upon enrolment in psychosocial care, and on 443 not-enrolled children (response rate 70.3%), all from one region. Results showed that enrolled children had more problems than children not enrolled in care. In child and adolescent mental healthcare (CAMH), relatively many children had internalizing problems, and in child and adolescent social care (CASC) relatively many children had externalizing, parenting, family and multiple problems. Regardless of the type of problem, care duration in preventive child healthcare (PCH) was relatively short; and in CASC and CAMH longer. After 3 and 12 months, rates of problem solution were highest in PCH. These rates were also substantial among children not in care. To conclude, our findings show that the system of psychosocial care functions as intended regarding the distribution of problems across care types. Extended demarcation of clients by problem type and severity towards type and contents of care may further improve the system.

Does the Delivery of CBT for Youth Anxiety Differ Across Research and Practice Settings?

Does delivery of the same manual-based individual cognitive-behavioral treatment (ICBT) program for youth anxiety differ across research and practice settings? We examined this question in a sample of 89 youths (M age = 10.56, SD = 1.99; 63.70% Caucasian; 52.80% male) diagnosed with a primary anxiety disorder. The youths received (a) ICBT in a research setting, (b) ICBT in practice settings, or (c) non-manual-based usual care (UC) in practice settings. Treatment delivery was assessed using four theory-based subscales (Cognitive-behavioral, Psychodynamic, Client-Centered, Family) from the Therapy Process Observational Coding System for Child Psychotherapy-Revised Strategies scale (TPOCS-RS). Reliable independent coders, using the TPOCS-RS, rated 954 treatment sessions from two randomized controlled trials (1 efficacy and 1 effectiveness trial). In both settings, therapists trained and supervised in ICBT delivered comparable levels of cognitive-behavioral interventions at the beginning of treatment. However, therapists trained in ICBT in the research setting increased their use of cognitive-behavioral interventions as treatment progressed whereas their practice setting counterparts waned over time. Relative to the two ICBT groups, the UC therapists delivered a significantly higher dose of psychodynamic and family interventions and a significantly lower dose of cognitive-behavioral interventions. Overall, results indicate that there were more similarities than differences in manual-based ICBT delivery across research and practice settings. Future research should explore why the delivery of cognitive-behavioral interventions in the ICBT program changed over time and across settings, and whether the answers to these questions could inform implementation of ICBT programs.

Considering harm and safety in youth mental health: a call for attention and action

The possibility of harm from mental health provision, and in particular harm from youth mental health provision, has been largely overlooked. We contend that if we continue to assume youth mental health services can do no harm, and all that is needed is more services, we continue to risk the possibility that the safety of children and young people is unintentionally compromised. We propose a three level framework for considering harm from youth mental health provision (1. ineffective engagement, 2. ineffective practice and 3. adverse events) and suggest how this framework could be used to support quality improvement in services.

Implementation and Evaluation of Evidence-Based Psychotherapeutic Practices for Youth in a Mental Health Organization

The current initiative and program evaluation study is a demonstration of the research to practice process in youth-focused psychotherapy. We collaborated within a community-university partnership to create practice and research infrastructure in order to develop, implement, and evaluate two new models of service founded on evidence-based psychotherapeutic practice parameters. The two new service models incorporated validated interventions to address behavior problems in elementary age children, and depression in adolescents, which were delivered in separate but similarly run intensive outpatient programs within a mental health setting. We utilized a rigorous training, technical assistance, fidelity monitoring, and outcome measurement strategy to promote the integrity and quality of services provided. The resultant programs were delivered with acceptable to high fidelity and effects on youth and parenting measures collected during program and from pre to post showed a decrease in targeted problems in youth and positive benefits for families. This initiative and program evaluation adds to the accumulating research-to-practice literature in children's mental health.

User-Centered Design for Psychosocial Intervention Development and Implementation

The current paper articulates how common difficulties encountered when attempting to implement or scale-up evidence-based treatments are exacerbated by fundamental design problems, which may be addressed by a set of principles and methods drawn from the contemporary field of user-centered design. User-centered design is an approach to product development that grounds the process in information collected about the individuals and settings where products will ultimately be used. To demonstrate the utility of this perspective, we present four design concepts and methods: (a) clear identification of end users and their needs, (b) prototyping/rapid iteration, (c) simplifying existing intervention parameters/procedures, and (d) exploiting natural constraints. We conclude with a brief design-focused research agenda for the developers and implementers of evidence-based treatments.

Child, caregiver, and therapist perspectives on therapeutic alliance in usual care child psychotherapy

This study examined the temporal stability and cross-informant agreement on multiple perspectives of child and caregiver alliance with therapists in usual care psychotherapy. Baseline predictors of alliance were also examined. Children with disruptive behavior problems (n = 209) and their caregivers were followed for up to 16 months after initiating psychotherapy at a community-based clinic. Alliance was rated by children, caregivers, and therapists every 4 months for as long as families participated in treatment. Repeated-measures analyses using linear mixed models with random intercepts were conducted to determine whether child and caregiver alliance differed across time, as well to examine factors associated with each perspective on alliance. Intraclass correlations between child, caregiver, and therapist reports of alliance were also examined. Alliance was rated relatively high overall across perspectives. Clients (children and caregivers) tended to report the strongest and most stable alliance, while therapists reported the weakest alliance and perceived deteriorations in child alliance over time. Inter-informant agreement was variable for child and caregiver alliance; agreement was moderate between clients and therapists. Several predictors of alliance emerged, including child gender, anxiety diagnosis, caregiver race/ethnicity, and therapist experience. This study provides methodological information about reports of therapeutic alliance across time and informants that can inform current efforts to understand the alliance-outcome association.

Improving community-based mental health care for children: translating knowledge into action

There is urgent need for improvement in community-based mental health care for children and families. Multiple studies have documented serious limitations in the effectiveness of "usual care." Fortunately, many empirically-supported strategies to improve care have been developed, and thus there is now a great deal of knowledge available to address this significant public health problem. The goal of this selective review is to highlight and synthesize that empirically-supported knowledge to stimulate and facilitate the needed translation of knowledge into action. The review provides a sound foundation for constructing improved services by consolidating descriptive data on the status quo in children's mental health care, as well as evidence for an array of promising strategies to improve (a) Service access and engagement; (b) Delivery of evidence-based practices; and (c) Outcome accountability. A multi-level framework is used to highlight recommended care improvement targets.