Applying Treatment Outcome Research in Clinical Practice: Techniques for Adapting Interventions to the Real World

Identifying Beneficial Training Elements: Clinician Perceptions of Learning the Evidence-Based GenerationPMTO Intervention

Improving the process through which mental health professionals are trained in evidence-based practices (EBPs) represents an important opportunity for extending the implementation of EBPs in community settings. In this study, we used a qualitative approach to examine the specific training elements that were beneficial to clinicians' experiences learning an evidence-based intervention. Individual, semi-structured interviews were conducted with mental health professionals completing training in the GenerationPMTO parenting intervention. Data were analyzed using the tenets of thematic analysis. Overall, participants reported positive experiences in the training and growth in their attitudes, knowledge, and confidence in GenerationPMTO. The qualitative findings also suggested seven specific training elements that participants perceived as beneficial: support, role plays, engagement, structure, writing/visuals, working with training families, and experiencing the GenPMTO model. These results are discussed within the context of the existing literature on EBP training and more broadly as they relate to expanding the implementation of evidence-based interventions. We also suggest implications for practice meant to enhance future EBP training efforts.

Effects of a modular teacher coaching program on child attention problems and disruptive behavior and on teachers’ self-efficacy and stress

Symptoms of attention-deficit/hyperactivity and oppositional defiant disorder are associated with a multitude of psychosocial developmental risks, e.g. academic underachievement. Various cognitive behavioral interventions have proven to be effective in reducing problem behavior in school settings. Drawing on this previous work and on our parent-focused preventive and therapeutic programs, we developed the school-based coaching for elementary school teachers of children with attention deficits or disruptive behavior problems (SCEP). Based on functional behavior assessment, SCEP addresses teachers of children with severe externalizing behavior problems in an individualized modular manner. It consists of a one-day training course and fortnightly one-to-one or team-coaching sessions. We analyzed the effects of SCEP in a within-subject control group design ( N = 60), with student attention problems and rule-breaking behavior during class as the primary outcome measure. SCEP was found to reduce problem behavior during lessons, with small to medium effect sizes ( d = 0.42–0.6). After the intervention, teachers reported changes in their use of praise and felt more confident managing the class ( d = 0.58). The results of SCEP are discussed in light of multi-tiered preventive approaches that suggest extensive individualized interventions based on functional behavior analysis for children with severe problem behavior.

Evidence Base Update for Brief, Free, and Accessible Youth Mental Health Measures

Evidence-based assessment (EBA) is foundational to high-quality mental health care for youth and is a critical component of evidence-based practice delivery, yet is underused in the community. Administration time and measure cost are barriers to use; thus, identifying and disseminating brief, free, and accessible measures are critical. This Evidence Base Update evaluates the empirical literature for brief, free, and accessible measures with psychometric support to inform research and practice with youth. A systematic review using PubMed and PsycINFO identified measures in the following domains: overall mental health, anxiety, depression, disruptive behavior, traumatic stress, disordered eating, suicidality, bipolar/mania, psychosis, and substance use. To be eligible for inclusion, measures needed to be brief (50 items or less), free, accessible, and have psychometric support for their use with youth. Eligible measures were evaluated using adapted criteria established by De Los Reyes and Langer (2018) and were classified as having excellent, good, or adequate psychometric properties. A total of 672 measures were identified; 95 (14%) met inclusion criteria. Of those, 21 (22%) were "excellent," 34 (36%) were "good," and 40 (42%) were "adequate." Few measures had support for their use to routinely monitor progress in therapy. Few measures with excellent psychometric support were identified for disordered eating, suicidality, psychosis, and substance use. Future research should evaluate existing measures for use with routine progress monitoring and ease of implementation in community settings. Measure development is needed for disordered eating, suicidality, psychosis, and substance use to increase availability of brief, free, accessible, and validated measures.

A Case Study of Parent–Child Interaction Therapy

The authors present a case study of “Katie,” a 4-year-old girl diagnosed with oppositional defiant disorder (ODD). Treatment was conducted with Katie and her family using Parent–Child Interaction Therapy (PCIT). However, client-centered adaptations were made to improve the feasibility of the treatment and its ecological validity in a community setting. Katie demonstrated marked reduction in ODD symptoms during treatment and no longer met criteria for ODD at discharge and throughout follow-up periods. A hybrid model was utilized whereby PCIT components were delivered in both clinic and in-home settings. Client-centered adaptations and the benefits of treatment in the in-home setting are discussed. The authors contend that use of appropriate client-centered clinical flexibility, when implementing a manualized, empirically supported, and evidence-based treatment, can assist in bridging the “science–practice gap” allowing for appropriate flexibility and individualization, while also promoting the use of empirically supported and validated treatment approaches.

Do evidence-based interventions work when tested in the "real world?" A systematic review and meta-analysis of parent management training for the treatment of child disruptive behavior

Evidence-based interventions are often unavailable in everyday clinical settings. This may partly reflect practitioners' assumptions that research evidence does not reflect "real-world" conditions. To examine this further, we systematically assessed the clinical effectiveness of parent management training (PMT) for the treatment of child disruptive behavior across different real-world practice contexts. We identified 28 relevant randomized controlled trials from a systematic search of electronic bibliographic databases and conducted a meta-analysis of child outcomes across trials. Planned subgroup analyses involved comparisons between studies grouped according to individual real-world practice criteria and total real-world practice criteria scores, reflecting the extent to which PMT was delivered by non-specialist therapists, to a clinic-referred population, in a routine setting, and as part of a routine service. Meta-analysis revealed a significant overall advantage for PMT compared with waitlist control conditions. Subgroup analyses did not demonstrate significant differences in effect size estimates according to the total number of real-world practice criteria met by studies. Moreover, no consistent relationships were found between specific practice criteria and effect size estimates. In conclusion, PMT appears to be an effective treatment for children with disruptive behavior problems. There was no clear evidence that conducting PMT in real-world practice contexts is a deterrent to achieving effective child behavior outcomes, although relative advantage to "usual care" was not directly examined and the power of the analysis was limited as a result of significant heterogeneity. More research is needed to investigate whether this finding is generalizable to other psychological interventions. Suggestions are also made for developing more differentiated criteria to assist with evaluating the specific applicability of research evidence to different care providers.

A clinical trial of in-home CBT for depressed mothers in home visitation

Despite negative outcomes for depressed mothers and their children, no treatment specifically designed to address maternal depression in the context of home visitation has emerged. In-Home Cognitive Behavioral Therapy (IH-CBT) is an adapted treatment that is delivered in the home, focuses on the needs of new mothers, and leverages ongoing home visiting to optimize engagement and outcomes. This study examined the efficacy of IH-CBT using a randomized clinical trial. Subjects were 93 new mothers in a home visiting program. Mothers with major depressive disorder identified at 3months postpartum were randomized into IH-CBT and ongoing home visitation (n=47) or standard home visitation (SHV; n=46) in which they received home visitation alone and could obtain treatment in the community. Depression was measured at pre- and posttreatment, and 3-month follow-up using interviews, clinician ratings, and self-report. Mothers receiving IH-CBT showed improvements in all indicators of depression relative to the SHV condition and these gains were maintained at follow-up. For example, 70.7% of mothers receiving IH-CBT were no longer depressed at posttreatment in terms of meeting criteria for major depressive disorder compared to 30.2% in the SHV group. These findings suggest that IH-CBT is an efficacious treatment for depressed mothers in home visitation programs.

Beyond context to the skyline: thinking in 3D

Sweeping and profound structural, regulatory, and fiscal changes are rapidly reshaping the contours of health and mental health practice. The community-based practice contexts described in the excellent review by Garland and colleagues are being fundamentally altered with different business models, regional networks, accountability standards, and incentive structures. If community-based mental health services are to remain viable, the two-dimensional and flat research and practice paradigm has to be replaced with three-dimensional thinking. Failure to take seriously the changes that are happening to the larger healthcare context and respond actively through significant system redesign will lead to the demise of specialty mental health services.

The Summer Treatment Program Meets the South Side of Chicago: Bridging Science and Service in Urban After-School Programs

BACKGROUND: This paper describes efforts to apply the principles and strategies of an empirically-supported treatment for children with disruptive behaviour problems to a park after-school program serving children in urban poverty. METHOD: Collaboration with staff proceeded in stages: (1) relationship building, needs assessment, and resource mapping; (2) intervention adaptation and implementation; and (3) implementation support, problem-solving, and sustainability. RESULTS: Four tools capitalised on inherent strengths of the parks, accommodated child and staff needs, and emerged as feasible and effective: Group Discussion, Good Behaviour Game, Peers as Leaders, and Good News Notes. CONCLUSIONS: Recreational settings offer opportunities for mental health promotion for children in urban poverty.

The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations

Evidence-based treatments (EBT) are underutilized in community settings, where consumers are often seen for treatment. Underutilization of EBTs may be related to a lack of empirically informed and supported training strategies. The goals of this review are to understand the state of the literature for training therapists in psychotherapy skills and to offer recommendations to improve research in this area. Results of this review of 55 studies evaluating six training methods indicate that multi-component trainings have been studied most often and have most consistently demonstrated positive training outcomes relative to other training methods. Studies evaluating utility of reading, self-directed trainings, and workshops have documented that these methods do not routinely produce positive outcomes. Workshop follow-ups help to sustain outcomes. Little is known about the impact of train-the-trainer methods. Methodological flaws and factors that may influence training outcome and future directions are also reviewed.

Predicting utilization of evidence-based parenting interventions with organizational, service-provider and client variables

Multidisciplinary service providers (N = 611) who underwent training in the Triple P-Positive Parenting Program participated in a structured interview 6 months following training to determine their level of post-training program use and to identify any facilitators and barriers to program use. Findings revealed that practitioners who had received training in Group Triple P, received positive client feedback, had experienced only minor barriers to implementation, and had consulted with other Triple P practitioners following training were more likely to become high users of the program. Practitioners were less likely to use the program when they had lower levels of confidence in delivering Triple P and in consulting with parents in general, had difficulties in incorporating Triple P into their work, and where there was low workplace support. These findings highlight the importance of considering the broader post training work environment of service providers as a determinant of subsequent program use.

Cognitive-behavioral therapy for obsessive-compulsive disorder: a non-randomized comparison of intensive and weekly approaches

This study examined the relative efficacy of intensive versus weekly cognitive-behavioral therapy (CBT) for adults with obsessive-compulsive disorder (OCD). Sixty-two adults with OCD received either 14 sessions of weekly (n=30) or intensive CBT (n=32; daily psychotherapy sessions) in a non-randomized format. Assessments were conducted at Pre-treatment, Post-treatment, and 3-month Follow-up by raters who were blind to treatment group at the Pre-treatment assessment. Intensive and weekly CBT were similar in efficacy at Post-treatment and Follow-up and associated with large treatment effect sizes. Since many people with OCD do not have access to trained CBT providers, intensive treatment may be a viable option in such cases.

Implementation of evidence-based services for youth: assessing provider knowledge

Although provider knowledge is a potential barrier in the dissemination of evidence-based services for youth, research in this area is currently limited by a lack of instrumentation. The present study examined the utility of the Knowledge of Evidence-Based Services Questionnaire (KEBSQ), a 40-item self-report measure designed to assess reporter knowledge of evidence-based practices (EBPs) in the treatment of youth psychopathology. The KEBSQ items encompass practice elements identified in both empirically supported and unsupported protocols used in the treatment of four prevalent childhood problem areas: anxious/avoidant, depressed/withdrawn, disruptive behavior, and attention/hyperactivity. Findings from the present investigation lend support for the basic psychometric properties of the KEBSQ. Results supported temporal stability, discriminative validity, and sensitivity to training. Practical implications to the dissemination of EBPs, areas for future research, and limitations are discussed.

Are Depressed Adolescents Routinely Offered CBT? A Brief Review of Current Practice

An increasing body of research in support of cognitive-behavioural therapy (CBT) for adolescent depression has emerged during the last two decades. However, it has been suggested that empirically supported treatments are seldom carried out in clinical practice. Although the reasons for this are likely to be diverse, it is argued that mental health services have an ethical responsibility to offer evidence-based interventions. Whether empirically supported interventions, such as CBT, are consistently offered to depressed adolescents attending Child and Adolescent Mental Health Services (CAMHS) is currently unknown. A primary aim of this study was to survey the use of CBT for depression in a number of United Kingdom (UK) CAMHS settings. A postal questionnaire was sent to 117 members of the BABCP Children, Adolescents and Families Special Interest Branch, of which 44 completed questionnaires were returned. Descriptive statistics indicate that just over half of the organizations represented routinely offered CBT to depressed adolescents. CBT practice and the transportation of evidence-based research findings to CAMHS settings are discussed.

The flexible application of a manualized treatment for PTSD symptoms and functional impairment related to the 9/11 World Trade Center attack

The purpose of this treatment effectiveness study was to evaluate the flexible application of a manualized cognitive behavioral treatment (CBT) for PTSD and related symptoms in survivors of the 9/11 terrorist attack on the World Trade Center. Treatment delivery ranged from 12 to 25 sessions; therapist experience ranged from no prior training to extensive training in CBT; and training and supervision of clinicians in the treatment manual was considerably less than that required in a randomized clinical trial (RCT). Paired t-tests demonstrated significant pre-post reductions in symptoms of PTSD and depression for the flexible application of the treatment. A benchmarking analysis revealed that the moderate-to-large effect sizes found for these variables were similar to those obtained in an RCT of the same treatment. Furthermore, effect sizes on measures of outcomes particularly relevant to this population of mass violence survivors such as functional impairment, use of alcohol and drugs to cope, and use of social support to cope, were also medium to large.

Developing evidence-based interventions for deployment into school settings: A case example highlighting key issues of efficacy and effectiveness

This paper describes the development of an after-school program for middle-school students that simultaneously addressed issues of efficacy and effectiveness in an effort to create an evidence-based intervention (EBI) that can be implemented in school settings. The topics highlighted in this case example are intended to address the growing concern that over-focusing on efficacy in well-controlled, grant-funded studies has resulted in a generation of EBI that are not acceptable, feasible, or sustainable in the majority of applied settings. Our case example focuses on six key issues highlighted in the published research literature: sustainability, recruitment, methodological design, flexibility, training, and meaningful outcomes. We briefly summarize each of these issues and relate them to our experiences conducting a successful pilot study. A summary of the pilot study results is also presented to lend support for the utilization of the EBI model of dissemination described in this paper.

Practitioner attitudes toward evidence-based practice: themes and challenges

The widespread successful implementation of evidence-based practices (EBPs) into community mental health settings will require a thorough understanding of practitioner attitudes toward these approaches. This study reports on the results of two community mental health practitioner focus groups investigating attitudes toward EBPs, perceived challenges to implementing EBPs, and recommendations for researchers interested in facilitating EBP use in community settings. The participants were child and adolescent mental health professionals (N=19) from two community mental health centers. The focus groups were taped, transcribed and coded for themes. Major themes included concerns regarding the applicability of some research supporting EBPs, a desire for a greater emphasis on the therapeutic relationship, and the need for flexibility within treatment protocols. Themes are discussed within the context of the recent movement toward implementing EBPs in community settings and recent research related to focus group themes.

Evidence-Based Assessment of Child and Adolescent Disorders: Issues and Challenges

The main purpose of this article and this special section is to encourage greater attention to evidence-based assessment (EBA) in the development of a scientifically supported clinical child and adolescent psychology. This increased attention is especially important in light of (a) the omission of assessment considerations in recent efforts to promote evidence-based treatments for children and (b) ongoing changes in the nature of clinical child assessment. We discuss several key considerations in the development of guidelines for EBA, including the purposes of assessment, the role of disorder or problem specificity, the scope of assessment, assessment process parameters, possible "cross-cutting" assessment issues, psychometric considerations, and issues related to the clinician's integration of assessment data. We conclude the article with suggestions for how current, summary information on EBA can be developed, maintained, and disseminated.

A theoretical model of common process factors in youth and family therapy

Recently there has been an increasing emphasis in the youth and family mental health treatment literature on the use of empirically supported treatments (ESTs). In contrast there has been scant attention paid to more universal aspects of the therapy process that may have even greater impact upon therapy outcomes. It is likely that the success of the techniques proposed by ESTs may depend on the presence of common process factors. In this article, the authors explore the status of common process factors research in the youth and family therapy literature, and propose a theoretical model linking specific therapeutic relationship variables and treatment outcomes for children and adolescents. This model is intended to guide synthesis of the empirical evidence for common process factors in youth and family treatment and to stimulate future research on common process factors.

Effectiveness Research in the Context of School-Based Mental Health

There are many challenges to transporting evidence-based treatments from laboratories into real-world settings. However, if we hope to make our evidence-based treatments accessible and available to children and usable by community professionals, effectiveness research is imperative. We argue that schools represent an ideal real world setting in which to conduct such research. The goals of this paper are to present the advantages of conducting effectiveness research in the school setting and to encourage clinical researchers to engage in effectiveness research in this "location," as there are benefits to children, pre-service professionals, communities, and researchers. We attempt to further advance the literature by discussing the challenges associated with this work and by providing a case example (The Youth Experiencing Success in School [Y.E.S.S.] Program) that demonstrates real world application of our recommendations.

Treatment utility of psychological assessment

This article focuses on treatment utility. A definition of treatment utility was provided by S. C. Hayes, R. O. Nelson, and R. B. Jarrett (1987): "We propose to use the phrase the treatment utility of assessment to refer to the degree to which assessment is shown to contribute to beneficial treatment outcome" (p. 963). Various methodologies to examine the treatment utility of assessment are summarized. Treatment utility studies using various assessment procedures (i.e., diagnosis and functional analysis) and various disorders (i.e., unipolar depression, social or interpersonal problems, and phobic disorders) are described. Suggestions are made as to when elaborated assessment and/or treatment utility studies are needed. Limitations on the generalizability of results of any particular treatment utility study are presented. Despite progress, for most assessment procedures and devices, the treatment utility question remains; What is the degree to which assessment is shown to contribute to beneficial treatment outcome?

Treating child abuse-related posttraumatic stress and comorbid substance abuse in adolescents

OBJECTIVE

Child abuse is a risk factor for developing Posttraumatic Stress Disorder (PTSD) and subsequent Substance Use Disorder (SUD). The purpose of this review is to summarize current knowledge about effective treatments for adolescent abuse-related PTSD, SUD, and the co-occurrence of these conditions.

METHOD

The literature on empirical treatment studies for these conditions in adolescence was reviewed, summarized, and synthesized.

RESULTS

Randomized controlled studies of abuse-related PTSD and SUD in adolescents have supported the efficacy of cognitive behaviorally-based individual and family treatment components. Components overlap considerably in empirically supported treatments for each disorder. An integrated treatment approach is described for use in adolescents with abuse-related PTSD and SUD, with recommendations for optimizing services for this population and for future research.

CONCLUSIONS

The available evidence on effective treatments suggests that integrated PTSD- and SUD-focused cognitive-behavioral and family treatment for adolescents with comorbid abuse-related PTSD and SUD may optimize outcomes for this population.