The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations

Show Me, Don't Tell Me: Behavioral Rehearsal as a Training and Analogue Fidelity Tool

Behavioral rehearsal, when a trainee engages in a simulated interaction with another individual, is an underutilized but potentially cost-effective and feasible solution for two difficult questions in implementation science: how to improve training, a commonly used implementation strategy, and how to feasibly measure fidelity using analogue methods in community settings. This paper provides practical information on how to develop and use behavioral rehearsal for both of these purposes to implementation researchers. Therefore, we focus on development and use of behavioral rehearsal as a training and analogue fidelity tool in the context of three illustrative studies.

Using the core curriculum on childhood trauma to strengthen clinical knowledge in evidence-based practitioners

The high prevalence of trauma exposure in mental health service-seeking populations, combined with advances in evidence-based practice, competency-based training, common-elements research, and adult learning make this an opportune time to train the mental health workforce in trauma competencies. The Core Curriculum on Childhood Trauma (CCCT) utilizes a five-tiered conceptual framework (comprising Empirical Evidence, Core Trauma Concepts, Intervention Objectives, Practice Elements, and Skills), coupled with problem-based learning, to build foundational trauma knowledge and clinical reasoning skills. We present findings from three studies: Study 1 found that social work graduate students' participation in a CCCT course (N = 1,031) was linked to significant pre-post increases in self-reported confidence in applying core trauma concepts to their clinical work. Study 2 found significant pre-post increases in self-reported conceptual readiness (N = 576) and field readiness (N = 303) among social work graduate students participating in a "Gold Standard Plus" educational model that integrated classroom instruction in core trauma concepts, training in evidence-based trauma treatment (EBTT), and implementation of that EBTT in a supervised field placement. Students ranked the core concepts course as an equivalent or greater contributor to field readiness compared to standard EBTT training. Study 3 used qualitative methods to "distill" common elements (35 intervention objectives, 59 practice elements) from 26 manualized trauma interventions. The CCCT is a promising tool for educating "next-generation" evidence-based practitioners who possess competencies needed to implement modularized, individually tailored trauma interventions by strengthening clinical knowledge, clinical reasoning, and familiarity with common elements.

Improving transportability of a cognitive-behavioral treatment intervention for anxiety in youth with autism spectrum disorders: results from a US-Canada collaboration

Anxiety disorders frequently co-occur in youth with autism spectrum disorders. In addition to developing efficacious treatments for anxiety in children with autism spectrum disorders, it is important to examine the transportability of these treatments to real-world settings. Study aims were to (a) train clinicians to deliver Facing Your Fears: Group Therapy for Managing Anxiety in Children with High-Functioning Autism Spectrum Disorders to fidelity and (b) examine feasibility of the program for novel settings. A secondary aim was to examine preliminary youth treatment outcome. Results indicated that clinicians obtained excellent fidelity following a workshop and ongoing consultation. Acceptability ratings indicated that Facing Your Fears Therapy was viewed favorably, and critiques were incorporated into program revisions. Meaningful reductions in anxiety were reported posttreatment for 53% of children. Results support the initial effectiveness and transportability of Facing Your Fears Therapy in new clinical settings.

Disseminating contingency management: impacts of staff training and implementation at an opiate treatment program

Guided by a comprehensive implementation model, this study examined training/implementation processes for a tailored contingency management (CM) intervention instituted at a Clinical Trials Network-affiliate opioid treatment program (OTP). Staff-level training outcomes (intervention delivery skill, knowledge, and adoption readiness) were assessed before and after a 16-hour training, and again following a 90-day trial implementation period. Management-level implementation outcomes (intervention cost, feasibility, and sustainability) were assessed at study conclusion in a qualitative interview with OTP management. Intervention effectiveness was also assessed via independent chart review of trial CM implementation vs. a historical control period. Results included: 1) robust, durable increases in delivery skill, knowledge, and adoption readiness among trained staff; 2) positive managerial perspectives of intervention cost, feasibility, and sustainability; and 3) significant clinical impacts on targeted patient indices. Collective results offer support for the study's collaborative intervention design and the applied, skills-based focus of staff training processes. Implications for CM dissemination are discussed.

Developing a systematic evaluation approach for training programs within a train-the-trainer model for youth cognitive behavior therapy

The purpose of this small pilot study was three-fold: (a) to begin development of a coding scheme for supervisor and therapist skill acquisition, (b) to preliminarily investigate a pilot train-the-trainer paradigm for skill development, and (c) to evaluate self-reported versus observed indicators of skill mastery in that pilot program. Participants included four supervisor-therapist dyads (N = 8) working with public mental health sector youth. Master trainers taught cognitive-behavioral therapy techniques to supervisors, who in turn trained therapists on these techniques. Supervisor and therapist skill acquisition and supervisor use of teaching strategies were repeatedly assessed through coding of scripted role-plays with a multiple-baseline across participants and behaviors design. The coding system, the Practice Element Train the Trainer - Supervisor/Therapist Versions of the Therapy Process Observational Coding System for Child Psychotherapy, was developed and evaluated though the course of the investigation. The coding scheme demonstrated excellent reliability (ICCs [1,2] = 0.81-0.91) across 168 video recordings. As calculated through within-subject effect sizes, supervisor and therapist participants, respectively, evidenced skill improvements related to teaching and performing therapy techniques. Self-reported indicators of skill mastery were inflated in comparison to observed skill mastery. Findings lend initial support for further developing an evaluative approach for a train-the-trainer effort focused on disseminating evidence-based practices.

Facilitators and barriers associated with implementation of evidence-based psychotherapy in community settings

OBJECTIVE Despite widespread use of individual outpatient psychotherapies among community mental health centers (CMHCs), few studies have examined implementation of these psychotherapies. This exploratory qualitative study identified key themes associated with the implementation of an empirically supported psychotherapy in CMHCs. METHODS The authors conducted semistructured interviews with 12 key informants from four CMHCs that had implemented interpersonal and social rhythm therapy (IPSRT). Their responses were categorized into key themes. RESULTS Five major themes were identified: pretraining familiarity with IPSRT, administrative support for implementation, IPSRT fit with usual practice and clinic culture, implementation team and plan, and supervision and consultation. Discussion of these themes varied among participants from clinics considered successful or unsuccessful implementers. CONCLUSIONS Participants identified both key themes and several strategies for facilitating implementation. The findings suggest that when these key factors are present, outcome-enhancing treatments can be implemented and sustained, even in clinics with limited resources.

Implementation strategies: recommendations for specifying and reporting

Implementation strategies have unparalleled importance in implementation science, as they constitute the 'how to' component of changing healthcare practice. Yet, implementation researchers and other stakeholders are not able to fully utilize the findings of studies focusing on implementation strategies because they are often inconsistently labelled and poorly described, are rarely justified theoretically, lack operational definitions or manuals to guide their use, and are part of 'packaged' approaches whose specific elements are poorly understood. We address the challenges of specifying and reporting implementation strategies encountered by researchers who design, conduct, and report research on implementation strategies. Specifically, we propose guidelines for naming, defining, and operationalizing implementation strategies in terms of seven dimensions: actor, the action, action targets, temporality, dose, implementation outcomes addressed, and theoretical justification. Ultimately, implementation strategies cannot be used in practice or tested in research without a full description of their components and how they should be used. As with all intervention research, their descriptions must be precise enough to enable measurement and 'reproducibility.' We propose these recommendations to improve the reporting of implementation strategies in research studies and to stimulate further identification of elements pertinent to implementation strategies that should be included in reporting guidelines for implementation strategies.

Implementation strategies: recommendations for specifying and reporting

Implementation strategies have unparalleled importance in implementation science, as they constitute the 'how to' component of changing healthcare practice. Yet, implementation researchers and other stakeholders are not able to fully utilize the findings of studies focusing on implementation strategies because they are often inconsistently labelled and poorly described, are rarely justified theoretically, lack operational definitions or manuals to guide their use, and are part of 'packaged' approaches whose specific elements are poorly understood. We address the challenges of specifying and reporting implementation strategies encountered by researchers who design, conduct, and report research on implementation strategies. Specifically, we propose guidelines for naming, defining, and operationalizing implementation strategies in terms of seven dimensions: actor, the action, action targets, temporality, dose, implementation outcomes addressed, and theoretical justification. Ultimately, implementation strategies cannot be used in practice or tested in research without a full description of their components and how they should be used. As with all intervention research, their descriptions must be precise enough to enable measurement and 'reproducibility.' We propose these recommendations to improve the reporting of implementation strategies in research studies and to stimulate further identification of elements pertinent to implementation strategies that should be included in reporting guidelines for implementation strategies.

Implementation strategies: recommendations for specifying and reporting

Implementation strategies have unparalleled importance in implementation science, as they constitute the ‘how to’ component of changing healthcare practice. Yet, implementation researchers and other stakeholders are not able to fully utilize the findings of studies focusing on implementation strategies because they are often inconsistently labelled and poorly described, are rarely justified theoretically, lack operational definitions or manuals to guide their use, and are part of ‘packaged’ approaches whose specific elements are poorly understood. We address the challenges of specifying and reporting implementation strategies encountered by researchers who design, conduct, and report research on implementation strategies. Specifically, we propose guidelines for naming, defining, and operationalizing implementation strategies in terms of seven dimensions: actor, the action, action targets, temporality, dose, implementation outcomes addressed, and theoretical justification. Ultimately, implementation strategies cannot be used in practice or tested in research without a full description of their components and how they should be used. As with all intervention research, their descriptions must be precise enough to enable measurement and ‘reproducibility.’ We propose these recommendations to improve the reporting of implementation strategies in research studies and to stimulate further identification of elements pertinent to implementation strategies that should be included in reporting guidelines for implementation strategies.

More practice, less preach? the role of supervision processes and therapist characteristics in EBP implementation

Identifying predictors of evidence-based practice (EBP) use, such as supervision processes and therapist characteristics, may support dissemination. Therapists (N = 57) received training and supervision in EBPs to treat community-based youth (N = 136). Supervision involving modeling and role-play predicted higher overall practice use than supervision involving discussion, and modeling predicted practice use in the next therapy session. No therapist characteristics predicted practice use, but therapist sex and age moderated the supervision and practice use relation. Supervision involving discussion predicted practice use for male therapists only, and modeling and role-play in supervision predicted practice use for older, not younger, therapists.

Therapists perspectives on the effective elements of consultation following training

Consultation is an effective implementation strategy to improve uptake of evidence-based practices for youth. However, little is known about what makes consultation effective. The present study used qualitative methods to explore therapists perspectives about consultation. We interviewed 50 therapists who had been trained 2 years prior in cognitive-behavioral therapy for child anxiety. Three themes emerged regarding effective elements of consultation: (1) connectedness with other therapists and the consultant, (2) authentic interactions around actual cases, and (3) the responsiveness of the consultant to the needs of individual therapists. Recommendations for the design of future consultation endeavors are offered.

Identification, modification, and implementation of an evidence-based psychotherapy for children in a low-income country: the use of TF-CBT in Zambia

BACKGROUND

The need to address the treatment gap in mental health services in low- and middle-income countries (LMIC) is well recognized and particularly neglected among children and adolescents. Recent literature with adult populations suggests that evidence-based mental health treatments are effective, feasible, and cross-culturally modifiable for use in LMIC. This paper addresses a gap in the literature documenting pre-trial processes. We describe the process of selecting an intervention to meet the needs of a particular population and the process of cross-cultural adaptation.

METHODS

Community-based participatory research principles were implemented for intervention selection, including joint meetings with stakeholders, review of qualitative research, and review of the literature. Trauma-focused Cognitive Behavioral Therapy (TF-CBT) was chosen as the evidence-based practice for modification and feasibility testing. The TF-CBT adaptation process, rooted within an apprenticeship model of training and supervision, is presented. Clinical case notes were reviewed to document modifications.

RESULTS

Choosing an intervention can work as a collaborative process with community involvement. Results also show that modifications were focused primarily on implementation techniques rather than changes in TF-CBT core elements.

CONCLUSIONS

Studies documenting implementation processes are critical to understanding why intervention choices are made and how the adaptations are generated in global mental health. More articles are needed on how to implement evidence-based treatments in LMIC.

An anatomy of CBT training: trainees’ endorsements of elements, sources and modalities of learning during a postgraduate CBT training course

This study examines trainees’ evaluations of a postgraduate CBT training course to determine which aspects are perceived as having the greatest effect on competence. A course evaluation was completed by three cohorts (n = 73) of a yearlong master's-level CBT training course, and statistical comparisons were conducted to examine the perceived impact of discrete aspects of training. There were significant differences between endorsements of the impact of various aspects of the learning experience. Overall, supervision was perceived to have a greater influence on trainees’ competence than clinical instruction. Interactions with trainers were given the highest rating, followed by learning from personal reflection and peer interactions. The varied aspects of extensive CBT training were perceived by students to be differentially effective in enhancing CBT skills. Supervision was perceived to have the strongest impact on competence, and trainees’ own reflections on their practice in preparation for supervision perceived as an important contributor to supervision's efficacy. The relatively low rating of peer-related learning suggests acceptability for distance training (e.g. internet-based), which excludes peer interaction found in traditional training. Due to limitations inherent in study design, only tentative conclusions can be offered. Further research involving more rigorous hypothesis testing is recommended.

From distal to proximal: Routine educational data monitoring in school-based mental health

Research and practice in school-based mental health (SBMH) typically includes educational variables only as distal outcomes, resulting from improvements in mental health symptoms rather than directly from mental health intervention. Although sometimes appropriate, this approach also has the potential to inhibit the integration of mental health and schools. The current paper applies an existing model of data-driven decision making (Daleiden & Chorpita, 2005) to detail how SBMH can better integrate routine monitoring of school and academic outcomes into four evidence bases: general services research evidence, case histories, local aggregate, and causal mechanisms. The importance of developing new consultation protocols specific to data-driven decision making in SBMH as well as supportive infrastructure (e.g., measurement feedback systems) to support the collection and use of educational data is also described.

Mental health clinicians' experiences of implementing evidence-based treatments

Implementation research has tremendous potential to bridge the research-practice gap; however, we know more about barriers to evidence-based care than the factors that contribute to the adoption and sustainability of evidence-based treatments. In this qualitative study the authors explore the experiences of clinicians (N = 11) who were implementing evidence-based treatments, highlighting the factors that they perceived to be most critical to successful implementation. The clinicians' narratives reveal many leverage points that can inform administrators, clinical supervisors, and clinicians who wish to implement evidence-based treatments, as well as other stakeholders who wish to develop and test strategies for moving evidence-based treatments into routine care.

MAPping Los Angeles County: taking an evidence-informed model of mental health care to scale

We describe the scaling up of an evidence-informed model of care, Managing and Adapting Practice (MAP) in Los Angeles County, California. MAP complemented an array of evidence-based programs selected by the county as part of a large system reform effort designed to improve care for children and adolescents. In addition, we discuss the MAP model for training therapists and present data both on how the training model performed and on the outcomes of youths treated by therapists trained in MAP. We examined the success of two different training pathways for MAP therapists: (a) national training model and (b) MAP agency supervisor model (i.e., train the trainer). We also examined utilization of MAP and outcomes of clients served by MAP. Both the national training and MAP agency supervisor model were successful in producing MAP therapists in a timely fashion and with acceptable competency scores. Furthermore, a large number of clients were receiving MAP services. Finally, outcomes for youth treated with MAP were strong, with effect sizes ranging from .59 to .80 on the Youth Outcome Questionnaire. These data support the notion that scaling up a mental health services approach in a system can be achieved through a strong and broad partnership among relevant stakeholders, can involve a train-the-trainer model, and can result in strong outcomes for clients.

Implementing an evidence-based practices training curriculum to U.S. Army child and family behavioral health providers

Military Children who have experienced multiple deployments of one or both parents are more likely to have emotional and behavioral difficulties compared with their civilian peers (e.g., Chandra et al., 2010 ). The U.S. Army Medical Command has tasked the Child, Adolescent and Family Behavioral Health Office (CAF-BHO) to develop programs to address the behavioral health (BH) needs of Army Children and Families. This article will describe the efforts of the CAF-BHO Training Section to disseminate and implement an evidence based practice training curriculum for Army Child and Family Behavioral Health Providers. Specifically, this article will detail: (a) the decision making strategy used to identify the training protocol for dissemination, (b) adaptations to the training program and treatment protocol to fit a Military context, and (c) efforts to implement, maximize and sustain utilization of evidence-based practices by Military BH providers over a large geographical area with limited resources.

A mixed methods multiple case study of implementation as usual in children's social service organizations: study protocol

BACKGROUND

Improving quality in children's mental health and social service settings will require implementation strategies capable of moving effective treatments and other innovations (e.g., assessment tools) into routine care. It is likely that efforts to identify, develop, and refine implementation strategies will be more successful if they are informed by relevant stakeholders and are responsive to the strengths and limitations of the contexts and implementation processes identified in usual care settings. This study will describe: the types of implementation strategies used; how organizational leaders make decisions about what to implement and how to approach the implementation process; organizational stakeholders' perceptions of different implementation strategies; and the potential influence of organizational culture and climate on implementation strategy selection, implementation decision-making, and stakeholders' perceptions of implementation strategies.

METHODS/DESIGN

This study is a mixed methods multiple case study of seven children's social service organizations in one Midwestern city in the United States that compose the control group of a larger randomized controlled trial. Qualitative data will include semi-structured interviews with organizational leaders (e.g., CEOs/directors, clinical directors, program managers) and a review of documents (e.g., implementation and quality improvement plans, program manuals, etc.) that will shed light on implementation decision-making and specific implementation strategies that are used to implement new programs and practices. Additionally, focus groups with clinicians will explore their perceptions of a range of implementation strategies. This qualitative work will inform the development of a Web-based survey that will assess the perceived effectiveness, relative importance, acceptability, feasibility, and appropriateness of implementation strategies from the perspective of both clinicians and organizational leaders. Finally, the Organizational Social Context measure will be used to assess organizational culture and climate. Qualitative, quantitative, and mixed methods data will be analyzed and interpreted at the case level as well as across cases in order to highlight meaningful similarities, differences, and site-specific experiences.

DISCUSSION

This study is designed to inform efforts to develop more effective implementation strategies by fully describing the implementation experiences of a sample of community-based organizations that provide mental health services to youth in one Midwestern city.

Improving practice in community-based settings: a randomized trial of supervision - study protocol

Background

Evidence-based treatments for child mental health problems are not consistently available in public mental health settings. Expanding availability requires workforce training. However, research has demonstrated that training alone is not sufficient for changing provider behavior, suggesting that ongoing intervention-specific supervision or consultation is required. Supervision is notably under-investigated, particularly as provided in public mental health. The degree to which supervision in this setting includes ‘gold standard’ supervision elements from efficacy trials (e.g., session review, model fidelity, outcome monitoring, skill-building) is unknown. The current federally-funded investigation leverages the Washington State Trauma-focused Cognitive Behavioral Therapy Initiative to describe usual supervision practices and test the impact of systematic implementation of gold standard supervision strategies on treatment fidelity and clinical outcomes.

Methods/Design

The study has two phases. We will conduct an initial descriptive study (Phase I) of supervision practices within public mental health in Washington State followed by a randomized controlled trial of gold standard supervision strategies (Phase II), with randomization at the clinician level (i.e., supervisors provide both conditions). Study participants will be 35 supervisors and 130 clinicians in community mental health centers. We will enroll one child per clinician in Phase I (N = 130) and three children per clinician in Phase II (N = 390). We use a multi-level mixed within- and between-subjects longitudinal design. Audio recordings of supervision and therapy sessions will be collected and coded throughout both phases. Child outcome data will be collected at the beginning of treatment and at three and six months into treatment.

Discussion

This study will provide insight into how supervisors can optimally support clinicians delivering evidence-based treatments. Phase I will provide descriptive information, currently unavailable in the literature, about commonly used supervision strategies in community mental health. The Phase II randomized controlled trial of gold standard supervision strategies is, to our knowledge, the first experimental study of gold standard supervision strategies in community mental health and will yield needed information about how to leverage supervision to improve clinician fidelity and client outcomes.

Trial registration

ClinicalTrials.gov NCT01800266

Who's using treatment manuals? A national survey of practicing therapists

Treatment manuals are currently the most common way treatments are disseminated to practicing clinicians, although little is known about the rates with which practicing therapists incorporate these manuals into their practice. In light of a widely acknowledged research-practice gap, understanding how often therapists are using manuals is important for shaping future dissemination efforts. This study collected data on rates of manual use among a national sample of mental health clinicians representative of those likely to be targeted in dissemination efforts (N = 756), as well as predictors of use. Results indicated that few clinicians (< 10%) routinely incorporated manuals into their practice, although most employed them to some degree. Predictors of manual use included greater openness to new treatments, younger age, and a cognitive-behavioral treatment orientation (ps < .05). Implications for future dissemination efforts are discussed.

Mental health clinicians' motivation to invest in training: results from a practice-based research network survey

OBJECTIVES

Little is known about why clinicians seek training or about their willingness to invest in it.

METHODS

Results from a Web-based survey of 318 clinicians in a practice-based research network were used to examine factors that motivate clinicians to seek training or forgo training ("deal breakers") and their willingness to invest time and money in training.

RESULTS

Clinicians desired training that teaches advanced versus basic clinical skills, that covers an area they see as central to the needs of their clients, and that provides continuing education credit. Training that requires clinical supervision or the use of a manualized intervention was not a deal breaker for most clinicians. However, the amount of time and money most clinicians reported being willing to invest in training fell far short of the requirements for learning most evidence-based treatments.

CONCLUSIONS

Training strategies that combine high intensity with lower cost may be needed.

A randomized controlled dismantling trial of post-workshop consultation strategies to increase effectiveness and fidelity to an evidence-based psychotherapy for Posttraumatic Stress Disorder

Background

Posttraumatic Stress Disorder (PTSD) is a serious mental health condition with substantial costs to individuals and society. Among military veterans, the lifetime prevalence of PTSD has been estimated to be as high as 20%. Numerous research studies have demonstrated that short-term cognitive-behavioral psychotherapies, such as Cognitive Processing Therapy (CPT), lead to substantial and sustained improvements in PTSD symptoms. Despite known benefits, only a minority of clinicians provide these therapies. Transferring this research knowledge into clinical settings remains one of the largest hurdles to improving the health of veterans with PTSD. Attending a workshop alone is insufficient to promote adequate knowledge transfer and sustained skill; however, relatively little research has been conducted to identify effective post-training support strategies.

Methods

The current study investigates whether clinicians receiving post-workshop support (six-month duration) will deliver CPT with greater fidelity (i.e., psychotherapy adherence and competence) and have improved patient outcomes compared with clinicians receiving no formal post-workshop support. The study conditions are: technology-enhanced group tele-consultation; standard group tele-consultation; and fidelity assessment with no consultation. The primary outcome is independent assessment (via audio-recordings) of the clinicians’ adherence and competence in delivering CPT. The secondary outcome is observed changes in patient symptoms during and following treatment as a function of clinician fidelity. Post-consultation interviews with clinicians will help identify facilitators and barriers to psychotherapy skill acquisition. The study results will inform how best to implement and transfer evidence-based psychotherapy (e.g., CPT) to clinical settings to attain comparable outcomes to those observed in research settings.

Discussion

Findings will deepen our understanding of how much and what type of support is needed following a workshop to help clinicians become proficient in delivering a new protocol. Several influences on clinician learning and patient outcomes will be discussed. An evidence-based model of clinical consultation will be developed, with the ultimate goal of informing policy and influencing best practice in clinical consultation.

Trial registration

ClinicalTrials.gov: NCT01861769

Parent-child interaction therapy: a manualized intervention for the therapeutic child welfare sector

Children who have experienced maltreatment can often display behavioral difficulties; their parents may lack disciplinary knowledge, be less sensitive to their children, and engage in coercive parenting practices. Parent-child interaction therapy (PCIT) is a well-known, evidence-based treatment (EBT) for child behavior problems and within the last decade has garnered significant evidence to suggest its utility for parents engaged in child maltreatment. This article uses a case example to describe PCIT treatment phases and PCIT research within the child maltreatment sector is synthesized with particular focus on treatment modifications. Successful augmentations and modifications include a motivation component, keeping therapeutic time shorter rather than longer, and whether to incorporate individual counseling and in-home PCIT are also considered. Practical strategies from both a therapeutic and research experience are discussed.

Engineering youth service system infrastructure: Hawaii's continued efforts at large-scale implementation through knowledge management strategies

Hawaii's Child and Adolescent Mental Health Division provides a unique illustration of a youth public mental health system with a long and successful history of large-scale quality improvement initiatives. Many advances are linked to flexibly organizing and applying knowledge gained from the scientific literature and move beyond installing a limited number of brand-named treatment approaches that might be directly relevant only to a small handful of system youth. This article takes a knowledge-to-action perspective and outlines five knowledge management strategies currently under way in Hawaii. Each strategy represents one component of a larger coordinated effort at engineering a service system focused on delivering both brand-named treatment approaches and complimentary strategies informed by the evidence base. The five knowledge management examples are (a) a set of modular-based professional training activities for currently practicing therapists, (b) an outreach initiative for supporting youth evidence-based practices training at Hawaii's mental health-related professional programs, (c) an effort to increase consumer knowledge of and demand for youth evidence-based practices, (d) a practice and progress agency performance feedback system, and (e) a sampling of system-level research studies focused on understanding treatment as usual. We end by outlining a small set of lessons learned and a longer term vision for embedding these efforts into the system's infrastructure.

Clinicians' perspectives on cognitive therapy in community mental health settings: implications for training and implementation

Policymakers are investing significant resources in large-scale training and implementation programs for evidence-based psychological treatments (EBPTs) in public mental health systems. However, relatively little research has been conducted to understand factors that may influence the success of efforts to implement EBPTs for adult consumers of mental health services. In a formative investigation during the development of a program to implement cognitive therapy (CT) in a community mental health system, we surveyed and interviewed clinicians and clinical administrators to identify potential influences on CT implementation within their agencies. Four primary themes were identified. Two related to attitudes towards CT: (1) ability to address client needs and issues that are perceived as most central to their presenting problems, and (2) reluctance to fully implement CT. Two themes were relevant to context: (1) agency-level barriers, specifically workload and productivity concerns and reactions to change, and (2) agency-level facilitators, specifically, treatment planning requirements and openness to training. These findings provide information that can be used to develop strategies to facilitate the implementation of CT interventions for clients being treated in public-sector settings.

The dissemination and implementation of psychological treatments: problems and solutions

Treatment researchers expend their efforts identifying effective treatments, and for whom and how they work, but there are matters over and above these that are of concern when it comes to dissemination and implementation. These include the clinical range of the interventions concerned, the ease with which they can be learned, and their mode of delivery. It is these three topics, as they apply to the psychological treatment of eating disorders, that form the focus of this article. Alongside these considerations, we discuss how modern technology has the potential to transform both treatment and training.

Training in and implementation of Acceptance and Commitment Therapy for depression in the Veterans Health Administration: therapist and patient outcomes

OBJECTIVE

The U.S. Department of Veterans Affairs has implemented a national dissemination and training initiative to promote the availability of Acceptance and Commitment Therapy for depression (ACT-D). This paper reports on therapist and patient outcomes associated with competency-based training in and implementation of ACT-D.

METHOD

Therapist and patient outcomes were assessed on eleven cohorts of therapists (n = 391) and their patients (n = 745).

RESULTS

Three-hundred thirty four therapists successfully completed all requirements of the Training Program. Ninety-six percent of therapists achieved competency by the end of training, compared to 21% at the outset of training. Mixed effects model analysis indicated therapists' overall ACT-D competency scores increased from 76 to 112 (conditional SD = 6.6), p < 0.001. Moreover, training was associated with significantly increased therapist self-efficacy and positive attitudes toward ACT-D. Therapeutic alliance increased significantly over the course of therapy. Mixed effects model analysis revealed that mean BDI-II scores decreased from 30 at baseline assessment to 19 (conditional SD = 5.6) at final assessment, t(367) = -20.3, p < 0.001. Quality of life scores also increased.

CONCLUSIONS

Training in and implementation of ACT-D in the treatment of Veterans is associated with significant increases in therapist competency and robust improvements in patient outcomes.

Dissemination and Implementation of Evidence-Based Practices: Training and Consultation as Implementation Strategies

To provide effective treatment for individuals with mental health needs, there is a movement to deploy evidence-based practices (EBPs) developed in research settings into community settings. Training clinicians in EBPs is often used as the primary implementation strategy in these efforts, despite evidence suggesting that training alone does not change therapist behavior. A promising implementation strategy that can be combined with training is consultation, or ongoing support. This paper reviews the literature on consultation following initial training. A model of consultation is presented as well as preliminary findings regarding effective consultation techniques. Future directions are offered.

Continuing education to go: capacity building in psychotherapies for front-line mental health workers in underserviced communities

OBJECTIVE

To address the gaps between need and access, and between treatment guidelines and their implementation for mental illness, through capacity building of front-line health workers.

METHODS

Following a learning needs assessment, work-based continuing education courses in evidence-supported psychotherapies were developed for front-line workers in underserviced community settings. The 5-hour courses on the fundamentals of cognitive-behavioural therapy, interpersonal psychotherapy, motivational interviewing, and dialectical behaviour therapy each included videotaped captioned simulations, interactive lesson plans, and clinical practice behaviour reminders. Two courses, sequentially offered in 7 underserviced settings, were subjected to a mixed methods evaluation. Ninety-three nonmedical front-line workers enrolled in the program. Repeated measures analysis of variance was used to assess pre- and postintervention changes in knowledge and self-efficacy. Qualitative data from 5 semistructured focus groups with 25 participants were also analyzed.

RESULTS

Significant pre- and postintervention changes in knowledge (P < 0.001) were found in course completers. Counselling self-efficacy improved in participants who took the first course offered (P = 0.001). Dropouts were much less frequent in peer-led, small-group learning than in a self-directed format. Qualitative analysis revealed improved confidence, morale, self-reported practice behaviour changes, and increased comfort in working with difficult clients.

CONCLUSION

This work-based, multimodal, interactive, interprofessional curriculum for knowledge translation of psychotherapeutic techniques is feasible and helpful. A peer-led group format is preferred over self-directed learning. Its application can build capacity of front-line health workers in helping patients who suffer from common mental disorders.

Teaching therapeutic assessment for self-harm in adolescents: training outcomes

OBJECTIVES

To describe the teaching programme of Therapeutic Assessment (TA), a brief intervention at the point of initial assessment for adolescents with self-harm; to describe trainees' preferences and choices regarding their use of specific aspects of TA.

DESIGN

This is a comparative study investigating the differences in the TA skills before and after training. This design was chosen to establish whether or not TA training is efficacious.

METHODS

 Twenty-four clinicians volunteered to participate in five half-day TA training sessions. Their scores on the Therapeutic Assessment Quality Assurance Tool (TAQAT, primary outcome measure) were compared before and after training. Satisfaction with training and therapeutic strategy choices as well as ability to perform TA in an RCT were investigated.

RESULTS

Clinicians who participated in TA training had significantly increased scores on TAQAT after training. The clinicians who achieved the required quality of TA post assessments were likely to be able to carry out TA in an RCT with high fidelity. In addition, prior to training, significant differences in the quality of assessments as measured by TAQAT were identified depending on the experience of the clinician. This discrepancy was no longer present post training. Therapeutic strategy based on solution-focused brief therapy (SFBT) was the option of choice post training.

CONCLUSIONS

TA training is feasible and associated with improved quality of self-harm assessment.

PRACTITIONER POINTS

TA is a brief intervention associated with improved treatment engagement. TA training is feasible and is associated with improved quality of self-harm assessment. SFBT-based exit is the most commonly used strategy in TA.

Making a Case for Treatment Integrity as a Psychosocial Treatment Quality Indicator for Youth Mental Health Care

Measures of treatment integrity are needed to advance clinical research in general and are viewed as particularly relevant for dissemination and implementation research. Although some efforts to develop such measures are underway, a conceptual and methodological framework will help guide these efforts. The purpose of this article is to demonstrate how frameworks adapted from the psychosocial treatment, therapy process, healthcare, and business literatures can be used to address this gap. We propose that components of treatment integrity (i.e., adherence, differentiation, competence, alliance, client involvement) pulled from the treatment technology and process literatures can be used as quality indicators of treatment implementation and thereby guide quality improvement efforts in practice settings. Further, we discuss how treatment integrity indices can be used in feedback systems that utilize benchmarking to expedite the process of translating evidence-based practices to service settings.

Advances in Applying Treatment Integrity Research for Dissemination and Implementation Science: Introduction to Special Issue

This special series focuses upon the ways in which research on treatment integrity, a multidimensional construct including assessment of the content and quality of a psychosocial treatment delivered to a client as well as relational elements, can inform dissemination and implementation science. The five articles for this special series illustrate how treatment integrity concepts and methods can be applied across different levels of the mental health service system to advance dissemination and implementation science. In this introductory article, we provide an overview of treatment integrity research and describe three broad conceptual models that are relevant to the articles in the series. We conclude with a brief description of each of the five articles in the series.

Training community mental health therapists to deliver a package of evidence-based practice strategies for school-age children with autism spectrum disorders: a pilot study

Research on moving evidence-based practice (EBP) intervention strategies to community service settings for children with autism spectrum disorders (ASD) is urgently needed. The current pilot study addresses this need by examining the feasibility, acceptability and preliminary outcomes of training therapists practicing in community mental health (CMH) clinics to deliver a package of EBP strategies aimed to reduce challenging behaviors in school-age children with ASD. Results indicate that CMH therapists participated in both initial and ongoing training, were able to deliver the intervention with fidelity, and perceived the intervention strategies as useful. Parents participated in almost all sessions with their children and remained in therapy when therapists delivered the intervention. Meaningful reductions in child problem behaviors occurred over 5 months providing promising support for the intervention.

Anxious youth in research and service clinics

With the current focus on increasing utilization of empirically supported treatments, knowledge of sample differences and similarities has increasing importance. The present study compared anxiety-disordered youth (age 7-13) from (a) five Norwegian service clinics (SC, N = 111) to (b) a university research clinic (RC) in Philadelphia, USA (N = 144) on pre-treatment characteristics measured by the Multidimensional Anxiety Scale for Children, Child Behavior Checklist, Teacher Report Form, Anxiety Disorders Interview Schedule, and Children's Global Assessment Scale (CGAS). SC youth demonstrated higher levels of anxiety based on child- (d = 0.42-1.04) and parent-report (d = 0.53) and conduct problems based on parent-report (d = 0.43) compared to RC youth. SC youth was more functionally impaired on the CGAS (d = 0.97), whereas RC youth evidenced a greater number of diagnoses (d = 0.63). The two samples were equivalent regarding parent-reported symptoms of affective, somatic, attention-deficit/hyperactivity, and oppositional problems. Future directions and clinical implications are discussed.

Improving community-based mental health care for children: translating knowledge into action

There is urgent need for improvement in community-based mental health care for children and families. Multiple studies have documented serious limitations in the effectiveness of "usual care." Fortunately, many empirically-supported strategies to improve care have been developed, and thus there is now a great deal of knowledge available to address this significant public health problem. The goal of this selective review is to highlight and synthesize that empirically-supported knowledge to stimulate and facilitate the needed translation of knowledge into action. The review provides a sound foundation for constructing improved services by consolidating descriptive data on the status quo in children's mental health care, as well as evidence for an array of promising strategies to improve (a) Service access and engagement; (b) Delivery of evidence-based practices; and (c) Outcome accountability. A multi-level framework is used to highlight recommended care improvement targets.

Correspondence of motivational interviewing adherence and competence ratings in real and role-played client sessions

Treatment integrity ratings (adherence and competence) are frequently used as outcome measures in clinician training studies, drawn from recorded real client or role-played client sessions. However, it is unknown whether clinician adherence and competence are similar in real client and role-played sessions or whether real and role-play clients provide similar opportunities for skill demonstration. In this study, we examined the correspondence of treatment adherence and competence ratings obtained in real client and role-played sessions for 91 clinicians trained in motivational interviewing (MI), using data from a multisite trial examining 3 methods of clinician training (Martino et al., 2011). Results indicated overall poor integrity rating correspondence across the 2 session types, as indicated by weak correlations (rs=.05-.27). Clinicians were rated significantly more MI adherent overall and specifically used more advanced MI strategies in role-played than real client sessions at several assessment time points (ds=0.36, 0.42). Real clients, in comparison to the role-play actor, demonstrated greater motivation at the beginning of the session (d=1.09), discussion of unrelated topics (d=0.70), and alliance with the clinician (d=0.72). These findings suggest that MI integrity rating data obtained from real client and role-played sessions may not be interchangeable. More research is needed to improve the procedures and psychometric strength of treatment integrity assessment based on role-played sessions.

Training School Mental Health Providers to Deliver Cognitive-Behavioral Therapy

Anxiety disorders are among the most prevalent mental health difficulties experienced by youth. A well-established literature has identified cognitive-behavior therapy (CBT) as the gold-standard psychosocial treatment for youth anxiety disorders. Access to CBT in community clinics is limited, but a potential venue for the provision of CBT for child anxiety disorders is the school setting. The present study examined a subset of data from a larger study in which therapists from a variety of settings, including schools, were trained in CBT for child anxiety (N = 17). The study investigated the relationship between provider- and organizational-level variables associated with training and implementation among school mental health providers. The present findings indicate a positive relationship between provider attitudes and adherence to CBT. Self-reported barriers to implementation were also identified. Integrating CBT into school mental health providers' repertoires through training and consultation is a critical step for dissemination and implementation of empirically supported psychosocial treatments.

Train-the-trainer: implementation of cognitive behavioural guided self-help for recurrent binge eating in a naturalistic setting

OBJECTIVE

The present study evaluated the feasibility and acceptability of cognitive behavioural guided self-help (CBTgsh) for recurrent binge eating using the train-the-trainer implementation strategy.

METHOD

After receiving expert-led training in CBTgsh, a master's-level graduate student in clinical psychology subsequently trained and supervised less experienced graduate students to implement the treatment in an open clinical trial. Participants were 38 treatment-seeking students at a university counselling centre with recurrent binge eating, featuring cases of bulimia nervosa, binge eating disorder, or eating disorder not otherwise specified.

RESULTS

Intent-to-treat (ITT) analyses revealed 42.1% abstinence from binge eating at post-treatment and 47.4% at one-month follow-up. Participants reported significant pretreatment to post-treatment reductions on measures of specific eating disorder psychopathology, general psychopathology, and functional impairment and high levels of treatment acceptability.

CONCLUSIONS

These results provide 'proof-of-concept' for the train-the-trainer implementation strategy and add to the evidence supporting the feasibility and effectiveness of CBTgsh in routine clinical care.

Training and consultation to promote implementation of an empirically supported treatment: a randomized trial

OBJECTIVE

The study evaluated the efficacy of three training modalities and the impact of ongoing consultation after training. Cognitive-behavioral therapy (CBT) for anxiety among youths, an empirically supported treatment, was used as the exemplar. Participants were randomly assigned to one of three one-day workshops to examine the efficacy of training modality: routine training (training as usual), computer training (computerized version of training as usual), and augmented training (training that emphasized active learning). After training, all participants received three months of ongoing consultation that included case consultation, didactics, and problem solving.

METHODS

Participants were 115 community therapists (mean age of 35.9 years; 90% were women). Outcome measures included the Adherence and Skill Checklist, used to rate a performance-based role-play; a knowledge test; and the Training Satisfaction Rating Scale.

RESULTS

All three training modalities resulted in limited gains in therapist adherence, skill, and knowledge. There was no significant effect of modality on adherence, skill, or knowledge from pretraining to posttraining. Participants were more satisfied with augmented and routine training than with computer training. Most important, number of consultation hours after training significantly predicted higher therapist adherence and skill at the three-month follow-up.

CONCLUSIONS

The findings suggest that training alone did not result in therapist behavior change. The inclusion of ongoing consultation was critical to influencing therapist adherence and skill. Implications for implementation science and mental health services research are discussed.

A translational model of research-practice integration

We propose a four-level, recursive Research-Practice Integration framework as a heuristic to (a) integrate and reflect on the articles in this Special Section as contributing to a bidirectional bridge between research and practice, and (b) consider additional opportunities to address the research-practice gap. Level 1 addresses Treatment Validation studies and includes an article by Lochman and colleagues concerning the programmatic adaptation, implementation, and dissemination of the empirically supported Coping Power treatment program for youth aggression. Level 2 translation, Training in Evidence-Based Practice, includes a paper by Hershenberg, Drabick, and Vivian, which focuses on the critical role that predoctoral training plays in bridging the research-practice gap. Level 3 addresses the Assessment of Clinical Utility and Feedback to Research aspects of translation. The articles by Lambert and Youn, Kraus, and Castonguay illustrate the use of commercial outcome packages that enable psychotherapists to integrate ongoing client assessment, thus enhancing the effectiveness of treatment implementation and providing data that can be fed back to researchers. Lastly, Level 4 translation, the Cross-Level Integrative Research and Communication, concerns research efforts that integrate data from clinical practice and all other levels of translation, as well as communication efforts among all stakeholders, such as researchers, psychotherapists, and clients. Using a two-chair technique as a framework for his discussion, Wolfe's article depicts the struggle inherent in research-practice integration efforts and proposes a rapprochement that highlights advancements in the field.

Effects of workshop trainings on evidence-based practice knowledge and attitudes among youth community mental health providers

Enhancing the public health impact of evidence-based practices (EBPs) in usual care settings is a key priority of the National Institute of Mental Health. Longitudinal data from community mental health providers (N = 268) participating in a series of state-sponsored workshops in modular approaches to EBPs for youth are presented. EBP workshop attendance for youth anxiety resulted in increased knowledge for EBPs for anxiety (and not other conditions) and EBP workshop attendance for youth disruptive behaviors resulted in increased knowledge for EBPs for disruptive behaviors (and not other conditions). Providers' tendencies toward incorrectly classifying non-EBP therapies as evidence-based increased over time, suggesting that providers over-generalize the EBP label as a result of attending these types of workshops. Regarding EBP attitudes, most measures of attitudes improved when providers attended a workshop. Additionally, an overly inclusive view of what constitutes an EBP at intake was related to significant decreases in openness to trying EBPs over time, whereas more positive attitudes at intake was related to achieving a more refined view of what constitutes an EBP over the course of attending trainings. Study limitations and implications for implementation of EBPs in usual care settings are discussed.

Cognitive-behavioral guided self-help for eating disorders: effectiveness and scalability

Given the well-documented shortage of cognitive-behavioral therapy (CBT) for eating disorders, there is a compelling need for advances in dissemination. Guided self-help based on cognitive-behavioral principles (CBTgsh) provides a robust means of improving implementation and scalability of evidence-based treatment for eating disorders. It is a brief, cost-effective treatment that can be implemented by a wide range of mental health providers, including non-specialists, via face-to-face contact and internet-based technology. Controlled studies have shown that CBTgsh can be an effective treatment for binge eating disorder and bulimia nervosa, although it is contraindicated for anorexia nervosa. Several studies have shown that CBTgsh can be as effective as more complex specialty therapies and that it is not necessarily contraindicated for patients with comorbid conditions. Mental health providers with relatively minimal professional credentials have in some studies obtained results comparable to specialized clinicians. Establishing the nature of optimal "guidance" in CBTgsh and the level of expertise and training required for effective implementation is a research priority. Existing manuals used in CBTgsh are outdated and can be improved by incorporating the principles of enhanced transdiagnostic CBT. Obstacles to wider adoption of CBTgsh are identified.

Tobacco dependence counseling in a randomized multisite clinical trial

Pharmacotherapy trials for treating tobacco dependence would benefit from behavioral interventions providing treatment consistent with clinical practice guidelines but not directing participants to treatments not evaluated in the trial. The Smoke Free and Living It© behavioral intervention manual includes participant and interventionist guides and is designed to provide both practical counseling and intra-treatment support. We utilized this intervention manual in a multicenter, randomized clinical trial of smokers with attention deficit hyperactivity disorder. In this study, we evaluated how the interventional manual performed in a "train-the-trainer" model requiring uniform counseling across 6 sites and 15 interventionists. We analyzed the skill-adherence of the interventionists and the intervention-adherence of the participants. The 255 randomized participants completed 9.3±2.8 sessions (mean±SD), with 157 participants (61.6%) completing all 11 of the sessions and 221 (86.7%) completing at least 6 of the 11 sessions. Of the 163 sessions for which the study interventionists were evaluated, 156 (95.7%) were rated as adherent to protocol and "meeting expectations" on at least 6 of 7 established criteria, illustrating that fidelity can be maintained with minimal supervision. The self-help and interventionists guides of the Smoke Free and Living It manual can thus be used to provide behavioral intervention with a high rate of adherence by both the interventionists and the participants. This manual meets the requirements of the United States Public Health Service Clinical Practice Guideline, can be adapted to specific research protocols, and provides a useful option for behavioral intervention during clinical trials for smoking cessation.