Building an Evidence Base for Effective Supervision Practices: An Analogue Experiment of Supervision to Increase EBT Fidelity

The modular approach to therapy for youths with anxiety, depression, trauma, and conduct problems (MATCH): results from the Norwegian randomized-controlled trial

BACKGROUND

A randomized controlled trial was conducted to examine the effectiveness of the Modular Approach to Therapy for Youths with Anxiety, Depression, Trauma, and Conduct Problems (MATCH) for Norwegian youths referred to seven Child and Adolescent Psychiatric Outpatient Clinics. MATCH addresses comorbid problems that are common in children and youth, and its transdiagnostic design may therefore be more effective compared to standard treatments that often address single problems. MATCH has, however, never been evaluated in a Nordic context, and the present study aimed to fill this gap.

METHODS

A sample of 121 Norwegian youths (Mage = 9.83, 58.7% boys) was randomly assigned to MATCH (n = 73) or treatment as usual (TAU, n = 48). Primary treatment outcomes were youths' externalizing and internalizing problems as reported by parents, using the Child Behavior Checklist, the Behavior and Feelings Survey. In addition, the study included assessments of parent-reported Top Problems.

RESULTS

Overall, youths showed significant improvements in both externalizing and internalizing problems from intake to post-test. Results did not provide evidence that MATCH reduces symptoms of these problems compared to TAU.

CONCLUSIONS

The findings were inconclusive regarding whether MATCH was more effective than TAU in reducing youth internalizing and externalizing problems.

TRIAL REGISTRATION IDENTIFIER

ISRCTN24029895. Registration date: 8/8/2016.

An ISM-MICMAC-based study for identification and classification of preventable safety risk mitigation factors in mass housing projects following a BIM approach

Construction operation is among the most high-risk sectors in terms of work-related accident, making it highly challenging to surveil the safety of such projects. In construction projects, failure to observe safety represents a leading cause of fatal accidents, not to mention the losses incurred by such accidents to national assets of the country. Accordingly, recent decades have witnessed the emergence of modern techniques for improving the occupational safety of construction projects. The main purpose of the present research is to identify and classify different preventable risk mitigation factors in mass housing projects following a building information modeling (BIM) approach. The research methodology included interviews with relevant experts and elites followed by analysis of the data on the 12 identified-as-significant variables for mitigating the preventable risk factors in mass house construction projects by means of the inferential - structural modeling (ISM) in MICMAC software. In order to explore the relationships among and succession of different criteria and further classify them at different levels, ISM was implemented, with the MICMAC software used to analyze the direct and indirect influences, develop influence/dependence maps, and judge about the role of each criterion. Findings of the present research showed that the mutual relations (H3), the reward system (H6), the reporting system (H7), and the supervisors' supervision (H8) are autonomous variables and hence impose the smallest contributions to the system. Accordingly, they can be eliminated from the model though their effects may not be completely ignored. On the other hand, the employees' empowering (H4), the safety management system (H5), the teamwork (H9), the self-efficiency (H10), and the knowledge and awareness (H11) were identified as the linkage variables that fill in the gap between the safety and occupational accident reduction in the mass house construction projects. Further, the continuous improvement (H2) and the safe behavior (H12) were identified as dependent variables, implying that they exhibit the weakest influence coupled with highest dependence on any change in the conditions of the system. Last but not the least, the management commitment (H1) was identified as the only dependent variable which deserves lots of attention. This information can be helpful to safety decision-makers, end users, research organizations, and academic institutes who work to reduce the preventable risk factors in mass house construction projects.

Consultation Content and Techniques for measurement-Based Care Implementation in Youth Community Mental Health Settings

Measurement-based care (MBC) is an evidence-based practice (EBP) focused on regularly administering outcome measures to clients to inform clinical decision making. While MBC shows promise for improving youth treatment outcomes, therapist adoption remains low. Clinical consultation is one strategy that improves MBC implementation, but our limited understanding of consultation hinders the ability to optimize its impact. This research explored the content of, and techniques used during MBC consultation calls. Therapists (N = 55) in a randomized controlled trial treating adolescents with anxiety and/or depression were trained to utilize MBC with usual treatment using the Youth Outcome Questionnaire (YOQ) through an online measurement feedback system (MFS). Weekly ongoing consultation followed an initial workshop training in MBC. Case discussions (N = 294) during consultation calls were coded using a developed codebook, including 12 content and 10 consultant techniques. Results indicated that content focused predominantly on interpretation of client symptom and alliance report, planning for YOQ administration, and discussion of data with clients in session. Common consultant techniques included modeling and eliciting report viewing and interpretation, making clinical suggestions, and didactics about clinical and technical issues. Notably, role-play/behavioral rehearsal was not used. The prevalence of passive consultation techniques (suggestions, didactics) suggests a focus on teaching rather than active techniques (behavioral rehearsal, modeling), potentially influenced by the novelty of MBC and MFS. Technical aspects of MBC, such as measure administration and system usage, emerged as key consultation content, highlighting an unanticipated emphasis on logistics over clinical implementation. These findings underscore the evolving role of consultation in supporting MBC implementation and suggest that addressing technical challenges early in training might enhance adoption.

The Interrater Reliability of a Coding System for Measuring Mental Health Professionals' Decisions and Actions

OBJECTIVE

The clinical decisions and actions of evidence-based practice in psychology (EBPP) are largely underspecified and poorly understood, in part due to the lack of measurement methods. We tested the reliability of a behavioral coding system that characterizes a flow of interrelated activities that includes problem detection and prioritization, intervention selection and implementation, and review of intervention integrity and impact.

METHOD

The context included two publicly funded youth mental health service organizations located in geographically distinct and underresourced communities in the U.S. where service inequities are common. We sampled 84 digitally recorded and transcribed supervision events that included a sample of professionals who were mostly women (93.02%) and BIPOC (86.04%) whose self-reported race/ethnicity matched the youth populations they served. We coded these events for activities (e.g., considering) and their predicate content (i.e., problems or practices) and examined reliability of these codes applied to excerpts (i.e., small contiguous units of dialogue) as well as to complete events.

RESULTS

Interrater reliability estimates showed that, overall, coders reliably rated the occurrence and extensiveness of activities and content. Excerpt coding was generally more reliable than event coding. However, mathematical aggregation of excerpt coding offered a superior method for estimating event codes reliably, reducing individual subjectivity while providing event level synthesis of activities that are grounded in excerpt level details.

CONCLUSIONS

The assessment of clinical decisions and actions has the potential to unpack the black box of EBPP, with different methods best suited to different research questions and resource considerations.

Supervisory Alliance as a Moderator of the Effects of Behavioral Rehearsal on TF-CBT Fidelity: Results from a Randomized Trial of Supervision Strategies

Exposure is an important element of treatment for many evidence-based treatments but can be challenging to implement. Supervision strategies to support exposure delivery may be an important tool to facilitate the use of exposure techniques; however, they must be considered and used in the context of the supervisory alliance. The present study examined relations between supervisory alliance and fidelity to the trauma narrative (TN; i.e., imaginal exposure) component of Trauma-Focused Cognitive Behavioral Therapy. We also examined how supervisory alliance moderated the effect of behavioral rehearsal use in supervision on TN fidelity. We analyzed data from a randomized controlled trial, in which forty-two supervisors and their clinicians (N = 124) from 28 Washington State community-based mental health offices participated. Clinicians were randomized to receive one of two supervision conditions-symptom and fidelity monitoring (SFM) or SFM with behavioral rehearsal (SFM + BR). Supervisory alliance alone did not predict delivery (i.e., occurrence) or extensiveness of delivery of the trauma narrative. Client-focused supervisory alliance moderated the effectiveness of behavioral rehearsal-as client-focused alliance increased, the odds of delivering the TN also increased significantly. Future research should further investigate how to appropriately match supervision techniques with supervisory dyads and explore the interplay of alliance with supervision techniques a supervisor might employ.

Improving the quality of counseling and clinical supervision in opioid treatment programs: how can technology help?

BACKGROUND

The opioid epidemic has resulted in expanded substance use treatment services and strained the clinical workforce serving people with opioid use disorder. Focusing on evidence-based counseling practices like motivational interviewing may be of interest to counselors and their supervisors, but time-intensive adherence tasks like recording and feedback are aspirational in busy community-based opioid treatment programs. The need to improve and systematize clinical training and supervision might be addressed by the growing field of machine learning and natural language-based technology, which can promote counseling skill via self- and supervisor-monitoring of counseling session recordings.

METHODS

Counselors in an opioid treatment program were provided with an opportunity to use an artificial intelligence based, HIPAA compliant recording and supervision platform (Lyssn.io) to record counseling sessions. We then conducted four focus groups-two with counselors and two with supervisors-to understand the integration of technology with practice and supervision. Questions centered on the acceptability of the clinical supervision software and its potential in an OTP setting; we conducted a thematic coding of the responses.

RESULTS

The clinical supervision software was experienced by counselors and clinical supervisors as beneficial to counselor training, professional development, and clinical supervision. Focus group participants reported that the clinical supervision software could help counselors learn and improve motivational interviewing skills. Counselors said that using the technology highlights the value of counseling encounters (versus paperwork). Clinical supervisors noted that the clinical supervision software could help meet national clinical supervision guidelines and local requirements. Counselors and clinical supervisors alike talked about some of the potential challenges of requiring session recording.

CONCLUSIONS

Implementing evidence-based counseling practices can help the population served in OTPs; another benefit of focusing on clinical skills is to emphasize and hold up counselors' roles as worthy. Machine learning technology can have a positive impact on clinical practices among counselors and clinical supervisors in opioid treatment programs, settings whose clinical workforce continues to be challenged by the opioid epidemic. Using technology to focus on clinical skill building may enhance counselors' and clinical supervisors' overall experiences in their places of work.

Developing the Workforce of the Digital Future: mHealth Competency and Fidelity Measurement in Community-Based Care

Integrating mobile health (mHealth) interventions into settings that serve diverse patient populations requires that prerequisite professional competencies are delineated and that standards for clinical quality assurance can be pragmatically assessed. Heretofore, proposed mHealth competencies have been broad and have lacked a framework to support specific applications. We outline the meta-competencies identified in the literature relevant to mHealth interventions and demonstrate how these meta-competencies can be integrated with population- and intervention-related competencies to help guide a pragmatic approach to competency assessment. We present a use case based on FOCUS-an evidence-based mHealth intervention designed for individuals with serious mental illness and currently being implemented in geographically and demographically diverse community behavioral health settings. Subsequent to identifying the cross-cutting competencies relevant to the target population (outpatients experiencing psychotic symptoms), substratal intervention (Cognitive Behavioral Therapy for psychosis), and treatment modality (mHealth), we detail the development process of an mHealth fidelity monitoring system (mHealth-FMS). We adhered to a published sequential 5-step process to design a fidelity monitoring system that aligns with our integrated mHealth competency framework and that was guided by best practices prescribed by the Treatment Fidelity Workgroup of the National Institutes of Health Behavior Change Consortium. The mHealth-FMS is intended to enhance both clinical and implementation outcomes by grounding the mHealth interventionist and the system of care in which they operate in the core functions, tasks, knowledge, and competencies associated with system-integrated mHealth delivery. Future research will explore acceptability and feasibility of the mHealth-FMS.

Uncovering determinants of perceived feasibility of TF-CBT through coincidence analysis

Introduction

A mental health provider's perception of how well an intervention can be carried out in their context (i.e., feasibility) is an important implementation outcome. This article aims to identify determinants of feasibility of trauma-focused cognitive behavioral therapy (TF-CBT) through a case-based causal approach.

Method

Data come from an implementation-effectiveness study in which lay counselors (teachers and community health volunteers) implemented a culturally adapted manualized mental health intervention, TF-CBT, delivered to teens who were previously orphaned and were experiencing posttraumatic stress symptoms and prolonged grief in Western Kenya. The intervention team identified combinations of determinants that led to feasibility among teacher- and community health volunteer-counselors through coincidence analysis.

Results

Among teacher-counselors, organizational-level factors (implementation climate, implementation leadership) determined moderate and high levels of feasibility. Among community health volunteer-counselors, a strong relationship between a clinical supervisor and the supervisee was the most influential determinant of feasibility.

Conclusion

Methodology and findings from this article can guide the assessment of determinants of feasibility and the development of implementation strategies for manualized mental health interventions in contexts like Western Kenya.

Plain Language Summary

A mental health provider's perception of how easy a therapy is to use in their work setting (i.e., feasibility) can impact whether the provider uses the therapy in their setting. Implementation researchers have recommended finding practices and constructs that lead to important indicators that a therapy will be used. However, limited research to our knowledge has searched and found practices and constructs that might determine feasibility of a therapy. This article uses existing data from a large trial looking at the continued use of a trauma-focused therapy to find practices and constructs that lead to moderate and high levels of feasibility. We found that in settings with a strong organizational structure that organization and leadership support for the therapy led to teachers in Kenya to perceive the therapy as easy to use. On the other hand, in settings with a weaker organizational structure, outside support from a clinical supervisor led to community health volunteers in Kenya perceiving the therapy as easy to use. The findings from this article can guide context-specific recommendations for increasing perceived therapy feasibility at the provider-, organization-, and policy levels.

Toward Efficient, Sustainable, and Scalable Methods of Treatment Characterization: An Investigation of Coding Clinical Practice from Chart Notes

PURPOSE

Chart notes provide a low-cost data source that could help characterize what occurs in treatment with sufficient precision to improve management of care. This study assessed the interrater reliability of treatment content coded from chart notes and evaluated its concordance with content coded from transcribed treatment sessions.

METHOD

Fifty randomly selected and digitally recorded treatment events were transcribed and coded for practice content. Independent coders then applied the same code system to chart notes for these same treatment events.

ANALYSIS

We measured reliability and concordance of practice occurrence and extensiveness at two levels of specificity: practices (full procedures) and steps (subcomponents of those procedures).

RESULTS

For chart notes, practices had moderate interrater reliability (M k = 0.50, M ICC = 0.56) and steps had moderate (M ICC = 0.74) to substantial interrater reliability (M k = 0.78). On average, 2.54 practices and 5.64 steps were coded per chart note and 4.53 practices and 13.10 steps per transcript. Across sources, ratings for 64% of practices and 41% of steps correlated significantly, with those with significant correlations generally demonstrating moderate concordance (practice M r = 0.48; step M r = 0.47). Forty one percent of practices and 34% of steps from transcripts were also identified in the corresponding chart notes.

CONCLUSION

Chart notes provide an accessible data source for evaluating treatment content, with different levels of specificity posing tradeoffs for validity and reliability, which in turn may have implications for chart note interfaces, training, and new metrics to support accurate, reliable, and efficient measurement of clinical practice.

Consultation as an Implementation Strategy to Increase Fidelity of Measurement-Based Care Delivery in Community Mental Health Settings: An Observational Study

Regularly administering outcome measures to clients to inform clinical decision making, referred to as measurement-based care (MBC), has the potential to improve mental health treatment due to its applicability across disorders and treatment settings. Given the utility of MBC, understanding predictors of high-fidelity MBC implementation is important. Training and consultation are widely used implementation strategies to increase the fidelity of evidence-based practice delivery but have rarely been studied with MBC. The current study will examine the relationship between time clinicians spent discussing a case in consultation ("dosage") and MBC fidelity. Thirty clinicians and 56 youth were in the MBC condition of a randomized controlled trial and completed baseline questionnaires. Consultation dosage was extracted from call notes. MBC fidelity was measured using the implementation index, which combines rates of administering and viewing questionnaires, using objective data from the online MBC system. Multi-level modeling was used. Greater consultation dosage significantly predicted a higher implementation index (ß = 0.27, SE = 0.06, p < .001). For every 30 min spent discussing a case in consultation, the case's MBC fidelity increased by 8.1%. Greater consultation dosage significantly predicted higher rates of administration (ß = 0.07, SE = 0.03, p = 0.033) and feedback report viewing (ß = 0.24, SE = 0.06, p < .001). More consultation at the case level predicts greater MBC fidelity using objective measures. The greatest impact was increasing the clinician feedback report viewing rate. The results of this study can help inform future efforts to increase the fidelity with which MBC is delivered and to make consultation as efficient and effective as possible.

Setting and Provider Predictors of Implementation Success for an Eating Disorder Prevention Program Delivered by College Peer Educators

INTRODUCTION

College students face increased risk for a variety of mental health problems but experience barriers to treatment access. Prevention programs, including those implemented by peer educators, may decrease treatment needs and increase service access. We examined the implementation of an evidence-based eating disorder prevention program, Body Project, delivered by college peer educators at 63 colleges/universities, comparing three levels of implementation support: (1) Train-the-Trainer (TTT) training; (2) TTT plus a technical assistance workshop (TTT + TA); and (3) TTT + TA with one year of quality assurance calls (TTT + TA + QA). The present study tested the degree to which indicators proposed by the Consolidated Framework for Implementation Research (CFIR) were associated with core implementation outcomes.

METHOD

We tested whether indices of CFIR domains (i.e., perceived intervention characteristics, outer and inner setting factors, provider characteristics, and implementation process) were correlated with three implementation outcomes (program reach, fidelity, effectiveness) during a 1-year implementation period.

RESULTS

Greater program reach was associated with implementation process, specifically the completion of more implementation activities (β = 0.46). Greater program fidelity was associated with higher positive (β = 0.44) and lower negative (β = - 0.43) perceptions of the Body Project characteristics, and greater reported general support for evidence-based practices (β = 0.41). Greater effectiveness was associated with lower negative perceptions of Body Project characteristics (d = 0.49).

CONCLUSIONS

Several implementation determinants proposed by the CFIR model predicted outcomes, especially intervention fidelity. Across the outcomes of interest, implementation determinants related to peer educator and supervisor perceived characteristics of the specific intervention and general attitudes towards evidence-based practices emerged as robust predictors to inform future work investigating ongoing implementation and sustainability of programs in university settings.

Supporting Emotionally Exhausted Community Mental Health Therapists in Appropriately Adapting EBPs for Children and Adolescents

Evidence-based practices (EBPs) are often adapted during community implementation to improve EBP fit for clients and the service context. Augmenting EBPs with additional dosing and content may improve fit. However, reducing EBP content can reduce EBP effectiveness. Using multilevel regression models, this study examined whether supportive program climate and program-furnished EBP-specific implementation strategies (e.g., materials, ongoing training, in-house experts) are associated with augmenting and reducing adaptations, and whether therapist emotional exhaustion moderated these associations. Data were collected from surveys completed by 439 therapists from 102 programs 9 years after a system-driven EBP implementation initiative. Supportive program climate was associated with more augmenting adaptations. Emotional exhaustion was a significant moderator. When organizations used more EBP-specific implementation strategies, more emotionally exhausted therapists reduced EBPs less and less emotionally exhausted therapists augmented EBPs more. Findings provide guidance on how organizations can support appropriate EBP adaptations in spite of therapist emotional exhaustion.

Enhancing Quality of Care Through Evidence-Based Practice: Training and Supervision Experiences

OBJECTIVE

Evidence-based practice (EBP) is the preferred approach to treatment in mental health settings because it involves the integration of the best available research, clinical expertise, and patient values to optimize patient outcomes. Training on empirically supported treatments (ESTs) in mental health settings is an important component of EBP, and supervision of therapists' implementation of ESTs is critical for therapists to develop and maintain a strong EBP skill set. This study aimed to evaluate training and supervision histories of therapists in outpatient and inpatient psychiatric care settings as an essential first step in improving patient outcomes.

METHODS

Electronic surveys were completed by 69 therapists, most of whom had a master's degree, within a psychiatry and behavioral sciences department at an academic institution. Participating therapists were recruited from several outpatient and inpatient mental health settings serving children, adolescents, and adults.

RESULTS

Although most therapists reported completing some form of EST-related coursework, a majority did not receive any supervision related to implementation of ESTs (51% for cognitive-behavioral therapy cases, 76% for dialectical behavior therapy cases, and 52% for other EST cases) during graduate and postgraduate training.

CONCLUSIONS

Although research from the past decade has supported the need for improvements in training on ESTs, and especially in supervision, problems related to limited exposure to training and supervision among therapists still exist. These findings have implications for how mental health centers can evaluate staff members' EST training and supervision experiences, training needs, and associated training targets to improve the quality of routine care.

Optimizing outcomes, mechanisms, and recall of Cognitive Therapy for depression: Dose of constructive memory support strategies

OBJECTIVE

Poor memory for treatment is associated with worse patient outcomes. Therapist use of constructive memory support strategies, which help patients actively engage with treatment content, may improve patient memory for treatment. We sought to identify the dose of constructive memory support needed to optimize treatment outcomes, mechanisms, and patient recall.

METHOD

Adults with major depressive disorder (N = 178, mean age = 37.9, 63% female, 17% Hispanic or Latino/a) were randomized to Cognitive Therapy plus a Memory Support Intervention or Cognitive Therapy as usual. Because therapists from both groups used constructive memory support, treatment conditions were combined to maximize data. Depression and overall impairment were assessed before treatment, immediately post-treatment (POST), and six (6FU) and 12 months (12FU) after treatment. Patients completed measures of treatment mechanisms - utilization/competency in Cognitive Therapy skills - and treatment recall at POST, 6FU, and 12FU. Patient adherence to treatment was averaged across sessions.

RESULTS

Using Kaplan-Meier Survival Analyses, the optimal dose of constructive memory support was eight uses per session (sensitivity analysis range: 5-12 uses). Pre-treatment depression symptoms and patient perceptions of treatment may impact the optimal dose.

CONCLUSION

Eight uses of constructive memory support by therapists per session may optimize treatment outcomes, mechanisms, and recall over the long-term.

Taking a Magnifying Glass to Measurement-Based Care Consultation Sessions: with What Issues Do Mental Health Clinicians Struggle?

PURPOSE

Ongoing consultation following initial training is one of the most commonly deployed implementation strategies to facilitate uptake of evidence-based practices, such as measurement-based care (MBC). Group consultation provides an interactive experience with an expert and colleagues to get feedback on actual issues faced, yet there is little research that unpacks the questions raised in consultation and what types of issues are important to address.

METHODS

The current study characterized the questions and concerns raised by community mental health clinicians (N = 38 across six clinics) during group consultation sessions completed as part of an MBC implementation trial. We conducted a qualitative content analysis of consultation forms completed by clinicians before each MBC consultation session.

RESULTS

Clinicians sought MBC consultation for clients across a range of ages and levels of depression severity. Qualitative results revealed five main questions and concerns in consultation sessions: (1) how to administer the PHQ-9, (2) how to review PHQ-9 scores, (3) how to respond to PHQ-9 score, (4) the types of clients for whom MBC would be appropriate, and (5) how MBC could impact a clinician's usual care.

CONCLUSION

Findings highlight the need for ongoing consultation and limitations of workshop training alone. Practical recommendations for addressing the common questions and concerns identified are presented to support MBC use.

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy

BACKGROUND

Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers' implementation of evidence-based psychosocial interventions-(1) audit and feedback and (2) active learning.

METHODS

Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples.

RESULTS

In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ2=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ2=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples.

CONCLUSIONS

Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04096274 . Registered on 19 September 2019.

Training home visitors in mentalization-based practice: A qualitative case study of clinical supervision in mothering from the inside out

Attachment-based interventions are important for improving parent-child outcomes. These interventions must be scaled and made available to under-resourced communities. An important part of scaling these interventions is delineating and reproducing high-quality training, including clinical training which often requires the completion of a supervised case. However, descriptions and guidelines for clinical training are frequently broad or not available in the literature. A detailed description of clinical training could lead to further research to improve the effectiveness and dissemination of evidence-based interventions. Mothering from the Inside Out (MIO) is an attachment-based parenting intervention effective at reducing substance use and depression, improving caregiving, and enhancing child attachment. It is now being brought from research to community settings. This paper outlines the didactic and clinical training components of MIO. We then present a qualitative case study of one community-based counselor participating in the clinical training of MIO and employ qualitative methods to describe the main themes that arose during the training. We aim to illustrate how the trainer assisted the counselor in implementing the core components of MIO, which included (a) refining the language used in MIO sessions, (b) making space to explore mental states, and (c) addressing trauma. We conclude by presenting the implications of these findings.

Cross-training needs among community-based clinicians in HIV and substance use

BACKGROUND

People with double burden of HIV and substance use have poorer treatment engagement and worse treatment outcomes. Cross-training of HIV and substance use disorder clinicians can potentially enhance the scale up and quality of integrated care. Research is needed on clinicians' areas of greatest training need in order to inform training development.

METHODS

Data from semi-structured individual interviews with 16 HIV and 13 substance use disorder clinicians (N = 29) examining clinician perspectives on their training needs were analyzed using thematic analysis focused on both a priori and emergent subthemes.

RESULTS

Several key emergent subthemes were identified across the a priori themes of: 1) past training experiences; 2) gaps in training; and 3) training and supervision format/structure. Both HIV and substance use clinicians reported they had received minimal formal cross-training and had mostly been trained "on the job." Clinicians also emphasized gaps in training regarding sensitivity and anti-stigma, the latest medications for opioid use disorder, and HIV prevention/treatment and referral resources. Regarding training and supervision format, clinicians cited didactic workshops and ongoing supervision as appealing strategies.

CONCLUSIONS

Results show that lack of formal and updated training for clinicians is an important gap in providing integrated HIV and substance use treatment. Didactic workshops and ongoing support strategies that address stigma, medications for HIV and substance use disorder, and referral resources are likely to be particularly valuable.

Correspondence between practitioners’ self-assessment and independent motivational interviewing treatment integrity ratings

As evaluation of practitioners’ competence is largely based on self-report, accuracy in practitioners’ self-assessment is essential for ensuring high quality treatment-delivery. The aim of this study was to assess the relationship between independent observers’ ratings and practitioners’ self-reported treatment integrity ratings of Motivational interviewing (MI). Practitioners (N = 134) were randomized to two types of supervision [i.e., regular institutional group supervision, or individual telephone supervision based on the MI Treatment Integrity (MITI) code]. The mean age was 43.2 years (SD = 10.2), and 62.7 percent were females. All sessions were recorded and evaluated with the MITI, and the MI skills were self-assessed with a questionnaire over a period of 12 months. The associations between self-reported and objectively assessed MI skills were overall weak, but increased slightly from baseline to the 12-months assessment. However, the self-ratings from the group that received monthly objective feedback were not more accurate than those participating in regular group supervision. These results expand findings from previous studies and have important implications for assessment of practitioners’ treatment fidelity: Practitioners may learn to improve the accuracy of self-assessment of competence, but to ensure that patients receive intended care, adherence and competence should be assessed objectively.

Measuring Evidence-Based Treatment Strategies in Youth Community Mental Health Care: the Evidence-Based Strategies Scale

While there are established measures for fidelity to specific evidence-based treatments, there is no widely accepted, feasible measure of the use of evidence-based treatment strategies in youth mental health (MH) care. This study examined the factor structure of a provider self-report measure of evidence-based treatment strategy use, the Evidence-Based Strategies Scale (EBSS). MH providers completed the EBSS as part of a larger mailed survey. The factor structure of the EBSS was examined using exploratory factor analysis in a national, multidisciplinary sample of MH providers (N = 1092), and confirmatory factor analysis was subsequently conducted to replicate this factor structure in a state-wide, multidisciplinary sample of Medicaid MH providers (N = 780). Findings indicated a three-factor structure, representing working alliance, youth-focused, and family-focused evidence-based treatment strategies factors. The EBSS offers a potential method for measuring the evidence-based treatment strategies being delivered in youth community MH care, though more research is needed.

A Behavior-based Coding Tool for Assessing Supervisors' Adherence and Competence: Findings From a Motivational Interviewing Implementation Study

Supervision seems to be an essential part of therapist training, and thus also of implementing evidence-based practices. However, there is a shortage of valid and reliable instruments for objective assessment of supervision competence that include both global measures and frequency counts of behavior - two essential aspects of supervisory competence. This study tests the internal consistency and inter-rater reliability of an assessment tool that includes both these measures. Additionally, strategies and techniques used by ten supervisors in 35 Motivational interviewing supervision sessions are described. Codings were conducted after two separate coding training sessions. The internal consistency across the global measures was acceptable (α = 0.70; 0.71). After the second training, the inter-rater reliabilities for all frequency counts were in the moderate to good range, except for two that were in the poor range; inter-rater reliability for one of the four global measures was in the moderate range, and three were in the poor range. A prerequisite for identifying specific supervisor skills central to the development of therapist skills, teaching these skills to supervisors, and performing quality assurance of supervision, is to create instruments that can measure these behaviors. This study is a step in that direction.

Testing a community developed training protocol for an evidence-based treatment

The implementation of evidence-based treatments (EBTs) in community behavioral health settings is a recommended practice, yet training experienced by community-based clinicians may require novel and creative training methods. The current study focused on creating a training protocol for Alternative for Families: a Cognitive-Behavioral Therapy from both evidence-based foundations and community-based agency feedback to promote better EBT integration into agencies. Twenty-four clinicians from three agencies were trained using a community-informed training protocol. Outcomes for clinician-reports of organizational functioning, self-reports of skill and knowledge, and observational single-subject data of clinician skills were assessed. Minimal improvements were found for clinician skills across self-report and observations. More research on tailoring trainings to meet needs of community agencies and clinicians should be explored to determine best practice in wide-scale implementation efforts.

Self-Coding of Fidelity as a Potential Active Ingredient of Consultation to Improve Clinicians' Fidelity

A key goal for implementation science is the identification of evidence-based consultation protocols and the active ingredients within these protocols that drive clinician behavior change. The current study examined clinicians' self-coding of fidelity as a potential active ingredient of consultation for the Attachment and Biobehavioral Catch-up (ABC) intervention. It also examined two other potential predictors of clinician fidelity in response to consultation: dosage of consultation and working alliance. Twenty-nine clinicians (97% female, 62% White, M age = 34 years) participated in a year of weekly fidelity-focused ABC consultation sessions, for which clinicians self-coded fidelity and received consultant feedback on both their coding and their fidelity. Data from the ABC fidelity measure were available for 1067 sessions coded by consultants, and clinicians' self-coding accuracy was calculated from 1044 sessions coded by both clinicians and consultants. Alliance was measured with the Working Alliance Inventory-Trainee and Supervisor Versions. The study was observational, and fidelity and self-coding accuracy were modeled across time using hierarchical linear modeling. Clinicians' ABC fidelity, as well as their self-coding accuracy, increased over the course of consultation. Clinicians' self-coding accuracy predicted their initial fidelity and growth in fidelity. Working alliance was also linked to fidelity and self-coding accuracy. These results suggest that clinician self-coding should be further examined as an active ingredient of consultation. The study has important implications for the design of consultation procedures and fidelity assessments.

Performance of a Supervisor Observational Coding System and an Audit and Feedback Intervention

Workplace-based clinical supervision is common in community based mental health care for youth and families and could be a leveraged to scale and improve the implementation of evidence-based treatment (EBTs). Accurate methods are needed to measure, monitor, and support supervisor performance with limited disruption to workflow. Audit and Feedback (A&F) interventions may offer some promise in this regard. The study-a randomized controlled trial with 60 clinical supervisors measured longitudinally for 7 months-had two parts: (1) psychometric evaluation of an observational coding system for measuring adherence and competence of EBT supervision and (2) evaluation of an experimental Supervisor Audit and Feedback (SAF) intervention on outcomes of supervisor adherence and competence. All supervisors recorded and uploaded weekly supervision sessions for 7 months, and those in the experimental condition were provided a single, monthly web-based feedback report. Psychometric performance was evaluated using measurement models based in Item Response Theory, and the effect of the SAF intervention was evaluated using mixed-effects regression models. The observational instrument performed well across psychometric indicators of dimensionality, rating scale functionality, and item fit; however, coder reliability was lower for competence than for adherence. Statistically significant A&F effects were largely in the expected directions and consistent with hypotheses. The observational coding system performed well, and a monthly electronic feedback report showed promise in maintaining or improving community-based clinical supervisors' adherence and, to a lesser extent, competence. Limitations discussed include unknown generalizability to the supervision of other EBTs.

Paraprofessional Youth Mentoring: A Framework for Integrating Youth Mentoring with Helping Institutions and Professions

The demand for child mental health services, including those provided by psychologists, counselors, and social workers, exceeds the supply. This trend is expected to continue or worsen unless there are substantial structural changes in how mental health services are provided. We propose a framework for paraprofessional youth mentors, defined as a subgroup of professionally supervised, non-expert volunteer or paid mentors to whom aspects of professional helping tasks are delegated. Our proposal is aligned with historical and modern solutions to scaling mental health services, and this framework could simultaneously increase the number of youth receiving evidence-based mental health services and reduce the burden on existing systems of care. The framework defines three plausible tasks for paraprofessional mentors: (1) reducing barriers to mental health service, (2) increasing engagement in services, and (3) providing direct services. The safety and effectiveness of these task-shifting efforts will hinge on competency-based training and evaluation, supervision by professionals, and documentation of services rendered, all of which the field of youth mentoring currently lacks. We describe several requisite scientific, institutional, and regulatory advances that will be necessary to realize this variant of youth mentoring for a subgroup of youth who are presenting for assistance with mental health problems.

The antecedents of leader-member-exchange (LMX) relationships in African context: the influence of the supervisor’s feedback delivery-tactic

Purpose

The current paper aims to analyse the antecedents of leader–member exchange relationships (LMX) by specifically focusing on the influence of the supervisor’s feedback delivery tactic.

Design/methodology/approach

This study uses qualitative research methods with primary interviews as the main data source. Primary interviews with 40 managers from top supermarkets in Nigeria, South Africa and the UK were undertaken.

Findings

The authors found that both high-quality positive feedback and constructive criticisms produced the same feelings – more positive interpersonal relationships with their supervisors, higher levels of commitment to their organisations, higher job satisfaction and thus, high-quality LMX relationships. Where criticisms were delivered without greater interpersonal treatment, feedback was perceived as negative, and participants revealed lack of job satisfaction, lack of commitment to their organisations, poor interpersonal relationship with their supervisors, high turnover intent and thus low-quality LMX relationship.

Originality/value

To the best of the authors’ knowledge, the current paper is one of the first studies to highlight the consequences of different feedback delivery tactics on subsequent LMX quality particularly in African context. The authors specifically develop a process-based model of enhancing high-quality LMX, which shows the role of the supervisor’s feedback delivery tactic in the process. The authors also develop a process-based model that illustrates how negative/unconstructive feedback could result in a low-quality LMX. Finally, to the best of the authors’ knowledge, this paper is also one of the first to offer a comparative assessment between African and British (the UK) empirical settings and highlight some interesting dynamics concerning LMX quality and role of supervisor’s feedback delivery tactic.

In Search of the Common Elements of Clinical Supervision: A Systematic Review

The importance of clinical supervision for supporting effective implementation of evidence-based treatments (EBTs) is widely accepted; however, very little is known about which supervision practice elements contribute to implementation effectiveness. This systematic review aimed to generate a taxonomy of empirically-supported supervision practice elements that have been used in treatment trials and shown to independently predict improved EBT implementation. Supervision practice elements were identified using a two-phase, empirically-validated distillation process. In Phase I, a systematic review identified supervision protocols that had evidence of effectiveness based on (a) inclusion in one or more EBT trials, and (b) independent association with improved EBT implementation in one or more secondary studies. In Phase II, a hybrid deductive-inductive coding process was applied to the supervision protocols to characterize the nature and frequency of supervision practice elements across EBTs. Twenty-one of the 876 identified articles assessed the associations of supervision protocols with implementation or clinical outcomes, representing 13 separate studies. Coding and distillation of the supervision protocols resulted in a taxonomy of 21 supervision practice elements. The most frequently used elements were: reviewing supervisees' practice (92%; n = 12), clinical suggestions (85%; n = 11), behavioral rehearsal (77%; n = 10), elicitation (77%; n = 10), and fidelity assessment (77%; n = 10). This review identified supervision practice elements that could be targets for future research testing which elements are necessary and sufficient to support effective EBT implementation. Discrepancies between supervision practice elements observed in trials as compared to routine practice highlights the importance of research addressing supervision-focused implementation strategies.

A Novel Peer-to-Peer Coaching Program to Support Digital Mental Health: Design and Implementation

Many individuals in need of mental health services do not currently receive care. Scalable programs are needed to reduce the burden of mental illness among those without access to existing providers. Digital interventions present an avenue for increasing the reach of mental health services. These interventions often rely on paraprofessionals, or coaches, to support the treatment. Although existing programs hold immense promise, providers must ensure that treatments are delivered with high fidelity and adherence to the treatment model. In this paper, we first highlight the tension between the scalability and fidelity of mental health services. We then describe the design and implementation of a peer-to-peer coach training program to support a digital mental health intervention for undergraduate students within a university setting. We specifically note strategies for emphasizing fidelity within our scalable framework, including principles of learning theory and competency-based supervision. Finally, we discuss future applications of this work, including the potential adaptability of our model for use within other contexts.

Clinical supervision approach predicts evidence-based trauma treatment delivery in children's mental health

OBJECTIVE

Observational studies of practices used in clinical supervision-as-usual can be leveraged to advance the limited research on workplace-based supervision as an evidence-based treatment (EBT) implementation strategy. This exploratory observational study examined the presence of supervision approaches (comprised of supervision techniques) and whether these predicted clinicians' EBT technique delivery.

METHODS

Participants included 28 supervisors, 70 clinician supervisees, and 60 youth clients and guardians from 17 public mental health organizations. Data included audio recorded supervision-as-usual sessions over 1 year, audio recorded Trauma-focused Cognitive Behavioral Therapy (TF-CBT) treatment sessions with youth for 6 months, and youth-reported post-traumatic stress severity scores. Audio recordings of 438 supervision sessions were coded for session duration and the presence of 13 supervision techniques and intensity of their coverage. Audio recordings of 465 treatment sessions were coded for presence and intensity of coverage of TF-CBT practice elements. Agglomerative hierarchical cluster analysis examined the presence of clusters of supervision technique use, termed supervision approaches. Generalized estimating equations estimated the relation between supervision approaches and delivery of TF-CBT elements.

RESULTS

Two supervision approaches were identified- Supportive-Directive and Supportive- that discriminated between use of five supervision techniques. Clinicians who received a higher proportion of supportive-directive supervision sessions had greater odds of delivering the trauma narrative with a client.

CONCLUSION

Findings suggest that patterns of supervision techniques can be identified and may shape EBT delivery. Supervision approaches show some evidence of being tailored to individual clinicians. Implications for the development of supervision implementation strategies and future directions are discussed.

Improving the feasibility of fidelity measurement for community-based quality assurance: Partial- versus full-session observations of supervisor adherence and competence

Background

Clinical supervision is a common quality assurance method for supporting the implementation and sustainment of evidence-based interventions (EBIs) in community mental health settings. However, assessing and supporting supervisor fidelity requires efficient and effective measurement methods. This study evaluated two observational coding approaches that are potentially more efficient than coding full sessions: a randomly selected 15-min segment and the first case discussion of the session.

Method

Data were leveraged from a randomized trial of an Audit and Feedback (A&F) intervention for supervisor Adherence and Competence. Supervisors ( N = 57) recorded and uploaded weekly group supervision sessions for 7 months, with one session observationally coded each month ( N = 374). Of the coded sessions, one was randomly selected for each supervisor, and a random 15-min segment was coded. Additionally, the first case discussion was coded for the full sample of sessions.

Results

Across all models (and controlling for the proportion of the session covered by the partial observation), Adherence and Competence scores from partial observations were positively and significantly associated with scores from full sessions. In all cases, partial observations were most accurate when the level of Adherence and Competence was moderate. At lower levels, partial observations were underestimates, and at higher levels, they were overestimates.

Conclusions

The results suggest that efficient observational measurement can be achieved while retaining a general level of measurement effectiveness. Practically, first-case discussions are easier to implement, whereas 15-min segments have fewer potential threats to validity. Evaluation of resource requirements is needed, along with determining whether A&F effects are retained if feedback is based on partial observations. Nevertheless, more efficient observational coding could increase the feasibility of routine fidelity monitoring and quality assurance strategies, including A&F, which ultimately could support the implementation and sustainment of effective supervision practices and EBIs in community practice settings. Plain Language Summary: When delivering evidence-based mental health interventions in community-based practice settings, a common quality assurance method is clinical supervision. To support supervisors, assessment methods are needed, and those methods need to be both efficient and effective. Ideally, supervision sessions would be recorded, and trained coders would rate the supervisor’s use of specific strategies. In most settings, though, this requires too many resources. The present study evaluated a more efficient approach. The data came from an existing randomized trial of an Audit and Feedback intervention for enhancing supervisor Adherence and Competence. This included 57 supervisors and 374 sessions across seven months of monitoring. Instead of rating full supervision sessions, a more efficient approach was to have coders rate partial sessions. Two types of partial observations were considered: a randomly selected 15-minute segment of the session and the first case discussion of the session. The aim was to see if partial observations and full observations led to similar conclusions about Adherence and Competence. In all cases, they did. The scores were most similar for sessions with moderate levels of Adherence and Competence. If Adherence and Competence were low, partial observations were underestimates, but if they were high, partial observations were overestimates. Observing partial sessions is more efficient, but in terms of accuracy, the benefits and limitations should be evaluated in light of how the scores will be used. Additionally, future research should consider whether Audit and Feedback interventions have the same effect if feedback is based on observations of partial sessions.

"I see your punitive measure and I raise you a person-centered bar": Supervisory Strategies to Promote Adoption of Person-Centered Care

Person-centered care remains a high priority within community mental health services. Clinical supervision is an embedded resource for professional development and promotion of high quality care. This study examined supervisory strategies during the implementation of person-centered care planning (PCCP) across two northeastern US States. A criterion sample of supervisor-provider teams participated in qualitative interviews (N = 34) and direct observation from 2016 to 2017. Modified grounded theory analyses were conducted and three supervisory strategies were identified. Supervisory attunement to providers (knowing their audience), active collaborative engagement with providers (practicing together), and infusing reminders and opportunities for feedback (chipping away) were critical strategies to engage providers in adopting PCCP. These strategies changed providers' practice patterns by improving supervisors' calibration to dynamic contextual and individual needs during implementation and communicating supervisors' expectations of PCCP enactment. Workplace-based clinical supervision holds promise as a key intervention point to embed high quality care.

Role-Play Assessment of Therapist Adherence and Skill in Implementation of Trauma-Focused Cognitive-Behavioral Therapy

Numerous efforts are underway to train clinicians in evidence-based practices. Unfortunately, the field has few practical measures of therapist adherence and skill with which to judge the success of these training and implementation efforts. One possible assessment method is using behavioral rehearsal, or role-play, as an analogue for therapist in-session behavior. The current study describes aspects of reliability, validity and utility of a behavioral role-play assessment developed to evaluate therapist adherence and skill in implementing Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT). TF-CBT role-play assessments were conducted with a sample of 43 therapists as part of a larger training study. The TF-CBT role-play assessments were independently coded for TF-CBT adherence and skill by a certified TF-CBT trainer and three clinical psychology doctoral students. Findings indicated good interrater reliability for the individual items (ICC: M = .71, SD = .15). Regarding utility, 67.19% (n = 43/64) of contacted therapists completed the role-play assessment, which took an average of 30 min (M = 31.42, SD = 5.65) to complete and 60 min (M = 62.84, SD = 11.31) to code. Therapists with a master's degree were more likely to complete the role-play assessment than those with other degrees but no other differences in demographic variables, practice characteristics, or TF-CBT knowledge or training were found between participants and nonparticipants. Role-play assessments may offer an alternative to observational coding for assessing therapist adherence and skill, particularly in contexts where session recordings are not feasible.

Challenging rigidity in Anorexia (treatment, training and supervision): questioning manual adherence in the face of complexity

BACKGROUND

Anorexia Nervosa is a debilitating illness. While there have been many advancements to treatment protocols and outcomes for people with eating disorders, the field acknowledges there remains considerable room for improvement. This timely Special Edition of the Journal of Eating Disorders has invited those of us in the field to consider a range of topics in aid of this task, including potential modifications and implementation of evidence-based practice, specific and common psychotherapy factors, treatment manuals, adherence and individualising treatment approaches for individuals and families. BODY: In this paper, we briefly outline the key manualised treatments currently available to treat children, adolescents and adults with Anorexia Nervosa, considering the benefits, potential reasons for adaptations and limitations. We then review the current evidence for training strict adherence to treatment manuals which is often a key focus in training and supervision, questioning the association of increased treatment adherence with improved therapeutic outcome. We then summarise some key evidence behind other therapeutic factors which have been demonstrated to affect outcome regardless of which manual is implemented, such as readiness to change and therapeutic alliance.

CONCLUSION

The paper concludes with implications and considerations for future research, clinical guidelines, training and supervision, highlighting the need to consider the therapeutic relationship and processes alongside manual content to conduct best evidence-informed practice.

Family-based treatment for children and adolescents with eating disorders: a mixed-methods evaluation of a blended evidence-based implementation approach

In this study, we evaluated a blended implementation approach with teams learning to provide family-based treatment (FBT) to adolescents with eating disorders. Four sites participated in a sequential mixed method pre-post study to evaluate the implementation of FBT in their clinical settings. The implementation approach included: (a) preparatory site visits; (b) the establishment of implementation teams; (c) a training workshop; (d) monthly clinical consultation; (e) monthly implementation consultation; and (f) fidelity assessment. Quantitative measures examining attitudes toward evidence-based practice, organizational learning environment and organizational readiness for change, as well as, individual readiness for change were delivered pre- and postimplementation. Correlational analyses were used to examine associations between baseline variables and therapist fidelity to FBT. Fundamental qualitative description guided the sampling and data collection for the qualitative interviews performed at the conclusion of the study. Seventeen individuals participated in this study (nine therapists, four medical practitioners, and four administrators). The predetermined threshold of implementation success of 80% fidelity in every FBT session was achieved by only one therapist. However, mean fidelity scores were similar to those reported in other studies. Participant attitudes, readiness, and self-efficacy were not associated with fidelity and did not change significantly from pre- to postimplementation. In qualitative interviews, all participants reported that the implementation intervention was helpful in adopting FBT. Our blended implementation approach was well received by participants. A larger trial is needed to determine which implementation factors predict FBT fidelity and impact patient outcomes.

Virtual Standardized Patients for Mental Health Education

PURPOSE OF REVIEW

The training of psychiatrists and other mental health professionals requires education on a range of interpersonal, communication, and psychotherapy techniques. Classroom and workshop training must be augmented by experiential learning with feedback for skill implementation with fidelity. Virtual standardized patients (VSPs) are computerized conversational agents that can support experiential learning through standardized, consequence-free training environments at reduced costs.

RECENT FINDINGS

Research on mental health VSPs is rife with feasibility and acceptability pilot studies across various training populations and settings. Users have generally reported positive reactions to training with VSPs, though frustrations with some VSP speech recognition or VSP response relevance has been reported. Several studies have demonstrated a promising transfer of clinical skills from VSP training to human standardized patients and randomized trials supporting improved skill relative to reading or academic study are encouraging. As technology improves and natural language processing and accurate computer response generation for broad ranging conversational topics emerges, the field would benefit from research on the characteristics of effective VSPs for a range of purposes and trainee populations. Well-designed randomized evaluations of VSPs relative to best practices in education are needed, particularly regarding the impact of VSPs on clinical practice among actual patients.

Clinical Supervision of Mental Health Services: A Systematic Review of Supervision Characteristics and Practices Associated with Formative and Restorative Outcomes

In this review, the authors examined supervision characteristics and practices associated with formative (e.g., skill development) and restorative (e.g., well-being) provider outcomes. We used qualitative review to summarize supervision characteristics associated with desired outcomes. Then, we applied a distillation approach (Chorpita et al., 2005) to identify practices associated with formative and restorative outcomes. The most common practices for promoting formative outcomes were corrective feedback, discussing intervention, and role play. Findings indicate several supervision strategies have demonstrated empirical support for improving formative outcomes. However, more rigorous research is needed in community settings, particularly for understanding which strategies improve restorative outcomes.

Clinical Supervision in Community Mental Health: Characterizing Supervision as Usual and Exploring Predictors of Supervision Content and Process

Clinical supervision can be leveraged to support implementation of evidence-based practices in community mental health settings, though it has been understudied. This study focuses on 32 supervisors at 23 mental health organizations in Philadelphia. We describe characteristics of supervisors and organizations and explore predictors of supervision content and process. Results highlight a low focus on evidence-based content and low use of active supervision processes. They underscore the need for further attention to the community mental health context when designing supervision-targeted implementation strategies. Future work should assess whether supervision models specific to community mental health are needed.

Making Implementation Last: The Impact of Training Design on the Sustainability of an Evidence-Based Treatment in a Randomized Controlled Trial

Although advances have been made in facilitating the implementation of evidence-based treatments, little is known about the most effective way to sustain their use over time. The current study examined the sustainability of one evidence-based treatment, Parent-Child Interaction Therapy (PCIT), following a statewide implementation trial testing three training methods: Cascading Model, Learning Collaborative, and Distance Education. Participants included 100 clinicians and 50 administrators from 50 organizations across Pennsylvania. Clinicians and administrators reported on sustainability at 24-months, as measured by the number of clients receiving PCIT and the continued use of the PCIT protocol. Multi-level path analysis was utilized to examine the role of training on sustainability. Clinicians and administrators reported high levels of sustainability at 24-months. Clinicians in the Cascading Model reported greater average PCIT caseloads at 24-months, whereas clinicians in the Learning Collaborative reported greater full use of the PCIT protocol at 24-months. Attending consultation calls was associated with delivering PCIT to fewer families. Implications for the sustainable delivery of PCIT beyond the training year as well as for the broader field of implementation science are discussed.

Strategies that Promote Therapist Engagement in Active and Experiential Learning: Micro-Level Sequential Analysis

Therapists' active learning increases treatment fidelity, but research is needed on supervisory strategies to engage therapists in active learning. This study used sequential analysis to examine consultant behaviors associated with increased and decreased probability of eliciting therapists' active learning. The study included 162 consultation sessions from 27 community therapists implementing Attachment and Biobehavioral Catch-up. Consultants' client discussion, information provision, and modeling were associated with reduced likelihood of active learning. Consultants' questions, engagement in active learning strategies, use of video, and silence were associated with greater likelihood of therapist active learning. These findings inform supervisors' attempts to encourage active learning.

Changes in Community Clinicians' Attitudes and Competence following a Transdiagnostic Cognitive Behavioral Therapy Training

Background

Although the literature suggest that attitudes toward evidence-based practices (EBPs) are associated with provider use of EBPs, less is known about the association between attitudes and how competently EBPs are delivered. This study examined how initial attitudes and competence relate to improvements in attitudes and competence following EBP training.

Methods

Program evaluation data was collected during implementation of an EBP in a large community mental health network. Clinicians (N=891) received intensive training in cognitive behavioral therapy followed by six months of consultation. Attitudes were assessed using the Evidence-Based Practice Attitude Scale, and competence was assessed using the Cognitive Therapy Rating Scale. Data were analyzed by fitting three latent change score models to examine the relationship between changes in attitudes and competence across the training and within its two phases (workshop phase, consultation phase).

Results

Latent change models identified significant improvement in attitudes (Ms _latent change≥1.03, SEs≤ 0.18, zs≥6.55, ps< .001) and competence (Ms _{latent change} ≥14.16, SEs≤3.10, zs≥2.82, ps<.001) across the full training and in each phase. Higher pre-workshop attitudes predicted significantly greater change in competence in the workshop and across training (bs≥1.62, SEs≤0.90, z≥1.09, p<.04, β≥0.10); however, contrary to our hypothesis, post-workshop attitudes did not significantly predict change in competence in consultation (b=1.62, SE=0.86, z=1.87, p=.06, β=0.09). Change in attitudes and change in competence in the training period and within the two phases were not significantly correlated.

Conclusions

Results indicate that pre-training attitudes about EBPs present a target for implementation interventions, given their relation to changes in both attitudes and competence throughout training. Following participation in in itial training workshops, other factors such as subjective norms, implementation culture, or system-level policy shifts may be more predictive of change in competence through consultation.

Are you sitting (un)comfortably? Action-based supervision and supervisory drift

Action-based methods such as behavioural experiments, role-play and (by extension) ‘chairwork’ are powerful techniques recommended in core supervisory texts for cognitive behavioural therapy (CBT). Despite this, experiential methods are seldom used by supervisors, suggesting that supervision often drifts from a ‘doing process’ to a ‘talking process’. A number of factors contribute to this divergence from best practice, including limited confidence and a lack of familiarity with experiential procedures amongst supervisors. To address this, the current paper presents a variety of action-based techniques for enhancing supervisees’ technical, perceptual, interpersonal, reflective and personal competencies. Behavioural experiments, empty-chair, multi-chair and role-playing exercises for maintaining treatment fidelity, enhancing empathic attunement, repairing therapeutic ruptures, resolving impasses and working through negative countertransference are described, amongst others. Further research is needed to establish the nature and extent of supervisory drift, as well as the efficacy of action-based methods.

Key learning aims

As a result of reading this paper, readers should:(1)

Effectiveness of training methods for delivery of evidence-based psychotherapies: a systematic review

BACKGROUND

Extensive efforts have been made to train mental health providers in evidence-based psychotherapies (EBPs); there is increasing attention focused on the methods through which providers are trained to deliver EBPs. Evaluating EBP training methods is an important step in determining which methods are most effective in increasing provider skill and improving client outcomes.

METHODS

We searched MEDLINE (Ovid) and PsycINFO for randomized controlled trials published from 1990 through June 2019 that evaluated EBP training methods to determine the effectiveness of EBP training modalities on implementation (provider and cost) and client outcomes. Eligible studies (N = 28) were evaluated for risk of bias, and the overall strength of evidence was assessed for each outcome. Data was extracted by a single investigator and confirmed by a second; risk of bias and strength of evidence were independently rated by two investigators and determined by consensus.

RESULTS

Overall, EBP training improved short-term provider satisfaction, EBP knowledge, and adherence compared to no training or self-study of training materials (low to moderate strength of evidence). Training in an EBP did not increase treatment adoption compared to no training or self-study. No specific active EBP training modality was found to consistently increase provider EBP knowledge, skill acquisition/adherence, competence, adoption, or satisfaction compared to another active training modality. Findings were mixed regarding the additive benefit of post-training consultation on these outcomes. No studies evaluated changes in provider outcomes with regards to training costs and few studies reported on client outcomes.

LIMITATIONS

The majority of included studies had a moderate risk of bias and strength of evidence for the outcomes of interest was generally low or insufficient. Few studies reported effect sizes. The ability to identify the most effective EBP training methods was limited by low strength of evidence for the outcomes of interest and substantial heterogeneity among studies.

CONCLUSIONS

EBP training may have increased short-term provider satisfaction, EBP knowledge, and adherence though not adoption. Evidence was insufficient on training costs and client outcomes. Future research is needed on EBP training methods, implementation, sustainability, client outcomes, and costs to ensure efforts to train providers in EBPs are effective, efficient, and durable.

TRIAL REGISTRATION

The protocol for this review is registered in PROSPERO (CRD42018093381).