Annual Research Review: The state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): a modified Delphi study

BACKGROUND

Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources.

METHOD

MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability and inclusion (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis.

RESULTS

In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application.

DISCUSSION

MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

Determinants of Exposure Therapy Implementation in Clinical Practice for the Treatment of Anxiety, OCD, and PTSD: A Systematic Review

Exposure therapy (ET) forms a vital part of effective psychotherapy for anxiety-related presentations including anxiety disorders, obsessive-compulsive disorder (OCD), and post-traumatic stress disorder (PTSD), and is often underutilised in clinical practice. Using the Theoretical Domains Framework (TDF), this systematic review synthesised existing literature on the determinants of ET implementation for anxiety-related presentations and examined differences across presentations and developmental subgroups. Fifty-two eligible studies were assessed using the Mixed Methods Appraisal Tool, with 389 results (99%) mapped onto the TDF. Results suggested that clinicians' negative beliefs about the consequences of ET were commonly associated with reduced implementation. It also appeared that whilst broad unspecified ET training may be related to improved implementation for anxiety disorders; greater implementation for complex presentations (i.e., PTSD) likely requires more specialised training involving practical components. A subset of domains (e.g., social/professional role and identity) accounted for most results, whilst some remain unexplored (i.e., optimism; reinforcement; memory, attention, and decision processes) or underexplored (i.e., behavioural regulation). Likewise, specific presentations and developmental subgroups (i.e., PTSD and adults) represented a greater proportion of results in the literature than others (i.e., OCD and youth). Future research exploring ET implementation, across specific presentations and developmental subgroups, would benefit from integrating implementation science frameworks to guide the development of targeted, comprehensive strategies to close the research-practice gap of ET for the treatment of anxiety-related presentations.

Future Directions for Community-Engaged Research in Clinical Psychological Science with Youth

Despite advances in clinical science, the burden of mental health problems among youth is not improving. To tackle this burden, clinical science with youth needs methods that include youth and family perspectives on context and public health. In this paper, we illustrate how community-engaged research (CEnR) methods center these perspectives. Although CEnR methods are well-established in other disciplines (e.g. social work, community psychology), they are underutilized in clinical science with youth. This is due in part to misperceptions of CEnR as resource-intensive, overly contextualized, incompatible with experimentally controlled modes of inquiry, or irrelevant to understanding youth mental health. By contrast, CEnR methods can provide real-world impact, contextualized clinical solutions, and sustainable outcomes. A key advantage of CEnR strategies is their flexibility-they fall across a continuum that centers community engagement as a core principle, and thus can be infused in a variety of research efforts, even those that center experimental control (e.g. randomized controlled trials). This paper provides a brief overview of this continuum of strategies and its application to youth-focused clinical science. We then discuss future directions of CEnR in clinical science with youth, as well as structural changes needed to advance this work. The goals of this paper are to help demystify CEnR and encourage clinical scientists to consider adopting methods that better consider context and intentionally engage the communities that our work seeks to serve.

Effects of Training in Cognitive Behavioural Therapy and Motivational Interviewing on Mental Health Practitioner Behaviour: A Systematic Review and Meta-Analysis

Effective training of mental health professionals is crucial for bridging the gap between research and practice when delivering cognitive behavioural therapy (CBT) and motivational interviewing (MI) within community settings. However, previous research has provided inconclusive evidence regarding the impact of training efforts. The current study aimed to systematically search, review and synthesize the literature on CBT and MI training to assess its effect on practitioner behavioural outcomes. Following prospective registration, a literature search was conducted for studies where mental health practitioners were exposed to training in face-to-face CBT or MI, reporting on at least one quantitative practitioner behavioural outcome. A total of 116 studies were eligible for the systematic review, and 20 studies were included in four meta-analyses. The systematic review highlights the need to establish psychometrically valid outcome measures for practitioner behaviour. Results of the meta-analyses suggest that training has a greater effect on practitioner behaviour change compared to receiving no training or reading a treatment manual. Training combined with consultation/supervision was found to be more effective than training alone, and no differences were found between face-to-face and online training. Results should be interpreted with caution due to methodological limitations in the primary studies, large heterogeneity, and small samples in the meta-analyses. Future directions are discussed.

How can a behavioral economics lens contribute to implementation science?

BACKGROUND

Implementation science in health is an interdisciplinary field with an emphasis on supporting behavior change required when clinicians and other actors implement evidence-based practices within organizational constraints. Behavioral economics has emerged in parallel and works towards developing realistic models of how humans behave and categorizes a wide range of features of choices that can influence behavior. We argue that implementation science can be enhanced by the incorporation of approaches from behavioral economics. Main body First, we provide a general overview of implementation science and ways in which implementation science has been limited to date. Second, we review principles of behavioral economics and describe how concepts from BE have been successfully applied to healthcare including nudges deployed in the electronic health record. For example, de-implementation of low-value prescribing has been supported by changing the default in the electronic health record. We then describe what a behavioral economics lens offers to existing implementation science theories, models and frameworks, including rich and realistic models of human behavior, additional research methods such as pre-mortems and behavioral design, and low-cost and scalable implementation strategies. We argue that insights from behavioral economics can guide the design of implementation strategies and the interpretation of implementation studies. Key objections to incorporating behavioral economics are addressed, including concerns about sustainment and at what level the strategies work.

CONCLUSION

Scholars should consider augmenting implementation science theories, models, and frameworks with relevant insights from behavioral economics. By drawing on these additional insights, implementation scientists have the potential to boost efforts to expand the provision and availability of high quality care.

Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial

BACKGROUND

Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity.

METHODS

Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses.

RESULTS

LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045).

CONCLUSIONS

LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

Enhancing Behavioral Health Implementation in a Care Coordination Program at a Federally Qualified Health Center: A Case Study Applying Implementation Frameworks

Federally Qualified Health Centers are charged with providing comprehensive health care in traditionally underserved areas, underscoring their importance in caring for and promoting health equity for the large portion of historically marginalized communities in this setting. There is a significant need to ensure Federally Qualified Health Centers are equipped to appropriately address the immense behavioral health needs common among patients served. Care coordination is an evidence-based model that is increasingly utilized in Federally Qualified Health Centers to improve care equity and outcomes. Addressing and supporting behavioral health needs is a key aspect of such care coordination models. Context-specific considerations and programmatic supports, particularly those that address the needs of care coordinators and the complex patients they serve, are needed to ensure such models can appropriately meet and address the behavioral health concerns of the diverse populations served. The goal of this study was to present a mixed-methods case study that systematically applies implementation frameworks to conduct a needs and context assessment to inform the development and testing of evidence-based practice strategies and implementation support as part of a care coordination program within a partnered Federally Qualified Health Center.

Supervisory Alliance as a Moderator of the Effects of Behavioral Rehearsal on TF-CBT Fidelity: Results from a Randomized Trial of Supervision Strategies

Exposure is an important element of treatment for many evidence-based treatments but can be challenging to implement. Supervision strategies to support exposure delivery may be an important tool to facilitate the use of exposure techniques; however, they must be considered and used in the context of the supervisory alliance. The present study examined relations between supervisory alliance and fidelity to the trauma narrative (TN; i.e., imaginal exposure) component of Trauma-Focused Cognitive Behavioral Therapy. We also examined how supervisory alliance moderated the effect of behavioral rehearsal use in supervision on TN fidelity. We analyzed data from a randomized controlled trial, in which forty-two supervisors and their clinicians (N = 124) from 28 Washington State community-based mental health offices participated. Clinicians were randomized to receive one of two supervision conditions-symptom and fidelity monitoring (SFM) or SFM with behavioral rehearsal (SFM + BR). Supervisory alliance alone did not predict delivery (i.e., occurrence) or extensiveness of delivery of the trauma narrative. Client-focused supervisory alliance moderated the effectiveness of behavioral rehearsal-as client-focused alliance increased, the odds of delivering the TN also increased significantly. Future research should further investigate how to appropriately match supervision techniques with supervisory dyads and explore the interplay of alliance with supervision techniques a supervisor might employ.

Action recommendations review in community-based therapy and depression and anxiety outcomes: a machine learning approach

BACKGROUND

While the positive impact of homework completion on symptom alleviation is well-established, the pivotal role of therapists in reviewing these assignments has been under-investigated. This study examined therapists' practice of assigning and reviewing action recommendations in therapy sessions, and how it correlates with patients' depression and anxiety outcomes.

METHODS

We analyzed 2,444 therapy sessions from community-based behavioral health programs. Machine learning models and natural language processing techniques were deployed to discern action recommendations and their subsequent reviews. The extent of the review was quantified by measuring the proportion of session dialogues reviewing action recommendations, a metric we refer to as "review percentage". Using Generalized Estimating Equations modeling, we evaluated the correlation between this metric and changes in clients' depression and anxiety scores.

RESULTS

Our models achieved 76% precision in capturing action recommendations and 71.1% in reviewing them. Using these models, we found that therapists typically provided clients with one to eight action recommendations per session to engage in outside therapy. However, only half of the sessions included a review of previously assigned action recommendations. We identified a significant interaction between the initial depression score and the review percentage (p = 0.045). When adjusting for this relationship, the review percentage was positively and significantly associated with a reduction in depression score (p = 0.032). This suggests that more frequent review of action recommendations in therapy relates to greater improvement in depression symptoms. Further analyses highlighted this association for mild depression (p = 0.024), but not for anxiety or moderate to severe depression.

CONCLUSIONS

An observed positive association exists between therapists' review of previous sessions' action recommendations and improved treatment outcomes among clients with mild depression, highlighting the possible advantages of consistently revisiting therapeutic homework in real-world therapy settings. Results underscore the importance of developing effective strategies to help therapists maintain continuity between therapy sessions, potentially enhancing the impact of therapy.

Four Steps To My Future (4STMF): acceptability, feasibility and exploratory outcomes of a universal school-based mental health and well-being programme, delivered to young adolescents in South Africa

OBJECTIVE

Mental health disorders affect many children in South Africa, where vulnerability is high, and treatment is limited. We sought to determine the feasibility and acceptability of a universally delivered classroom-based programme for the promotion of mental health in young adolescents.

METHOD

We pilot tested an 8 session, cognitive-behavioural therapy-based programme, 4 Steps To My Future (4STMF) in two schools. Participants were grade 5 learners (n = 222; Meanage  = 10.62 (Standard deviation = 0.69)). 4STMF was delivered in class time by trained psychology postgraduates. Feasibility (rates of parental opt-out, child assent, assessment completion at baseline and follow-up, programme completion, session attendance and programme fidelity), acceptability (teacher feedback and focus groups with learners), as well as demographic data and data on a battery of a psychological measures were collected at baseline, postintervention and at one-month follow-up.

RESULTS

Most eligible learners at both schools agreed to participate (85% - school 1; 91% - school 2) with more than 80% completing postintervention measures. Learner session attendance and programme fidelity were high. Teachers rated facilitators highly on confidence, preparedness, enthusiasm and classroom management and observed children to be enjoying the programme. Focus group data suggest that learners liked the programme, could recall the content and had shared some of the content with their family. An exploratory analysis of outcomes showed significant pre-post differences on self-esteem at school 1 and on emotion regulation at school 1 and school 2, maintained at follow-up.

CONCLUSIONS

This pilot study has shown that 4STMF can acceptably and feasibly be delivered, at classroom level, as a universal school-based prevention programme to young adolescent learners in South African primary schools. The programme could fit in with school context, could be delivered by nonspecialists, showed significant improvements on self-esteem and emotion regulation and was liked by the learners.

Attitudes Toward and Usage of Evidence-Based Mental Health Practices for Autistic Youth in Bangladesh and Germany: A Cross-Cultural Comparison

The implementation of evidence-based practices (EBPs) for autistic youth is a critical concern worldwide. Research examining factors facilitating the implementation of EBPs found that providers' attitudes are an important factor. In this study, we evaluated cross-cultural differences in attitudes toward and use of EBPs. We tested socio-demographic factors as predictors of attitudes, and attitudes as predictors of EBPs use among mental health professionals working with autistic youth in Bangladesh and Germany. We used purposeful sampling. Two-hundred-ninety-two professionals who worked in a clinical setting responded to the survey and fulfilled the inclusion criteria (101 in Bangladesh, 191 in Germany). Participants were asked to respond to nine subscales of the Evidence-Based Practice Attitude Scale-36 (EBPAS-36), to indicate which of nine types of treatments they used, and to provide sociodemographic data. Measurement invariance across countries could be established for four subscales of the EBPAS-36. Comparative analyses of attitudes showed that professionals in both countries were open to using EBPs, but German practitioners were more likely to use EBPs when they appealed to them. By contrast, Bangladeshi professionals claimed to be more likely to adopt an EBP when required and to be more willing to learn EBPs to enhance job security. The relationship between caseload and attitudes varied between countries. A broader variety of EBPs was used in Germany. The findings highlight the importance of considering country-specific factors when implementing EBPs. Directions for conducting comparative studies on mental health professionals' attitudes towards EBP including methodological considerations are discussed.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 3: study protocol to evaluate sustainment in a hybrid type 2 effectiveness-implementation cluster-randomized trial

BACKGROUND

Although research on the implementation of evidence-based psychological treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase-the Sustainment Phase-of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial-the Implementation Phase and Train-the-Trainer Phase-TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes.

METHODS

Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data.

DISCUSSION

The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT05956678 . Registered on July 21, 2023.

A scoping review of theories, models and frameworks used or proposed to evaluate knowledge mobilization strategies

BACKGROUND

Evaluating knowledge mobilization strategies (KMb) presents challenges for organizations seeking to understand their impact to improve KMb effectiveness. Moreover, the large number of theories, models, and frameworks (TMFs) available can be confusing for users. Therefore, the purpose of this scoping review was to identify and describe the characteristics of TMFs that have been used or proposed in the literature to evaluate KMb strategies.

METHODS

A scoping review methodology was used. Articles were identified through searches in electronic databases, previous reviews and reference lists of included articles. Titles, abstracts and full texts were screened in duplicate. Data were charted using a piloted data charting form. Data extracted included study characteristics, KMb characteristics, and TMFs used or proposed for KMb evaluation. An adapted version of Nilsen (Implement Sci 10:53, 2015) taxonomy and the Expert Recommendations for Implementing Change (ERIC) taxonomy (Powell et al. in Implement Sci 10:21, 2015) guided data synthesis.

RESULTS

Of the 4763 search results, 505 were retrieved, and 88 articles were eligible for review. These consisted of 40 theoretical articles (45.5%), 44 empirical studies (50.0%) and four protocols (4.5%). The majority were published after 2010 (n = 70, 79.5%) and were health related (n = 71, 80.7%). Half of the studied KMb strategies were implemented in only four countries: Canada, Australia, the United States and the United Kingdom (n = 42, 47.7%). One-third used existing TMFs (n = 28, 31.8%). According to the adapted Nilsen taxonomy, process models (n = 34, 38.6%) and evaluation frameworks (n = 28, 31.8%) were the two most frequent types of TMFs used or proposed to evaluate KMb. According to the ERIC taxonomy, activities to "train and educate stakeholders" (n = 46, 52.3%) were the most common, followed by activities to "develop stakeholder interrelationships" (n = 23, 26.1%). Analysis of the TMFs identified revealed relevant factors of interest for the evaluation of KMb strategies, classified into four dimensions: context, process, effects and impacts.

CONCLUSIONS

This scoping review provides an overview of the many KMb TMFs used or proposed. The results provide insight into potential dimensions and components to be considered when assessing KMb strategies.

Treatment mechanism of the WHO caregiver skills training intervention for autism delivered in community settings

This study examined the mechanism of effect of the WHO Caregiver Skills Training (CST) through secondary analysis of a pilot RCT conducted in community settings. Participants were 86 caregivers (77% mothers) of children with ASD (78% male, mean age: 44.8 months) randomized to CST (n = 43) or treatment as usual (n = 43). The primary outcomes, measured at baseline (t1), immediately post-intervention (t2), and 3 months post-intervention (t3), were derived from the coding of caregiver-child free play interactions with the Brief Observation of Social-Communication Change (BOSCC) and the Joint Engagement Rating Inventory scale (JERI). At t3 positive treatment main effects had been observed for caregiver skills supportive of the interaction and for flow of the interaction (JERI), albeit only non-significant changes in the expected direction for child outcomes: autism phenotypic behaviors (BOSCC), joint engagement and availability to interact (JERI). This study tested the theory of change of CST, hypothesizing that the intervention would lead to an improvement on all child and dyad outcomes through an increase in the caregiver skills supportive of the interaction. Serial mediation analyses revealed that the effect of the intervention was significantly influenced by change in caregiver skills. Participation in the intervention led to notable increases in caregiver skills at t2 and t3, which subsequently contributed to improvements at t3 in flow of the interaction, autism phenotypic behavior, joint engagement, and availability to interact. We confirmed our a priori hypothesis showing that change in caregiver skills significantly mediated the effect of treatment on the dyad primary outcome, as well as on the other child outcomes that had shown non-significant changes in the expected direction. Implications for intervention design and policy making in the context of public health services are discussed.

Uncovering determinants of perceived feasibility of TF-CBT through coincidence analysis

Introduction

A mental health provider's perception of how well an intervention can be carried out in their context (i.e., feasibility) is an important implementation outcome. This article aims to identify determinants of feasibility of trauma-focused cognitive behavioral therapy (TF-CBT) through a case-based causal approach.

Method

Data come from an implementation-effectiveness study in which lay counselors (teachers and community health volunteers) implemented a culturally adapted manualized mental health intervention, TF-CBT, delivered to teens who were previously orphaned and were experiencing posttraumatic stress symptoms and prolonged grief in Western Kenya. The intervention team identified combinations of determinants that led to feasibility among teacher- and community health volunteer-counselors through coincidence analysis.

Results

Among teacher-counselors, organizational-level factors (implementation climate, implementation leadership) determined moderate and high levels of feasibility. Among community health volunteer-counselors, a strong relationship between a clinical supervisor and the supervisee was the most influential determinant of feasibility.

Conclusion

Methodology and findings from this article can guide the assessment of determinants of feasibility and the development of implementation strategies for manualized mental health interventions in contexts like Western Kenya.

Plain Language Summary

A mental health provider's perception of how easy a therapy is to use in their work setting (i.e., feasibility) can impact whether the provider uses the therapy in their setting. Implementation researchers have recommended finding practices and constructs that lead to important indicators that a therapy will be used. However, limited research to our knowledge has searched and found practices and constructs that might determine feasibility of a therapy. This article uses existing data from a large trial looking at the continued use of a trauma-focused therapy to find practices and constructs that lead to moderate and high levels of feasibility. We found that in settings with a strong organizational structure that organization and leadership support for the therapy led to teachers in Kenya to perceive the therapy as easy to use. On the other hand, in settings with a weaker organizational structure, outside support from a clinical supervisor led to community health volunteers in Kenya perceiving the therapy as easy to use. The findings from this article can guide context-specific recommendations for increasing perceived therapy feasibility at the provider-, organization-, and policy levels.

Pilot evaluation of the Fiscal Mapping Process for sustainable financing of evidence-based youth mental health treatments: A comparative case study analysis

Background

Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies. Supports are needed that can guide service agencies in accessing sustainable funding for EBTs. We conducted a pilot evaluation of the Fiscal Mapping Process, an Excel-based strategic planning tool that helps service agency leaders identify and coordinate financing strategies for their EBT programs.

Method

Pilot testing of the Fiscal Mapping Process was completed with 10 youth mental health service agencies over a 12-month period, using trauma-focused cognitive-behavioral therapy or parent-child interaction therapy programs. Service agency representatives received initial training and monthly coaching in using the tool. We used case study methods to synthesize all available data (surveys, focus groups, coaching notes, document review) and contrast agency experiences to identify key findings through explanation building.

Results

Key evaluation findings related to the process and outcomes of using the Fiscal Mapping Process, as well as contextual influences. Process evaluation findings helped clarify the primary use case for the tool and identified the importance-and challenges-of engaging external collaborators. Outcome evaluation findings documented the impacts of the Fiscal Mapping Process on agency-reported sustainment capacities (strategic planning, funding stability), which fully explained reported improvements in outcomes (extent and likelihood)-although these impacts were incremental. Findings on contextual factors documented the influence of environmental and organizational capacities on engagement with the tool and concerns about equitable impacts, but also the view that the process could usefully generalize to other EBTs.

Conclusions

Our pilot evaluation of the Fiscal Mapping Process was promising. In future work, we plan to integrate the tool into EBT implementation initiatives and test its impact on long-term sustainment outcomes across various EBTs, while increasing attention to equity considerations.

Impact of a Virtual Suicide Safety Planning Training on Clinician Knowledge, Self-Efficacy, and Use of Safety Plans in Community Mental Health Clinics

OBJECTIVE

Safety planning is a critical evidence-based intervention used to prevent suicide among individuals who report suicidal ideation or behavior. There is a dearth of research on optimal ways to disseminate and implement safety plans in community settings. The present study examined one implementation strategy, a 1-hour virtual pre-implementation training, designed to teach clinicians to effectively use an electronic safety plan template (ESPT), integrated with suicide risk assessment tools, in the context of a measurement feedback system. We examined the effect of this training on clinician knowledge and self-efficacy in use of safety planning as well as ESPT completion rates.

METHOD

Thirty-six clinicians across two community-based clinical psychology training clinics completed the virtual pre-implementation training as well as pre- and post-training knowledge and self-efficacy assessments. Twenty-six clinicians completed a 6-month follow-up term.

RESULTS

Clinicians reported significant improvements in self-efficacy and knowledge from pre- to post-training. They retained significant improvements in self-efficacy and a trend toward greater knowledge at the 6-month follow-up. Of the clinicians who worked with suicidal youth, 81% attempted to use an ESPT and 63% successfully completed all sections of the ESPT. Reasons for partial completion included technological difficulties and time constraints.

CONCLUSION

A brief virtual pre-implementation training can improve clinician knowledge and self-efficacy in use of an ESPT with youth at risk for suicide. This strategy also holds the potential to improve the adoption of this novel evidence-based intervention in community-based settings.

Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health

OBJECTIVE

Measurement-based care (MBC), which collects session-by-session symptom data from patients and provides clinicians with feedback on treatment response, is a highly generalizable evidence-based practice with significant potential to improve the outcomes of mental health treatment in youth when implemented with fidelity; however, it is rarely used in community settings. This study tested whether an implementation strategy targeting organizational leadership and organizational implementation climate could improve MBC fidelity and clinical outcomes for youth in outpatient mental health clinics.

METHOD

In a cluster randomized trial, 21 clinics were assigned to the Leadership and Organizational Change for Implementation strategy plus training and technical assistance in MBC (k = 11, n = 117) or training and technical assistance only (k = 10, n = 117). Primary outcomes of MBC fidelity (assessed via electronic metadata) and youth symptom improvement (assessed via caregiver-reported change on the Shortform Assessment for Children Total Problem Score) were collected for consecutively enrolled youths (ages 4-18 years) who initiated treatment in the 12 months following MBC training. Outcomes of each youth were assessed for 6 months following baseline.

RESULTS

A total of 234 youths were enrolled and included in intent-to-treat analyses. At baseline, there were no significant differences by condition in clinic, clinician, or youth characteristics. Youths in clinics using the Leadership and Organizational Change for Implementation strategy experienced significantly higher MBC fidelity compared with youths in control clinics (23.1% vs 3.4%, p = .014), and exhibited significantly greater reductions in symptoms from baseline to 6 months (d = 0.31, 95% CI: 0.04-0.58, p = .023).

CONCLUSION

Implementation strategies targeting organizational leadership and focused implementation climate can improve fidelity to evidence-based practices and clinical outcomes of youth mental health services.

CLINICAL TRIAL REGISTRATION INFORMATION

Working to Implement and Sustain Digital Outcome Measures (WISDOM); https://clinicaltrials.gov/; NCT04096274.

An exposure-based implementation strategy to decrease clinician anxiety about implementing suicide prevention evidence-based practices: protocol for development and pilot testing (Project CALMER)

BACKGROUND

Clinicians often report that their own anxiety and low self-efficacy inhibit their use of evidence-based suicide prevention practices, including gold-standard screening and brief interventions. Exposure therapy to reduce clinician maladaptive anxiety and bolster self-efficacy use is a compelling but untested approach to improving the implementation of suicide prevention evidence-based practices (EBPs). This project brings together an interdisciplinary team to leverage decades of research on behavior change from exposure theory to design and pilot test an exposure-based implementation strategy (EBIS) to target clinician anxiety to improve suicide prevention EBP implementation.

METHODS

We will develop, iteratively refine, and pilot test an EBIS paired with implementation as usual (IAU; didactic training and consultation) in preparation for a larger study of the effect of this strategy on reducing clinician anxiety, improving self-efficacy, and increasing use of the Columbia Suicide Severity Rating Scale and the Safety Planning Intervention in outpatient mental health settings. Aim 1 of this study is to use participatory design methods to develop and refine the EBIS in collaboration with a stakeholder advisory board. Aim 2 is to iteratively refine the EBIS with up to 15 clinicians in a pilot field test using rapid cycle prototyping. Aim 3 is to test the refined EBIS in a pilot implementation trial. Forty community mental health clinicians will be randomized 1:1 to receive either IAU or IAU + EBIS for 12 weeks. Our primary outcomes are EBIS acceptability and feasibility, measured through questionnaires, interviews, and recruitment and retention statistics. Secondary outcomes are the engagement of target implementation mechanisms (clinician anxiety and self-efficacy related to implementation) and preliminary effectiveness of EBIS on implementation outcomes (adoption and fidelity) assessed via mixed methods (questionnaires, chart-stimulated recall, observer-coded role plays, and interviews).

DISCUSSION

Outcomes from this study will yield insight into the feasibility and utility of directly targeting clinician anxiety and self-efficacy as mechanistic processes informing the implementation of suicide prevention EBPs. Results will inform a fully powered hybrid effectiveness-implementation trial to test EBIS' effect on implementation and patient outcomes.

TRIAL REGISTRATION

Clinical Trials Registration Number: NCT05172609 . Registered on 12/29/2021.

Navigating the multiple dimensions of the creativity-mental disorder link: a Convergence Mental Health perspective

BACKGROUND

This paper discusses a paradox in mental health. It manifests as a relationship between adverse "bad" effects (suffering, clinical costs, loss of productivity) in individuals and populations and advantageous "good" aspects of mental disorders. These beneficial aspects (scientific, artistic and political accomplishments) emanate at the societal level through the frequently unprecedented creativity of people suffering from mental disorders and their relatives. Such gains can contribute to societal innovation and problem-solving. Especially in times of accelerated social-ecological change, approaches are needed that facilitate best-possible mental health care but also recognize creative ideas conducive to beneficial clinical and social-ecological innovations as soon as possible.

DISCUSSION

This paper emphasizes the need to account for creativity as a crucial component in evolving mental health systems and societies. It highlights the need for wide-ranging approaches and discusses how research targeting multiple facets (e.g., brain level, cognitive neuroscience, psychiatry, neurology, socio-cultural, economic and other factors) might further our understanding of the creativity-mental disorder link and its importance for innovating mental health systems and societies.

CONCLUSION

Our discussion clarifies that considerable research will be needed to obtain a better understanding of how creativity associated with mental disorders may help to create more sustainable societies on a fast-changing planet through innovative ideas. Given the current-state-of-the-art of research and healthcare management, our discussion is currently speculative. However, it provides a basis for how pros and cons might be studied in the future through transdisciplinary research and collaborations across sectors of society.

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): A modified Delphi study

Background

Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources.

Method

MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis.

Results

In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application.

Discussion

MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

Setting and Provider Predictors of Implementation Success for an Eating Disorder Prevention Program Delivered by College Peer Educators

INTRODUCTION

College students face increased risk for a variety of mental health problems but experience barriers to treatment access. Prevention programs, including those implemented by peer educators, may decrease treatment needs and increase service access. We examined the implementation of an evidence-based eating disorder prevention program, Body Project, delivered by college peer educators at 63 colleges/universities, comparing three levels of implementation support: (1) Train-the-Trainer (TTT) training; (2) TTT plus a technical assistance workshop (TTT + TA); and (3) TTT + TA with one year of quality assurance calls (TTT + TA + QA). The present study tested the degree to which indicators proposed by the Consolidated Framework for Implementation Research (CFIR) were associated with core implementation outcomes.

METHOD

We tested whether indices of CFIR domains (i.e., perceived intervention characteristics, outer and inner setting factors, provider characteristics, and implementation process) were correlated with three implementation outcomes (program reach, fidelity, effectiveness) during a 1-year implementation period.

RESULTS

Greater program reach was associated with implementation process, specifically the completion of more implementation activities (β = 0.46). Greater program fidelity was associated with higher positive (β = 0.44) and lower negative (β = - 0.43) perceptions of the Body Project characteristics, and greater reported general support for evidence-based practices (β = 0.41). Greater effectiveness was associated with lower negative perceptions of Body Project characteristics (d = 0.49).

CONCLUSIONS

Several implementation determinants proposed by the CFIR model predicted outcomes, especially intervention fidelity. Across the outcomes of interest, implementation determinants related to peer educator and supervisor perceived characteristics of the specific intervention and general attitudes towards evidence-based practices emerged as robust predictors to inform future work investigating ongoing implementation and sustainability of programs in university settings.

External and Internal Validity Considerations in Youth Effectiveness Trials: Lessons Learned from the COMET Study

OBJECTIVE

Effectiveness trials aim to increase the generalizability and public health impact of interventions. However, challenges associated with this design present threats to external and internal validity. This paper illustrates these challenges using data from a two-site randomized effectiveness trial, the Community Study of Outcome Monitoring for Emotional Disorders in Teens (COMET) and presents recommendations for future research.

METHOD

COMET was a randomized effectiveness trial conducted in 19 community mental health clinics in two states comparing three interventions: treatment as usual (TAU), TAU with measurement-based care (TAU+), and the Unified Protocol forTransdiagnostic Treatment of Emotional Disorders in Adolescents with MBC (UPA). Participants included 176 clinicians (mean age = 35.5; 85.8% cisgender female; 53.0% racially and/or ethnically minorized) and 196 adolescents (mean age = 14.7; 65.3% cisgender female; 69.4% racially and/or ethnically minorized). Analyses outlined participant flow from recruitment to study completion, described participant characteristics, and examined site differences.

RESULTS

Analysis of participant flow suggested that recruitment and retention of clinicians and adolescents was challenging, raising questions about whether participants were representative of participating clinics. Both the clinician and adolescent samples were racially and ethnically diverse and adolescents were low income and clinically complex. Significant site differences were observed in clinician and adolescent characteristics.

CONCLUSIONS

While this study was successful in recruiting a diverse and historically under-represented sample, difficulties in recruitment and retention raise questions about external validity and site differences present challenges to internal validity of study findings. Suggestions for future effectiveness studies, drawing from implementation science approaches, are discussed.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 3: Study protocol to evaluate sustainment in a hybrid type 2 effectiveness-implementation cluster-randomized trial

treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase - the Sustainment Phase - of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial - the Implementation Phase and Train-the-Trainer Phase - TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes.

Methods

Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data.

Discussion

The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change.

Trial Registration

ClinicalTrials.gov identifier: NCT05956678. Registered on July 21, 2023. https://classic.clinicaltrials.gov/ct2/show/NCT05956678?term=NCT05956678&draw=2&rank=1.

Zero Tolerance for Coercion? Historical, Cultural and Organisational Contexts for Effective Implementation of Coercion-Free Mental Health Services around the World

Coercion of service users/patients when receiving care and treatment has been a serious dilemma for mental health services since at least the 18th century, and the debate about how best to minimise or even eradicate compulsion remains intense. Coercion is now, once again and rightly, at the top of the international policy agenda and the COST Action 'FOSTREN' is one example of a renewed commitment by service user advocates, practitioners and researchers to move forward in seriously addressing this problem. The focus of service improvement efforts has moved from pure innovation to practical implementation of effective interventions based on an understanding of the historical, cultural and political realities in which mental health services operate. These realities and their impact on the potential for change vary between countries across Europe and beyond. This article provides a novel overview by focusing on the historical, cultural and political contexts which relate to successful implementation primarily in Europe, North America and Australasia so that policy and practice in these and other regions can be adopted with an awareness of these potentially relevant factors. It also outlines some key aspects of current knowledge about the leading coercion-reduction interventions which might be considered when redesigning mental health services.

De-implementation of Low-Value Practices for Autism Spectrum Disorder

Due to a variety of factors, Autism Spectrum Disorder (ASD) has long been tethered to use of low-value practice (LVP), arguably moreso than any other psychiatric or neurodevelopmental condition. Although dissemination of empirically supported treatments (EST) for autistic individuals has expanded markedly over the past decade, there has not been concomitant reduction in the use of LVP. It is critical that clinicians and scientists not only promote the implementation of EST, but also facilitate the de-implementation (abandonment and/or divestment) of ineffective or harmful practices. In this review, we describe a data-driven approach that can be used to identify LVP, drawing from established criteria for identification of evidence-based treatments (e.g., APA Division 12, National Clearinghouse on Autism Evidence and Practice; SAMHSA), as well as broader considerations such as social validity, cost, and parsimony. Herein, a data-based approach to LVP identification is proposed with a goal of improving quality of service access. Within an implementation science framework, we identify specific facilitators that sustain LVP use, and recommendations for subsequent de-implementation strategies are offered.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 2: study protocol for a hybrid type 2 effectiveness-implementation cluster-randomized trial using train-the-trainer

BACKGROUND

Train-the-trainer (TTT) is a promising method for implementing evidence-based psychological treatments (EBPTs) in community mental health centers (CMHCs). In TTT, expert trainers train locally embedded individuals (i.e., Generation 1 providers) to deliver an EBPT, who then train others (i.e., Generation 2 providers). The present study will evaluate implementation and effectiveness outcomes of an EBPT for sleep and circadian dysfunction-the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C)-delivered to CMHC patients with serious mental illness by Generation 2 providers (i.e., trained and supervised within CMHCs via TTT). Specifically, we will investigate whether adapting TranS-C to fit CMHC contexts improves Generation 2 (a) patient outcomes and (b) providers' perceptions of fit.

METHODS

TTT will be implemented in nine CMHCs in California, USA (N = 60 providers; N = 130 patients) via facilitation. CMHCs are cluster-randomized by county to Adapted TranS-C or Standard TranS-C. Within each CMHC, patients are randomized to immediate TranS-C or usual care followed by delayed treatment with TranS-C (UC-DT). Aim 1 will assess the effectiveness of TranS-C (combined Adapted and Standard), compared to UC-DT, on improvements in sleep and circadian problems, functional impairment, and psychiatric symptoms for Generation 2 patients. Aim 2 will evaluate whether Adapted TranS-C is superior to Standard TranS-C with respect to Generation 2 providers' perceptions of fit. Aim 3 will evaluate whether Generation 2 providers' perceived fit mediates the relation between TranS-C treatment condition and patient outcomes. Exploratory analyses will (1) evaluate whether the effectiveness of TranS-C for patient outcomes is moderated by generation, (2) compare Adapted and Standard TranS-C on patient perceptions of credibility/improvement and PhenX Toolkit outcomes (e.g., substance use, suicidality), and (3) evaluate other possible moderators.

DISCUSSION

This trial has potential to (a) inform the process of embedding local trainers and supervisors to expand delivery of a promising transdiagnostic treatment for sleep and circadian dysfunction, (b) add to the growing body of TTT literature by evaluating TTT outcomes with a novel treatment and population, and (c) advance our understanding of providers' perceptions of EBPT "fit" across TTT generations.

TRIAL REGISTRATION

ClinicalTrials.gov identifier NCT05805657 . Registered on April 10, 2023.

Needs assessment for behavioral parent training for ADHD in Brazil

Introduction

Attention-Deficit/Hyperactivity Disorder (ADHD) is a debilitating condition affecting children and their families worldwide. Behavioral parent training is a recommended form of empirically supported non-pharmacological intervention for young children with mild to moderate ADHD. However, access to such treatment is limited in many countries. Here we identify the treatment needs of Brazilian families with children demonstrating symptoms of ADHD, and the barriers families face in accessing behavioral treatment.

Methods

A qualitative needs assessment was undertaken with parents (n = 23), educators (n = 15), and healthcare providers (n = 16). Semi-structured telephone interviews were conducted, and common themes were identified through inductive coding of participants' responses.

Results

Participants reported a lack of accessible behavioral treatment, and delays in accessing treatment when available. The majority of parents had not received behavioral parent training, despite it being a recommended form of treatment. Parents, educators and healthcare providers strongly endorsed a need for practical tools to manage the behavior of children with ADHD.

Conclusion

Existing services might not meet the needs of children with ADHD and their families in Brazil. Easily accessed behavioral parent training programs are recommended to address the identified treatment gap for Brazilian children with ADHD and their families.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 2: Study protocol for a hybrid type 2 effectiveness-implementation cluster- randomized trial using train-the-trainer

Background

Train-the-trainer (TTT) is a promising method for implementing evidence-based psychological treatments (EBPTs) in community mental health centers (CMHCs). In TTT, expert trainers train locally embedded individuals (i.e., Generation 1 providers) to deliver an EBPT, who then train others (i.e., Generation 2 providers). The present study will evaluate implementation and effectiveness outcomes of an EBPT for sleep and circadian dysfunction-the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C)-delivered to CMHC patients with serious mental illness by Generation 2 providers (i.e., trained and supervised within CMHCs via TTT). Specifically, we will investigate whether adapting TranS-C to fit CMHC contexts improves Generation 2 (a) patient outcomes (b) providers' perceptions of fit.

Methods

TTT will be implemented in nine CMHCs in California, United States (N= 60 providers; N= 130 patients) via facilitation. CMHCs are cluster-randomized by county to Adapted TranS-C or Standard TranS-C. Within each CMHC, patients are randomized to immediate TranS-C or usual care followed by delayed treatment with TranS-C (UC-DT). Aim 1 will assess the effectiveness of TranS-C (combined Adapted and Standard), compared to UC-DT, on improvements in sleep and circadian problems, functional impairment, and psychiatric symptoms for Generation 2 patients. Aim 2 will evaluate whether Adapted TranS-C is superior to Standard TranS-C with respect to Generation 2 providers' perceptions of fit. Aim 3 will evaluate whether Generation 2 providers' perceived fit mediates the relation between TranS-C treatment condition and patient outcomes. Exploratory analyses will: (1) evaluate whether the effectiveness of TranS-C for patient outcomes is moderated by generation, (2) compare Adapted and Standard TranS-C on patient perceptions of credibility/improvement and PhenX Toolkit outcomes (e.g., substance use, suicidality); and (3) evaluate other possible moderators.

Discussion

This trial has potential to inform the process of (a) embedding local trainers and supervisors to expand delivery of a promising transdiagnostic treatment for sleep and circadian dysfunction, (b) adding to the growing body of TTT literature by evaluating TTT outcomes with a novel treatment and population, and (c) advancing our understanding of providers' perceptions of EBPT 'fit' across TTT generations.

Trial registration

Clinicaltrials.gov identifier: NCT05805657. Registered on April 10, 2023. https://clinicaltrials.gov/ct2/show/NCT05805657.

Intentional practice: a common language, approach and set of methods to design, adapt and implement contextualised wellbeing solutions

Reducing the "science-to-practice" gap has gained significant attention across multi-disciplinary settings, including school psychology and student wellbeing, trauma-informed practice, community and human services, and clinically focused health care. There has been increasing calls for complexity and contextualisation to be integrated within the implementation science literature. This includes the design and implementation of interventions spanning "systems" (whole-of-community capacity building initiatives), "programs" (e.g., evidence-based programs, clinical interventions) and "moment-to-moment" support or care. The latter includes responses and communication designed to deliver specific learning, growth or wellbeing outcomes, as personalised to an individual's presenting needs and context (e.g., trauma-informed practice). Collectively, this paper refers to these interventions as "wellbeing solutions". While the implementation science literature offers a range of theories, models and approaches to reduce the science-to-practice gap in wellbeing solution design and implementation, they do not operationalise interventions into the "moment", in a manner that honours both complexity and contextualisation. Furthermore, the literature's language and content is largely targeted towards scientific or professional audiences. This paper makes the argument that both best-practice science, and the frameworks that underpin their implementation, need to be "sticky", practical and visible for both scientific and non-scientific knowledge users. In response to these points, this paper introduces "intentional practice" as a common language, approach and set of methods, founded upon non-scientific language, to guide the design, adaptation and implementation of both simple and complex wellbeing solutions. It offers a bridge between scientists and knowledge users in the translation, refinement and contextualisation of interventions designed to deliver clinical, wellbeing, growth, therapeutic and behavioural outcomes. A definitional, contextual and applied overview of intentional practice is provided, including its purported application across educational, wellbeing, cross-cultural, clinical, therapeutic, programmatic and community capacity building contexts.

Team-focused implementation strategies to improve implementation of mental health screening and referral in rural Children's Advocacy Centers: study protocol for a pilot cluster randomized hybrid type 2 trial

BACKGROUND

Children's Advocacy Centers (CACs) use multidisciplinary teams to investigate and respond to maltreatment allegations. CACs play a critical role in connecting children with mental health needs to evidence-based mental health treatment, especially in low-resourced rural areas. Standardized mental health screening and referral protocols can improve CACs' capacity to identify children with mental health needs and encourage treatment engagement. In the team-based context of CACs, teamwork quality is likely to influence implementation processes and outcomes. Implementation strategies that target teams and apply the science of team effectiveness may enhance implementation outcomes in team-based settings.

METHODS

We will use Implementation Mapping to develop team-focused implementation strategies to support the implementation of the Care Process Model for Pediatric Traumatic Stress (CPM-PTS), a standardized screening and referral protocol. Team-focused strategies will integrate activities from effective team development interventions. We will pilot team-focused implementation in a cluster-randomized hybrid type 2 effectiveness-implementation trial. Four rural CACs will implement the CPM-PTS after being randomized to either team-focused implementation (n = 2 CACs) or standard implementation (n = 2 CACs). We will assess the feasibility of team-focused implementation and explore between-group differences in hypothesized team-level mechanisms of change and implementation outcomes (implementation aim). We will use a within-group pre-post design to test the effectiveness of the CPM-PTS in increasing caregivers' understanding of their child's mental health needs and caregivers' intentions to initiate mental health services (effectiveness aim).

CONCLUSIONS

Targeting multidisciplinary teams is an innovative approach to improving implementation outcomes. This study will be one of the first to test team-focused implementation strategies that integrate effective team development interventions. Results will inform efforts to implement evidence-based practices in team-based service settings.

TRIAL REGISTRATION

Clinicaltrials.gov, NCT05679154 . Registered on January 10, 2023.

Misalignment in Community Mental Health Leader and Therapist Ratings of Psychological Safety Climate Predicts Therapist Self-Efficacy with Evidence-Based Practices (EBPs)

Therapist self-efficacy in delivering evidence-based practices (EBPs) is associated with implementation outcomes, including adoption and sustainment in community mental health settings. Inner context organizational climate, including psychological safety, can proximally shape therapist learning experiences within EBP implementation. Psychologically safe environments are conducive to learning behaviors including taking risks, admitting mistakes, and seeking feedback. Organization leaders are instrumental in facilitating psychological safety, but may have differing perspectives of organizational climate than front-line therapists. Discrepant leader and therapist views of psychological safety may have independent associations with therapist EBP learning and implementation outcomes over and above average therapist perceptions of climate. This study examined survey data from 337 therapists and 123 leaders from 49 programs contracted to deliver multiple EBPs within a study examining determinants of sustainment within a large system-driven implementation. Both leaders and therapists completed measures of psychological safety climate and therapists reported on their self-efficacy in delivering multiple EBPs in children's mental health services. Polynomial regression and response surface analysis models were conducted to examine the associations of therapist and leader reports of psychological safety and therapist EBP self-efficacy. Greater discrepancies between leader and therapist reports of psychological safety, in either direction, were associated with lower therapist EBP self-efficacy. Alignment in leader and therapist views of psychological safety climate may impact EBP implementation outcomes. Strategies for improving alignment in perceptions and priorities among organizational members can be included in organizational implementation interventions and may represent unexamined implementation mechanisms of action.

Protocol for a randomised pilot study of a novel Parent-Child Interaction Therapy (PCIT) 're-implementation' intervention

BACKGROUND

Despite a number of clinicians having been trained in Parent-Child Interaction Therapy (PCIT) in Aotearoa/New Zealand, few are regularly delivering the treatment, with barriers to use including a lack of suitable equipment and lack of professional support. This pragmatic, parallel-arm, randomised, controlled pilot trial includes PCIT-trained clinicians who are not delivering, or only rarely utilising, this effective treatment. The study aims to assess the feasibility, acceptability and cultural responsivity of study methods and intervention components and to collect variance data on the proposed future primary outcome variable, in preparation for a future, larger trial.

METHODS

The trial will compare a novel 're-implementation' intervention with a refresher training and problem-solving control. Intervention components have been systematically developed to address barriers and facilitators to clinician use of PCIT using implementation theory, and a draft logic model with hypothesised mechanisms of action, derived from a series of preliminary studies. The intervention includes complimentary access to necessary equipment for PCIT implementation (audio-visual equipment, a 'pop-up' time-out space, toys), a mobile senior PCIT co-worker and an optional weekly PCIT consultation group, for a 6-month period. Outcomes will include the feasibility of recruitment and trial procedures; acceptability of the intervention package and data collection methods to clinicians; and clinician adoption of PCIT.

DISCUSSION

Relatively little research attention has been directed at interventions to resurrect stalled implementation efforts. Results from this pragmatic pilot RCT will refine and shape knowledge relating to what it might take to embed the ongoing delivery of PCIT in community settings, providing more children and families with access to this effective treatment.

TRIAL REGISTRATION

ANZCTR, ACTRN12622001022752, registered on July 21, 2022.

The Interaction Between General and Strategic Leadership and Climate on Their Multilevel Associations with Implementer Attitudes Toward Universal Prevention Programs for Youth Mental Health: A Cross-Sectional Study

Emerging literature has highlighted the importance of discerning general and strategic organizational context (OC) factors (e.g., leadership and climate) and their interaction effect on individual implementation behaviors (e.g., attitudes toward evidence-based practices; EBPs) in youth mental healthcare. This study aimed to examine how leadership and climate (general and strategic) are associated with implementer attitudes toward EBPs across the individual and organizational levels and their interaction effect in schools. A series of multilevel models (MLMs) were fitted on a diverse sample of schools actively implementing universal prevention programs for youth mental health (441 implementers from 52 schools). The organization-level aggregates and individual educators' perceptions of general and strategic leadership and climate, and their interaction terms, were entered as level-2 and level-1 predictors of four attitudinal dimensions (Requirement, Openness, Appeal, and Divergence) based on their level of measurement. At the organizational level, higher levels of strategic leadership and climate, but not their general counterparts, were consistently associated with more favorable attitudes in all four dimensions. At the individual/within-school level, higher levels of perceived general and strategic leadership and climate were associated with more favorable attitudes of Requirement and Openness. At the organizational/between-school level, general climate moderated the positive effect of strategic climate on implementers' perception of appeal and divergence of EBPs. Our findings indicate that leaders should make data-based decisions to allocate resources on strategic and/or general leadership and climate to foster favorable staff attitudes toward EBPs based on the level of measurement, implementation-specificity, and attitudinal dimensions.

Implementation strategies to address the determinants of adoption, implementation, and maintenance of a clinical decision support tool for emergency department buprenorphine initiation: a qualitative study

BACKGROUND

Untreated opioid use disorder (OUD) is a significant public health problem. Buprenorphine is an evidence-based treatment for OUD that can be initiated in and prescribed from emergency departments (EDs) and office settings. Adoption of buprenorphine initiation among ED clinicians is low. The EMBED pragmatic clinical trial investigated the effectiveness of a clinical decision support (CDS) tool to promote ED clinicians' behavior related to buprenorphine initiation in the ED. While the CDS intervention was not associated with increased rates of buprenorphine treatment for patients with OUD at intervention ED sites, attending physicians at intervention EDs were more likely to initiate buprenorphine at least once over the duration of the study compared to those in the usual care arms (44.4% vs 34.0%, P = 0.01). This suggests the CDS intervention may be associated with increased adoption of buprenorphine initiation. As a secondary aim, we sought to identify the determinants of CDS adoption, implementation, and maintenance in a variety of ED settings and geographic locations.

METHODS

We purposively sampled and conducted semi-structured, in-depth interviews with clinicians across EMBED trial sites randomized to the intervention arm from five healthcare systems. Interviews elicited clinician experiences regarding buprenorphine initiation and CDS use. Interviews were analyzed using directed content analysis informed by the Practical, Robust Implementation and Sustainability Model (PRISM). We used a hybrid approach (a priori codes informed by PRISM and emergent codes) for codebook development. ATLAS.ti (version 9.0) was used for data management. Coded data were analyzed within individual interview transcripts and across all interviews to identify major themes. This process involved (1) combining, comparing, and making connections between codes; (2) writing analytic memos about observed patterns; and (3) frequent team meetings to discuss emerging patterns.

RESULTS

Twenty-eight interviews were conducted. Major themes that influenced the successful adoption, implementation, and maintenance of the EMBED intervention and ED-initiated BUP were organizational culture and commitment, clinician training and support, the ability to connect patients to ongoing treatment, and the ability to tailor implementation to each ED. These findings informed the identification of implementation strategies (framed using PRISM domains) to enhance the ED initiation of buprenorphine.

CONCLUSION

The findings from this qualitative analysis can provide guidance to build better systems to promote the adoption of ED-initiated buprenorphine.

A Pre-Implementation Enhancement Strategy to Increase the Yield of Training and Consultation for School-Based Behavioral Preventive Practices: a Triple-Blind Randomized Controlled Trial

As the most common setting where youth access behavioral healthcare, the education sector frequently employs training and follow-up consultation as cornerstone implementation strategies to promote the implementation of evidence-based practices (EBPs). However, these strategies alone are not sufficient to promote desirable implementation (e.g., intervention fidelity) and youth behavioral outcomes (e.g., mitigated externalizing behaviors). Theory-informed pragmatic pre-implementation enhancement strategies (PIES) are needed to prevent the lackluster outcomes of training and consultation. Specifically, social cognitive theory explicates principles that inform the design of PIES content and specify mechanisms of behavior change (e.g., "intentions to implement" (ITI)) to target increasing providers' responsiveness to training and consultation. This triple-blind parallel randomized controlled trial preliminarily examined the efficacy of a pragmatic PIES based on social cognitive theories (SC-PIES) to improve implementation and youth behavioral outcomes from universal preventive EBPs in the education sector. Teachers from a diverse urban district were recruited and randomly assigned to the treatment (SC-PIES; ntreatment = 22) or active control condition (administrative meeting; ncontrol = 21). Based on the condition assigned, teachers received the SC-PIES or met with administrators before their EBP training. We assessed teachers' ITI, intervention fidelity, and youth behavioral outcome (academic engagement as an incompatible behavior to externalizing disorders) at baseline, immediately after training, and 6 weeks afterward. A series of ANCOVAs detected sizeable effects of SC-PIES, where teachers who received SC-PIES demonstrated significantly larger improvement in their ITI, intervention fidelity, and youth behaviors as compared to the control. Conditional analyses indicated that teachers' ITI partially mediated the effect of SC-PIES on intervention fidelity, which in turn led to improved youth behaviors. Findings suggest that theory-informed pragmatic PIES targeting providers' ITI can boost their responsiveness to implementation strategies, as reflected in improved implementation behaviors and youth behavioral outcomes. The results have implications for targeting motivational mechanisms of behavior change and situating preventive implementation strategies at the intersection between the preparation and active implementation stages of an implementation process. Limitations and implications for research and practice are discussed. Clinicaltrials.gov: NCT05240222. Registered on: 2/14/2022. Retrospectively registered.  https://clinicaltrials.gov/show/NCT05240222.

Perceptions of professionals about factors affecting the implementation of early childhood support systems: A case study evaluation from an ecological perspective

This evaluation research looked at factors affecting collaboration across the various levels of implementation of early childhood support systems. An ecological conceptual framework was used to examine barriers and facilitators to implementing social and health care services from the perspective of the professionals involved at the national, state, and local level of government. A single case study following a qualitative research strategy was conducted with a total of 29 professionals from social development and health ministries, one municipality and a family health center responsible for implementing the Chile Grows with You system. The results show that at the national level, the main factors include socio-political aspects, funding, and the empathy that professional employees hold for peers who are at lower rungs in the hierarchy. At the state level workplace conditions play a key role. For the local level those factors include information system management and the political support of the local government. The discussion section emphasizes the relevance of considering the interdependence of such factors that influence implementation outcomes and the need to move away from a single program evaluation to a multilevel implementation analysis of public policy.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 1: study protocol for a hybrid type 2 effectiveness-implementation cluster-randomized trial

BACKGROUND

Serious mental illness (SMI) can have devastating consequences. Unfortunately, many patients with SMI do not receive evidence-based psychological treatment (EBPTs) in routine practice settings. One barrier is poor "fit" between EBPTs and contexts in which they are implemented. The present study will evaluate implementation and effectiveness outcomes of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) implemented in community mental health centers (CMHCs). TranS-C was designed to target a range of SMI diagnoses by addressing a probable mechanism and predictor of SMI: sleep and circadian problems. We will investigate whether adapting TranS-C to fit CMHC contexts improves providers' perceptions of fit and patient outcomes.

METHODS

TranS-C will be implemented in at least ten counties in California, USA (N = 96 providers; N = 576 clients), via facilitation. CMHC sites are cluster-randomized by county to Adapted TranS-C or Standard TranS-C. Within each county, patients are randomized to immediate TranS-C or usual care followed by delayed treatment with TranS-C (UC-DT). Aim 1 will compare TranS-C (combined Adapted and Standard) with UC-DT on improvements in sleep and circadian problems, functional impairment, and psychiatric symptoms. Sleep and circadian problems will also be tested as a mediator between treatment condition (combined TranS-C versus UC-DT) and functional impairment/psychiatric symptoms. Aim 2 will evaluate whether Adapted TranS-C is superior to Standard TranS-C with respect to provider perceptions of fit. Aim 3 will evaluate whether the relation between TranS-C treatment condition (Adapted versus Standard) and patient outcomes is mediated by better provider perceptions of fit in the Adapted condition. Exploratory analyses will (1) compare Adapted versus Standard TranS-C on patient perceptions of credibility/improvement and select PhenX Toolkit outcomes and (2) evaluate possible moderators.

DISCUSSION

This trial has the potential to (a) expand support for TranS-C, a promising transdiagnostic treatment delivered to patients with SMI in CMHCs; (b) take steps toward addressing challenges faced by providers in delivering EBPTs (i.e., high caseloads, complex patients, poor fit); and (c) advance evidence on causal strategies (i.e., adapting treatments to fit context) in implementation science.

TRIAL REGISTRATION

Clinicaltrials.gov NCT04154631. Registered on 6 November 2019. https://clinicaltrials.gov/ct2/show/NCT04154631.

The current state of international research on the effectiveness of school nurses in promoting the health of children and adolescents: An overview of reviews

OBJECTIVE

School nurses are engaging worldwide to promote and protect children's health. Many researchers who examined the effectiveness of the school nurse criticized the inadequate methodology employed in many of the studies. We therefore carried out an evaluation on the effectiveness of school nurses based on a rigorous methodological approach.

METHODS

In this overview of reviews we performed an electronic databank search and global research results on the effectiveness of school nurses were sought. We identified 1,494 records through database search. Abstracts and full texts were screened and summarized using the dual control principle. We summarized the aspects of quality criteria as well as the significance of the effectiveness of the school nurse. In the first step, k = 16 systematic reviews were summarized and evaluated following the AMSTAR-2 guidelines. In a second step, j = 357 primary studies included in these k = 16 reviews were summarized and assessed following the GRADE guidelines.

RESULTS

Research results on the effectiveness of school nurses show that school nurses play a key role in improving the health of children with asthma (j = 6) and diabetes (j = 2), results on combating obesity are less certain (j = 6). The quality of identified reviews is mostly very low with only six studies of medium quality, of which one identified as a meta-analysis. A total of j = 289 primary studies were identified. Approximately 25% (j = 74) of identified primary studies were either randomized controlled trials (RCT) or observational studies, of which roughly 20% (j = 16) had a low risk of bias. Studies with physiological variables such as blood glucose or asthma labeling led to higher quality results.

CONCLUSION

This paper represents an initial contribution and recommends further evaluation of the effectiveness of school nurses, particularly in the areas of mental health or children from low socioeconomic backgrounds. The general lack of quality standards in school nursing research should be integrated into the scientific discourse of school nursing researchers to provide robust evidence for policy planners and researchers.

Associations between teamwork and implementation outcomes in multidisciplinary cross-sector teams implementing a mental health screening and referral protocol

PURPOSE

Teams play a central role in the implementation of new practices in settings providing team-based care. However, the implementation science literature has paid little attention to potentially important team-level constructs. Aspects of teamwork, including team interdependence, team functioning, and team performance, may affect implementation processes and outcomes. This cross-sectional study tests associations between teamwork and implementation antecedents and outcomes in a statewide initiative to implement a standardized mental health screening/referral protocol in Child Advocacy Centers (CACs).

METHODS

Multidisciplinary team members (N = 433) from 21 CACs completed measures of team interdependence; affective, behavioral, and cognitive team functioning; and team performance. Team members also rated the acceptability, appropriateness, and feasibility of the screening/referral protocol and implementation climate. The implementation outcomes of days to adoption and reach were independently assessed with administrative data. Associations between team constructs and implementation antecedents and outcomes were tested with linear mixed models and regression analyses.

RESULTS

Team task interdependence was positively associated with implementation climate and reach, and outcome interdependence was negatively correlated with days to adoption. Task and outcome interdependence were not associated with acceptability, appropriateness, or feasibility of the screening/referral protocol. Affective team functioning (i.e., greater liking, trust, and respect) was associated with greater acceptability, appropriateness, and feasibility. Behavioral and cognitive team functioning were not associated with any implementation outcomes in multivariable models. Team performance was positively associated with acceptability, appropriateness, feasibility, and implementation climate; performance was not associated with days to adoption or reach.

CONCLUSIONS

We found associations of team interdependence, functioning, and performance with both individual- and center-level implementation outcomes. Implementation strategies targeting teamwork, especially task interdependence, affective functioning, and performance, may contribute to improving implementation outcomes in team-based service settings.

Assessment of digital risks in child and adolescent mental health services: A mixed-method, theory-driven study of clinicians' experiences and perspectives

Children and adolescents in the UK spend increasingly more time in the digital world, raising societal fears about digital risks in this age group. Professionals are not always aware of the ever-developing research or guidance available around digital safety. This gap underscores the need to understand current experiences and determinants of digital risk assessment, including clinicians' views on barriers and facilitators. A mixed-method design was used. Fifty-three clinicians working in child and adolescent mental health services (CAMHS) in South England took part in a survey; of these 12 took part in semi-structured interviews. A psychological model of behavioural change (COM-B: capabilities, opportunities, motivation and behaviour) guided the analyses. Survey data revealed that clinicians showed awareness and concerns for several digital risk issues but there appeared to be gaps in their knowledge and practice. Interview data revealed different factors influencing staff enquiry about digital risks in CAMHS. These included aspects of capabilities (knowledge and skills), opportunities (resources, organisational context and empowerment of youth), and motivations (habit change, emotional experiences, and professional identity/role). Targeting both staff-level and organisation-level barriers to digital risk assessments in CAMHS is crucial. This study informs service improvement to ensure that children and young people safely navigate the digital world.

Organization-level variation in therapists' attitudes toward and use of measurement-based care

PURPOSE

Despite significant interest in improving behavioral health therapists' implementation of measurement-based care (MBC)-and widespread acknowledgment of the potential importance of organization-level determinants-little is known about the extent to which therapists' use of, and attitudes toward, MBC vary across and within provider organizations or the multilevel factors that predict this variation.

METHODS

Data were collected from 177 therapists delivering psychotherapy to youth in 21 specialty outpatient clinics in the USA. Primary outcomes were use of MBC for progress monitoring and treatment modification, measured by the nationally-normed Current Assessment of Practice Evaluation-Revised. Secondary outcomes were therapist attitudes towards MBC. Linear multilevel regression models tested the association of theory-informed clinic and therapist characteristics with these outcomes.

RESULTS

Use of MBC varied significantly across clinics, with means on progress monitoring ranging from values at the 25th to 93rd percentiles and means on treatment modification ranging from the 18th to 71st percentiles. At the clinic level, the most robust predictor of both outcomes was clinic climate for evidence-based practice implementation; at the therapist level, the most robust predictors were: attitudes regarding practicality, exposure to MBC in graduate training, and prior experience with MBC. Attitudes were most consistently related to clinic climate for evidence-based practice implementation, exposure to MBC in graduate training, and prior experience with MBC.

CONCLUSIONS

There is important variation in therapists' attitudes toward and use of MBC across clinics. Implementation strategies that target clinic climate for evidence-based practice implementation, graduate training, and practicality may enhance MBC implementation in behavioral health.

Measuring clinician stuck points about trauma-focused cognitive behavior therapy: The TF-CBT Stuck Points Questionnaire

Although evidence-based treatments (EBTs) for youth trauma have been developed, trauma-informed EBTs are rarely used in community settings. Clinician concerns about evidence-based trauma treatment may be a barrier to adoption and delivery. However, few instruments to assess clinician beliefs about specific EBTs, such as trauma-focused cognitive behavior therapy (TF-CBT) are available. This study evaluated an instrument of clinician concerns about TF-CBT, the TF-CBT Therapist Stuck Points questionnaire, in a sample of community mental health clinicians training in a year-long TF-CBT community-based learning collaborative. The 26 items in the instruments, which aim to assess clinician views on child trauma treatment and TF-CBT, indicate preliminary psychometric support (i.e., item-total correlations, internal consistency, negative correlations with measures of attitudes towards evidence-based practice). Scores on the TF-CBT Therapist Stuck Points questionnaire revealed that, on average, clinicians expressed concerns about having children talk about their trauma in session, the effectiveness of certain TF-CBT components, and whether to involve caregivers in treatment. Clinician doubts could be targeted during a TF-CBT implementation effort and clinical supervision to facilitate treatment delivery. Implications for assessing TF-CBT specific beliefs during implementation are discussed.

Required sample size to detect mediation in 3-level implementation studies

Background

Statistical tests of mediation are important for advancing implementation science; however, little research has examined the sample sizes needed to detect mediation in 3-level designs (e.g., organization, provider, patient) that are common in implementation research. Using a generalizable Monte Carlo simulation method, this paper examines the sample sizes required to detect mediation in 3-level designs under a range of conditions plausible for implementation studies.

Method

Statistical power was estimated for 17,496 3-level mediation designs in which the independent variable (X) resided at the highest cluster level (e.g., organization), the mediator (M) resided at the intermediate nested level (e.g., provider), and the outcome (Y) resided at the lowest nested level (e.g., patient). Designs varied by sample size per level, intraclass correlation coefficients of M and Y, effect sizes of the two paths constituting the indirect (mediation) effect (i.e., X→M and M→Y), and size of the direct effect. Power estimates were generated for all designs using two statistical models—conventional linear multilevel modeling of manifest variables (MVM) and multilevel structural equation modeling (MSEM)—for both 1- and 2-sided hypothesis tests.

Results

For 2-sided tests, statistical power to detect mediation was sufficient (≥0.8) in only 463 designs (2.6%) estimated using MVM and 228 designs (1.3%) estimated using MSEM; the minimum number of highest-level units needed to achieve adequate power was 40; the minimum total sample size was 900 observations. For 1-sided tests, 808 designs (4.6%) estimated using MVM and 369 designs (2.1%) estimated using MSEM had adequate power; the minimum number of highest-level units was 20; the minimum total sample was 600. At least one large effect size for either the X→M or M→Y path was necessary to achieve adequate power across all conditions.

Conclusions

While our analysis has important limitations, results suggest many of the 3-level mediation designs that can realistically be conducted in implementation research lack statistical power to detect mediation of highest-level independent variables unless effect sizes are large and 40 or more highest-level units are enrolled. We suggest strategies to increase statistical power for multilevel mediation designs and innovations to improve the feasibility of mediation tests in implementation research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01235-2.

Child and Adolescent Psychiatry and Underrepresented Youth With School Attendance Problems: Integration With Systems of Care, Advocacy, and Future Directions

School attendance problems (SAPs) include full-day and partial absences (eg, missing classes, tardiness) as well as difficulties going to or remaining in school. SAPs also include nonparticipation in distance/hybrid learning formats or lack of access to necessary technology or equipment. SAPs are particularly prevalent among students of color, students in poverty, students with disabilities, English language learners, and migrant populations. SAPs are often part of a complicated clinical picture of mental health (eg, emotional, neurodevelopmental, conduct disorders) and somatic (eg, abdominal, cardiovascular, respiratory problems) challenges. These challenges are exacerbated by disparities in socioeconomic status, childhood adversities, family structure, and neighborhood-level factors that have an impact on mental health outcomes.¹ SAPs have serious negative consequences in childhood (eg, lower academic achievement, greater risk of dropout) and in adulthood (eg, lower lifetime earning potential, greater occupational and mental health problems). Unfortunately, underrepresented youth with SAPs often have less access to proper care, especially psychiatric care.

Attitudes Towards Evidence-Based Practice of Professionals Working with Children and Adolescents with Autism Spectrum Disorder in Bangladesh

Like in many lower-middle-income countries (LMIC), progress in implementing evidence-based practices (EBPs) for children with autism spectrum disorder (ASD) has been slow in Bangladesh. This cross-sectional study examined professionals' attitudes towards evidence-based practice (EBP) for children and adolescents with ASD and explored how providers' demographic factors are related to attitudes to and adoption of EBPs in Bangladesh. The sample consisted of 150 mental health professionals and special teachers from the urban area of Dhaka. Attitudes were assessed by the Evidence-based Practice Attitude Scale-36. Findings indicated that professionals have favorable attitudes towards EBP. Their attitudes varied depending on service settings (public clinical, private clinical, and special school) and caseload per year. Professionals who work in private and special school settings claimed to be more willing to adopt an EBP when required and perceived a higher fit of EBPs and their work than those in public clinical settings. The number of different EBPs used also differed by service setting. Every type of intervention (except medication) was used by more professionals in special schools than in private and public clinical settings. Many professionals reported few barriers to the implementation of EBPs. These findings indicate conditions that are often conducive to the implementation of EBPs. However, these results do not reflect the situation in rural areas, in which poverty is more widespread and the number of specialized professionals is low.

Testing a Theory of Implementation Leadership and Climate Across Autism Evidence-Based Interventions of Varying Complexity

Prominent theories within the field of implementation science contend that organizational leaders can improve providers' fidelity to evidence-based practices (EBPs) by using focused implementation leadership behaviors that create an organizational climate for EBP implementation. However, this work has been criticized for overreliance on nonspecific, self-report fidelity measures and poor articulation of the boundary conditions that may attenuate leadership and climate's influence. This study tests the predictions of EBP implementation leadership and climate theory on observed fidelity to three school-based EBPs for autism that vary in complexity: pivotal response training (PRT), discrete trial training (DTT), and visual schedules (VS). Educators in kindergarten to third-grade autism support classrooms in 65 schools assessed their principals' EBP implementation leadership and school EBP implementation climate prior to the school year. Mid-school year, trained observers rated educator fidelity to all three interventions. Expert raters confirmed PRT was significantly more complex than DTT or VS using the Intervention Complexity Assessment Tool for Systematic Reviews. Linear regression analyses at the school level indicated principals' increased frequency of EBP implementation leadership predicted a higher school EBP implementation climate, which in turn predicted higher educator fidelity to PRT-however, there was no evidence of a relationship between implementation climate and fidelity to DTT or VS. Comparing principals whose EBP implementation leadership was ±1 SD from the mean, there was a significant indirect association of EBP implementation leadership with PRT fidelity through EBP implementation climate (d = 0.49, 95% CI [0.04, 0.93]). Strategies that target EBP implementation leadership and climate may support fidelity to complex behavioral interventions.

Implementation of Evidence-Based Practices within Treatment-As-Usual and Evidence-Based Practice Initiatives

Publicly funded initiatives are underway to improve implementation of evidence-based practices (EBP) in youth mental health services. However, we know little about the success of these initiatives or about EBP implementation independent of such initiatives. We examined EBP implementation in a treatment as usual (TAU) state and in six states with publicly funded EBP initiatives (EBPIs). In Study 1, we examined providers' use of practices derived from the evidence base (PDEB) and their predictors among 780 providers in a TAU state. In Study 2, we conducted a systematic review of implementation strategies, outcomes, and predictors of EBP use in six state funded EBPIs. Study 1 suggests TAU providers use PDEB alongside practices without consistent research support; provider racial/ethnic minority status, learning theory orientation, and manual use predict greater PDEB use. Study 2 indicates EBPIs employ multiple recommended implementation strategies with variable outcomes across studies and measurement approaches. Predictors of EBP use in EBPIs also varied, though training, setting, and youth age were consistent predictors across studies. While sample differences and inconsistent measurement across studies made direct comparisons somewhat tenuous, rates of PDEB use in the TAU sample appeared similar to those in publicly funded EBPIs. However, two states reported comparisons with TAU samples and found higher EBP implementation under EBPI. Different predictors impacted EBP use in TAU versus EBPIs. Our findings highlight the need for improved evaluation of EBPIs including clear reporting standards for outcomes and more consistent, standardized measurement of EBP use in order to better understand and improve EBPIs.

Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem

BACKGROUND

Implementation science is at a sufficiently advanced stage that it is appropriate for the field to reflect on progress thus far in achieving its vision, with a goal of charting a path forward. In this debate, we offer such reflections and report on potential threats that might stymie progress, as well as opportunities to enhance the success and impact of the field, from the perspective of a group of US-based researchers.

MAIN BODY

Ten mid-career extramurally funded US-based researchers completed a "pre-mortem" or a group brainstorming exercise that leverages prospective hindsight to imagine that an event has already occurred and to generate an explanation for it - to reduce the likelihood of a poor outcome. We came to consensus on six key themes related to threats and opportunities for the field: (1) insufficient impact, (2) too much emphasis on being a "legitimate science," (3) re-creation of the evidence-to-practice gap, (4) difficulty balancing accessibility and field coherence, (5) inability to align timelines and priorities with partners, and (6) overly complex implementation strategies and approaches.

CONCLUSION

We submit this debate piece to generate further discussion with other implementation partners as our field continues to develop and evolve. We hope the key opportunities identified will enhance the future of implementation research in the USA and spark discussion across international groups. We will continue to learn with humility about how best to implement with the goal of achieving equitable population health impact at scale.