Individual and organizational factors that affect implementation of evidence-based practices for children with autism in public schools: a cross-sectional observational study

The Role of Implementation Climate in Moderating Educator Use of Evidence-Based Practices and Outcomes for Autistic Students

Ensuring effective use of evidence-based practice (EBP) for autism in schools is imperative due to the significantly increasing number of autistic students receiving school services each year. High-quality EBP use has proven challenging in schools. Research indicates implementation climate, or how EBP are supported, rewarded, and valued, and EBP resources are related to successful implementation. However, limited understanding of system-level contextual factors that impact EBP implementation for school-based providers makes development of appropriate implementation supports challenging. Understanding these factors is crucial for selecting and tailoring implementation strategies to support EBP scale up. In this observational study, California school-based providers (n = 1084) completed surveys related to implementation climate, leadership, autism experience and EBP implementation (use, competence, knowledge). Student outcomes included state level academic and behavioral indicators. Using an implementation science framework (Aarons et al., in Administration and Policy in Mental Health and Mental Health Services Research 38:4-23, 2011) and multilevel modeling, we examined the relationship between EBP Implementation and student outcomes and the moderation effects of provider and district level factors. Higher implementation climate predicted better EBP implementation outcomes, and proved more impactful when provider hands-on autism experience was low. Greater EBP resources predicted a higher percentage of students who met math standards only when district poverty level was high. Our findings suggested moderating effects on EBP implementation from both provider and system level factors. Implementation climate and resources may be especially key in addressing equity issues related to high poverty schools in which teachers often have less autism experience.

A mixed-methods evaluation of organization and individual factors influencing provider intentions to use caregiver coaching in community-based early intervention

BACKGROUND

Most psycho-social interventions contain multiple components. Practitioners often vary in their implementation of different intervention components. Caregiver coaching is a multicomponent intervention for young autistic children that is highly effective but poorly implemented in community-based early intervention (EI). Previous research has shown that EI providers' intentions, and the determinants of their intentions, to implement caregiver coaching vary across components. Organizational culture and climate likely influence these psychological determinants of intention by affecting beliefs that underlie attitudes, norms, and self-efficacy to implement an intervention. Research in this area is limited, which limits the development of theoretically driven, multilevel implementation strategies to support multi-component interventions. This mixed methods study evaluated the relationships among organizational leadership, culture and climate, attitudes, norms, self-efficacy, and EI providers' intentions to implement the components of caregiver coaching.

METHODS

We surveyed 264 EI providers from 37 agencies regarding their intentions and determinants of intentions to use caregiver coaching. We also asked questions about the organizational culture, climate, and leadership in their agencies related to caregiver coaching. We used multilevel structural equation models to estimate associations among intentions, psychological determinants of intentions (attitudes, descriptive norms, injunctive norms, and self-efficacy), and organizational factors (implementation climate and leadership). We conducted qualitative interviews with 36 providers, stratified by strength of intentions to use coaching. We used mixed-methods analysis to gain an in-depth understanding of the organization and individual-level factors.

RESULTS

The associations among intentions, psychological determinants of intentions, and organizational factors varied across core components of caregiver coaching. Qualitative interviews elucidated how providers describe the importance of each component. For example, providers' attitudes toward coaching caregivers and their perceptions of caregivers' expectations for service were particularly salient themes related to their use of caregiver coaching.

CONCLUSION

Results highlight the importance of multi-level strategies that strategically target individual intervention components as well as organization-level and individual-level constructs. This approach holds promise for improving the implementation of complex, multicomponent, psychosocial interventions in community-based service systems.

Attitudes Toward and Usage of Evidence-Based Mental Health Practices for Autistic Youth in Bangladesh and Germany: A Cross-Cultural Comparison

The implementation of evidence-based practices (EBPs) for autistic youth is a critical concern worldwide. Research examining factors facilitating the implementation of EBPs found that providers' attitudes are an important factor. In this study, we evaluated cross-cultural differences in attitudes toward and use of EBPs. We tested socio-demographic factors as predictors of attitudes, and attitudes as predictors of EBPs use among mental health professionals working with autistic youth in Bangladesh and Germany. We used purposeful sampling. Two-hundred-ninety-two professionals who worked in a clinical setting responded to the survey and fulfilled the inclusion criteria (101 in Bangladesh, 191 in Germany). Participants were asked to respond to nine subscales of the Evidence-Based Practice Attitude Scale-36 (EBPAS-36), to indicate which of nine types of treatments they used, and to provide sociodemographic data. Measurement invariance across countries could be established for four subscales of the EBPAS-36. Comparative analyses of attitudes showed that professionals in both countries were open to using EBPs, but German practitioners were more likely to use EBPs when they appealed to them. By contrast, Bangladeshi professionals claimed to be more likely to adopt an EBP when required and to be more willing to learn EBPs to enhance job security. The relationship between caseload and attitudes varied between countries. A broader variety of EBPs was used in Germany. The findings highlight the importance of considering country-specific factors when implementing EBPs. Directions for conducting comparative studies on mental health professionals' attitudes towards EBP including methodological considerations are discussed.

Protocol for a hybrid type 3 effectiveness-implementation trial of a pragmatic individual-level implementation strategy for supporting school-based prevention programming

BACKGROUND

For approximately one in five children who have social, emotional, and behavioral (SEB) challenges, accessible evidence-based prevention practices (EBPPs) are critical. In the USA, schools are the primary setting for children's SEB service delivery. Still, EBPPs are rarely adopted and implemented by front-line educators (e.g., teachers) with sufficient fidelity to see effects. Given that individual behavior change is ultimately required for successful implementation, focusing on individual-level processes holds promise as a parsimonious approach to enhance impact. Beliefs and Attitudes for Successful Implementation in Schools for Teachers (BASIS-T) is a pragmatic, multifaceted pre-implementation strategy targeting volitional and motivational mechanisms of educators' behavior change to enhance implementation and student SEB outcomes. This study protocol describes a hybrid type 3 effectiveness-implementation trial designed to evaluate the main effects, mediators, and moderators of the BASIS-T implementation strategy as applied to Positive Greetings at the Door, a universal school-based EBPP previously demonstrated to reduce student disruptive behavior and increase academic engagement.

METHODS

This project uses a blocked randomized cohort design with an active comparison control (ACC) condition. We will recruit and include approximately 276 teachers from 46 schools randomly assigned to BASIS-T or ACC conditions. Aim 1 will evaluate the main effects of BASIS-T on proximal implementation mechanisms (attitudes, subjective norms, self-efficacy, intentions to implement, and maintenance self-efficacy), implementation outcomes (adoption, reach, fidelity, and sustainment), and child outcomes (SEB, attendance, discipline, achievement). Aim 2 will examine how, for whom, under what conditions, and how efficiently BASIS-T works, specifically by testing whether the effects of BASIS-T on child outcomes are (a) mediated via its putative mechanisms of behavior change, (b) moderated by teacher factors or school contextual factors, and (c) cost-effective.

DISCUSSION

This study will provide a rigorous test of BASIS-T-a pragmatic, theory-driven, and generalizable implementation strategy designed to target theoretically-derived motivational mechanisms-to increase the yield of standard EBPP training and support strategies.

TRIAL REGISTRATION

ClinicalTrials.gov ID: NCT05989568. Registered on May 30, 2023.

Implementation leadership and implementation climate in context: A single organization intrinsic case study for implementation of digital measurement-based care

Background

Although studies have demonstrated that implementation leadership and climate are important constructs in predicting evidence-based practice (EBP) implementation, concrete descriptions of how they operate during organizational implementation efforts are lacking. This case study fills that gap through an in-depth description of an organization with effective implementation leadership that successfully built a strong implementation climate. This case study provides an illustration of implementation leadership and climate in tangible, replicable terms to assist managers, practitioners, and researchers in addressing the organizational context in their own implementation projects.

Method

A single organization, intrinsic case study was employed to paint a multifaceted picture of how one organization leveraged implementation leadership to strengthen a climate for the successful implementation of digital measurement-based care. The case was drawn from a cluster-randomized trial designed to test the effects of a leadership-focused implementation strategy on youth-level fidelity and clinical outcomes of digital measurement-based care. Following the completion of the trial, case study activities commenced. Descriptive summaries of multiple data sources (including quantitative data on implementation leadership and climate, coaching call and organizational alignment meeting recordings and notes, and development plans) were produced and revised iteratively until consensus was reached. Leadership actions were analyzed for corresponding dimensions of implementation leadership and climate.

Results

Specific actions organizational leaders took, as well as the timing specific strategies were enacted, to create a climate for implementation are presented, along with lessons learned from this experience.

Conclusion

This case study offers concrete steps organizational leaders took to create a consistent and aligned message that the implementation of a specific EBP was a top priority in the agency. The general approach taken to create an implementation climate provides several lessons for leaders, especially for EBPs that have broad implications across an organization.

"On an island by myself": implications for the inclusion of autistic students in self-contained classrooms in public elementary schools

Introduction

Autistic students have limited access to inclusive classes and activities in their schools. Principals and special education teachers who directly teach and administer programs for autistic elementary students can offer critical insight into factors, such as educators' attitudes, that may impact inclusive opportunities in schools. These attitudes may serve as barriers to or facilitators of promoting an inclusive school setting.

Methods

Semi-structured interviews with 26 elementary school principals and 26 special education teachers explored their experiences implementing evidence-based practices for autistic students (pivotal response training, discrete trial training, and visual schedules) in 26 self-contained classrooms in the United States. Autism-specific culture and inclusion emerged as a theme, which was analyzed for this paper.

Results

An inductive approach to thematic analysis revealed principals' and special education teachers' perspectives regarding the "autism-specific culture" in the school, including attitudes towards and inclusion of autistic students in self-contained classrooms in the broader school environment. Analysis of text related to "autism-specific culture" detailed aspects of inclusion, factors (i.e., barriers and facilitators) affecting inclusion, principals' and special education teachers' attitudes towards autistic students placed in self-contained classrooms, attitudes of other school staff towards teachers in self-contained classrooms, and recommendations to support an inclusive school environment for autistic students.

Discussion

Results suggest that valuing "equal" access to classes and activities for autistic students in self-contained classrooms may not be sufficient for promoting an inclusive school environment, Educators may benefit from targeted strategies to facilitate inclusion. Strategies range from supporting educators' attitudes and knowledge of autism to shifting physical aspects of the school environment (e.g., location of classrooms). Additional implications for supporting the true inclusion (i.e., inclusion that goes beyond physical inclusion) involves of autistic students in self-contained classrooms schools are discussed.

Needs and Attitudes Related to the Adoption of Evidence-Based Practices Among Latinx Mental Health Professionals

Empirically supported interventions are warranted to achieve desired clinical outcomes and improve service delivery. Thus, efforts to identify, adopt, and implement Evidence-Based Practices (EBPs) are underway across various Latinx communities, including Puerto Ricans, where there is a growing recognition and prevalence of mental health and substance use disorders. This study investigated the needs and attitudes toward EBPs among an interdisciplinary sample of mental health professionals in Puerto Rico. An anonymous survey was distributed to social workers, psychologists, and professional counselors (N = 237). Using structural equation modeling, four dimensions of attitudes towards EBPs (openness, divergence, appeal, and requirement) were regressed on various individual and organizational factors. Some socio-demographic characteristics, educational opportunities, and organizational factors significantly contributed to specific attitudes related to the adoption of EBPs. Female participants and those working in rural settings scored higher in the openness to innovation dimension. Greater organizational support and graduating from a private institution were associated with more divergence from research, while married individuals and those trained on EBPs scored lower on this dimension. The lack of exposure to EBPs in college and younger age predicted greater interest in the appeal of adopting an EBP intervention (i.e., would adopt an EBP if it made sense). No differences in attitudes toward EBPs were found by professional discipline or work setting (i.e., clinical, community, and schools). Recommendations to increase openness and interest in the appeal of EBPs among Spanish-speaking professional communities in Latin America are highlighted.

Exploring the acceptability, feasibility, and appropriateness of a communication-friendly classroom tool for use in Irish schools: A qualitative inquiry

BACKGROUND

Ten percent of the school-aged population have speech, language, and communication needs (SLCN) that impact access to the curriculum. Successful implementation of classroom-based SLCN interventions can reduce barriers to learning, thereby improving educational outcomes for this vulnerable population. The challenges of implementing innovations in educational settings are well-documented, yet limited studies have addressed such considerations when developing, and piloting universal level SLCN interventions for use in Irish schools.

METHODS

A qualitative exploratory study was undertaken to establish the acceptability, feasibility, and appropriateness of a universal level SLCN intervention. An advisory panel of teachers (n = 8) and children with SLCN (n = 2) were engaged as co-researchers in the study. The Communication Supporting Classrooms Observation Tool, developed as part of the Better Communication Project in the UK, was trialled across a diverse sample of school settings (n = 5). Semi-structured interviews were conducted with school practitioners and school leaders, and a deductive content analysis was undertaken using the domains of the Consolidation Framework for Implementation Research.

DISCUSSION

The observation tool was viewed as acceptable with suggested additions. Integrating use of the tool within existing data-informed, school self-evaluation processes aimed at supporting school improvement was noted as a potential means of supporting implementation. A knowledge gap in relation to school-based models of support for SLCN was identified which may negatively impact implementation. An implementation strategy targeting coherence, cognitive engagement and contextual integration is indicated if the tool is to be normalised into routine practice in Irish classrooms. Implementation needs appeared to vary at the school level.

CONCLUSIONS

The importance of early-stage exploration to guide implementation planning with regards to developing and testing universal level interventions for SLCN in schools is highlighted. Engaging an advisory panel provides important insights to guide implementation decisions. Findings suggest an adaptive design is required when planning implementation studies targeting classroom setting.

Parent Coaching in Early Intervention for Autism Spectrum Disorder: A Brief Report

Coaching caregivers of young children on the autism spectrum is a critical component of parent-mediated interventions. Little information is available about how providers implement parent coaching for children on the autism spectrum in publicly funded early intervention systems. This study evaluated providers' use of parent coaching in an early intervention system. Twenty-five early intervention sessions were coded for fidelity to established caregiver coaching techniques. We found low use of coaching techniques overall, with significant variability in use of coaching across providers. When providers did coach caregivers, they used only a few coaching strategies (e.g., collaboration and in-vivo feedback). Results indicate that targeted training and implementation strategies focused on individual coaching components, instead of coaching more broadly, may be needed to improve the use of individual coaching strategies. A focus on strengthening the use of collaboration and in-vivo feedback may be key to improving coaching fidelity overall.

The Interaction Between General and Strategic Leadership and Climate on Their Multilevel Associations with Implementer Attitudes Toward Universal Prevention Programs for Youth Mental Health: A Cross-Sectional Study

Emerging literature has highlighted the importance of discerning general and strategic organizational context (OC) factors (e.g., leadership and climate) and their interaction effect on individual implementation behaviors (e.g., attitudes toward evidence-based practices; EBPs) in youth mental healthcare. This study aimed to examine how leadership and climate (general and strategic) are associated with implementer attitudes toward EBPs across the individual and organizational levels and their interaction effect in schools. A series of multilevel models (MLMs) were fitted on a diverse sample of schools actively implementing universal prevention programs for youth mental health (441 implementers from 52 schools). The organization-level aggregates and individual educators' perceptions of general and strategic leadership and climate, and their interaction terms, were entered as level-2 and level-1 predictors of four attitudinal dimensions (Requirement, Openness, Appeal, and Divergence) based on their level of measurement. At the organizational level, higher levels of strategic leadership and climate, but not their general counterparts, were consistently associated with more favorable attitudes in all four dimensions. At the individual/within-school level, higher levels of perceived general and strategic leadership and climate were associated with more favorable attitudes of Requirement and Openness. At the organizational/between-school level, general climate moderated the positive effect of strategic climate on implementers' perception of appeal and divergence of EBPs. Our findings indicate that leaders should make data-based decisions to allocate resources on strategic and/or general leadership and climate to foster favorable staff attitudes toward EBPs based on the level of measurement, implementation-specificity, and attitudinal dimensions.

Climate for evidence-based mental health care implementation in Germany: psychometric investigation of the Implementation Climate Scale (ICS)

Organizational implementation climate is an important construct in implementation research to describe to what extent implementation is expected, supported, and rewarded. Efforts in bridging the research-practice gap by implementing evidence-based practice (EBP) can benefit from consideration of implementation climate. The Implementation Climate Scale (ICS) is a psychometrically strong measure assessing employees' perceptions of the implementation climate. The present cross-sectional study aimed at providing a German translation and investigating its psychometric properties. The translation followed standard procedures for adapting psychometric instruments. German psychotherapists (N = 425) recruited online completed the ICS, the Evidence Based Practice Attitudes Scale (EBPAS-36D) and the Intention Scale for Providers (ISP). We conducted standard item and reliability analyses. Factorial validity was assessed by comparing an independent cluster model of Confirmatory Factorial Analysis (ICM-CFA), a Bifactor CFA, a Second-order CFA and an (Bifactor) Exploratory Structural Equation Model (ESEM). Measurement invariance was tested using multiple-group CFA and ESEM, convergent validity with correlation analysis between the ICS and the ISP subjective norms subscale (ISP-D-SN). The mean item difficulty was pi = .47, mean inter-item correlation r = .34, and mean item-total correlation ritc = .55. The total scale (ω = 0.91) and the subscales (ω = .79-.92) showed acceptable to high internal consistencies. The model fit indices were comparable and acceptable (Second-order CFA: RMSEA [90% CI] = .077 [.069; .085], SRMR = .078, CFI = .93). Multiple-group CFA and ESEM indicated scalar measurement invariance across gender and presence of a psychotherapy license. Psychotherapists in training reported higher educational support for EBP than licensed psychotherapists (T = 2.09, p = .037, d = 0.25). The expected high correlation between the ICS and the ISP-D-SN was found (r = .59, p < .001). Results for the German ICS confirm good psychometric properties including validity.

Study Protocol: Multi-level Determinants of Implementation and Sustainment in the Education Sector

Evidence-based programs (EBPs) delivered in elementary schools show great promise in reducing risk for emotional and behavioral disorders (EBDs). However, efforts to sustain EBPs in school face barriers. Improving EBP sustainment thus represents a priority, but little research exists to inform the development of sustainment strategies. To address this gap, the Sustaining Evidenced-Based Innovations through Multi-level Implementation Constructs (SEISMIC) project will: (a) Determine if malleable individual, intervention, and organizational factors predict EBP treatment fidelity and modifications during implementation, sustainment, or both; (b) Assess the impact of EBP fidelity and modifications on child outcomes during implementation and sustainment; and (c) Explore the mechanisms through which individual, intervention, and organizational factors influence sustainment outcomes. This protocol paper describes SEISMIC, which builds upon a federally-funded RCT evaluating BEST in CLASS, a teacher-delivered program for K-3rd grade children at risk for EBDs. The sample will include 96 teachers, 384 children, and 12 elementary schools. A multi-level, interrupted time series design will be used to examine the relationship between baseline factors, treatment fidelity, modifications, and child outcomes, followed by a mixed-method approach to elucidate the mechanisms that influence sustainment outcomes. Findings will be used to create a strategy to improve EBP sustainment in schools.

Individual and Contextual Factors Associated with Classroom Teachers' Intentions to Implement Classroom Physical Activity

Classroom-based physical activity (CPA) is an evidence-based practice that improves student physical activity outcomes, but national data suggest implementation is insufficient in US classrooms. The purpose of this study was to examine individual and contextual factors associated with elementary school teachers' intentions to implement CPA. We collected input survey data from 181 classroom teachers (10 schools; 98.4% participation among eligible teachers) across three separate cohorts to examine associations between individual and contextual constructs and future CPA implementation intentions. Data were analyzed using multilevel logistic regression. Individual-level characteristics of perceived autonomy for using CPA, perceived relative advantage/compatibility of CPA, and general openness to educational innovations were positively associated with intentions to implement CPA (p < 0.05). Teacher perceptions of contextual factors such as administrator support for CPA were also associated with implementation intentions. This study adds to prior evidence about the importance of theoretically determined constructs for understanding behavioral intentions among front-line implementers such as classroom teachers. Additional research is needed to evaluate interventions designed to change malleable factors, including teachers' perceptions, as well as changing school environments so that teachers perceive more autonomy to use CPA and have the training and resources that build skills for implementation.

Educators' experiences and reflections on the implementation of evidence-based practices for autistic students in public schools during the COVID-19 pandemic

Introduction

In 2020, the COVID-19 pandemic forced many schools to close their doors and transition to remote learning, disrupting how autistic students received school-based services and support. While school structure changes were challenging for all students, autistic students were uniquely affected, considering their reliance on predictability and routine; moreover, education settings are where most autistic children receive services. Much has been studied regarding the use of evidence-based practices (EBPs) for autistic students in traditional school settings, yet little is known about how educators use EBPs in remote learning environments in the wake of the COVID-19 pandemic.

Method

In this study, we explore educators' experiences with EBP implementation at the height of the pandemic and educators' reflections of its impact on autistic students and their school systems. Qualitative data were collected from 81 educators (general educators, special educators, and paraeducators) in semi-structured interviews regarding EBP use at the onset of the pandemic.

Results

Four themes emerged from interviews: (1) pandemic and remote learning environment challenges to inclusion and EBP use; (2) EBP use adaptations for remote learning environments; (3) pandemic and remote learning environment benefits for EBP use; and (4) considerations for EBP use beyond the pandemic.

Conclusion

These findings elucidate educators' experiences using EBPs during the COVID-19 pandemic and highlight important areas of consideration for autism-focused EBP implementation as remote instruction continues to be a learning format. More research is needed to understand how to best implement EBPs for autistic students in this emerging instruction context.

Individual and organizational factors as predictors of early evidence-based practice adoption in Michigan high schools: Baseline data from an implementation trial

Background

Adolescents increasingly access mental health services in schools. School mental health professionals (SPs; school counselors, social workers, etc.) can offer evidence-based mental health practices (EBPs) in schools, which may address access gaps and improve clinical outcomes. Although some studies have assessed factors associated with EBP adoption in schools, additional research focusing on SP- and school-level factors is warranted to support EBP implementation as SPs’ mental health delivery grows.

Methods

Baseline data were collected from SPs at Michigan high schools participating in a statewide trial to implement SP-delivered cognitive behavioral therapy (CBT) to students. Models examined factors associated with attitudes about EBPs, implementation climate, and implementation leadership, and their associations with CBT knowledge, training attendance, and pre-training CBT delivery.

Results

One hundred ninety-eight SPs at 107 schools (87%) completed a baseline survey. The mean Evidence-Based Practice Attitude Scale (EBPAS) total score was 2.9, and school-aggregated mean scores of the Implementation Climate Scale (ICS) and Implementation Leadership Scale (ILS) were 1.83 and 1.77, respectively, all on a scale ranging from 0 ( low) to 4 ( high). ICS and ILS scores were lower than typically reported in clinical settings, while EBPAS scores were higher. School characteristics were not significantly associated with EBPAS, ICS, or ILS scores, but scores did differ by SP role. Higher EBPAS scores were associated with more CBT knowledge (average marginal effect for 1 SD change [AME] = 0.15 points) and a higher probability of training completion (AME = 8 percentage points). Higher ICS scores were associated with a higher probability of pre-training CBT delivery (AME = 6 percentage points), and higher ILS scores were associated with higher probability of training completion (AME = 10 percentage points).

Conclusions

Our findings suggest that SPs’ attitudes toward EBPs and organizational support were positively associated with early signs of implementation success. As schools increasingly fill the adolescent mental healthcare access gap, efforts to strengthen both provider attitudes toward EBP and strategic organizational factors supporting EBP delivery will be key to encouraging EBP uptake in schools.

Plain Language Summary

Schools are an important setting in which adolescents receive mental healthcare. We need to better understand how to implement evidence-based practices (EBPs) in this setting to improve student mental health. This study examined the attitudes and perceptions of school professionals (SPs) as key contributors to the implementation of a particular EBP, the delivery of cognitive behavioral therapy (CBT) in schools. The study found that implementation climate and leadership scores in participating schools were lower than scores typically reported in clinical settings, while scores for SP attitudes about EBP adoption were higher than typical scores in clinical settings. Results further suggest that SPs with more positive attitudes toward EBPs are more knowledgeable of CBT and more likely to complete a 1-day CBT training. We also found that higher implementation climate scores were associated with SPs reporting pre-training CBT delivery (although this association was not statistically significant), and more implementation leadership was associated with SPs completing the CBT training. These findings suggest that SP attitudes toward EBPs and organizational support in schools are positively associated with early signs of implementation success. Early, low-intensity efforts to (1) improve SP attitudes about mental health EBPs, and (2) increase schools’ support for implementation may scaffold more intensive implementation efforts in schools down the road.

Teacher attitudes toward evidence-based practices: Exploratory and confirmatory analyses of the school-adapted evidence-based practice attitude scale

Background

The Evidence-Based Practice Attitudes Scale (EBPAS) is widely used in implementation research, but it has not been adapted and validated for use among general education teachers, who are most likely to deliver evidence-based prevention programs in schools, the most common setting where youth access social, emotional, and behavioral health services.

Method

School-based stakeholders and a research team comprised of experts in the implementation of evidence-based practices in schools adapted the EBPAS for teachers (the S-EBPAS). The adapted instrument was administered to a representative sample ( n = 441) of general education teachers (grades K—5) to assess the reliability and internal consistency via factor analyses. The S-EBPAS included two forms (i.e., EBP-agnostic and EBP-specific item referents), therefore, a multiple-group confirmatory factor analysis (CFA) was also performed to establish measurement invariance between the two forms.

Results

After adaptation and refinement, a 9-item, 3-factor structure was confirmed, with the final model supporting three first-order factors that load onto a second-order factor capturing attitudes toward adopting evidence-based practices. Multiple-group CFA analyses of measurement invariance indicated there were no significant differences between the two forms.

Conclusions

Overall, this study provides a brief, flexible instrument capturing attitudes toward adopting EBPs that has high reliability and internal consistency, which support its use among general education teachers in school settings implementing evidence-based practices.

Plain Language Summary

The Evidence-Based Practice Attitudes Scale (EBPAS) is a popular instrument for measuring attitudes toward evidence-based practices (EBPs). This instrument provides valuable information during implementation initiatives, such as whether providers or front-line implementers have favorable attitudes toward a given practice. The EBPAS has been used in many different settings, such as in community-based mental health clinics, medical hospitals, and in child welfare. However, it's use in schools has been limited, and it has not yet been tested with general education teachers, who are key implementers of evidence-based practices in schools. In order to trust that the scores from an instrument are accurate, it needs to be evaluated when scaling it out to new populations and settings. One popular method to determine this is to use factor analysis, which was employed in this study. This study fills the identified gap by assessing the reliability (i.e., accuracy) and internal consistency of the EBPAS among a representative sample of general education teachers. Findings from this study indicate that the school-adapted EBPAS (S-EBPAS) is a brief, nine-item instrument that provides a reliable estimate of teachers’ attitudes toward evidence-based practices. Our results also provide evidence that the S-EBPAS can be used to capture attitudes toward specific EBPs as well as attitudes toward EBP -agnostic. This study provides a flexible instrument that can be used by school-based implementation researchers, practitioners, and intermediaries at multiple phases of implementation projects, such as when exploring a new EBP to adopt.

Implementation readiness for evidence-based autism practices in school systems

Background

The increase in the number of autistic children being identified has led to increased demand on public schools to provide high-quality services. Effectively scaling up evidence-based practice (EBP) use for autistic students is challenging, given the complicated organization of special education. Teachers have significant challenges implementing autism EBP with fidelity. Factors such as implementation leadership and climate and attitudes toward EBP are linked to successful EBP use and may vary at different levels of the education system. Examining mechanisms of successful implementation is a critical step to support scale-up.

Method

In this observational study, conducted from September 2018 to March 2020, California school personnel (n = 2273) at multiple levels of the system completed surveys related to implementation climate, leadership, and attitudes toward EBP. Data were collected throughout California at the Special Education Local Plan Areas, County Office of Education, and district and school levels from educators and administrators working in public schools supporting autistic students. Multi-level modeling was conducted to characterize implementation readiness.

Results

Overall, implementation climate and leadership scores are low across levels with regional levels rated more positively than districts or schools. Attitudes toward EBP were moderate, with those working in schools having the poorest ratings and specialists/trainers and related service providers (e.g., speech-language pathologists) having the highest ratings.

Conclusions

Outcomes provide a unique opportunity to compare implementation factors across organizational levels with a large, statewide sample. These data provide guidance for developing implementation interventions at multiple levels of the education system to increase readiness for effective scale-up of autism EBP in schools. Personnel and leaders at different organizational levels may need differentiated training targeting improved implementation climate and leadership. Personnel within districts and schools may experience a particular benefit from leadership support for EBP implementation.

Attitudes Towards Evidence-Based Practice of Professionals Working with Children and Adolescents with Autism Spectrum Disorder in Bangladesh

Like in many lower-middle-income countries (LMIC), progress in implementing evidence-based practices (EBPs) for children with autism spectrum disorder (ASD) has been slow in Bangladesh. This cross-sectional study examined professionals' attitudes towards evidence-based practice (EBP) for children and adolescents with ASD and explored how providers' demographic factors are related to attitudes to and adoption of EBPs in Bangladesh. The sample consisted of 150 mental health professionals and special teachers from the urban area of Dhaka. Attitudes were assessed by the Evidence-based Practice Attitude Scale-36. Findings indicated that professionals have favorable attitudes towards EBP. Their attitudes varied depending on service settings (public clinical, private clinical, and special school) and caseload per year. Professionals who work in private and special school settings claimed to be more willing to adopt an EBP when required and perceived a higher fit of EBPs and their work than those in public clinical settings. The number of different EBPs used also differed by service setting. Every type of intervention (except medication) was used by more professionals in special schools than in private and public clinical settings. Many professionals reported few barriers to the implementation of EBPs. These findings indicate conditions that are often conducive to the implementation of EBPs. However, these results do not reflect the situation in rural areas, in which poverty is more widespread and the number of specialized professionals is low.

A Mixed-Method Study Examining Solutions to Common Barriers to Teachers' Adoption of Evidence-Based Classroom Practices

Objectives

We conducted a mixed-method focus group study to (a) assess the appropriateness and likely effectiveness of strategies that target individual behavior change mechanisms associated with perceived barriers of lack of time and unsupportive leadership and (b) identify recommendations regarding strategies for overcoming the barriers.

Method

Sample included 39 school-based staff (80% female, 77% White) across two districts in the Midwest. Mixed methods included a simultaneous approach.

Results

Lack of time and supportive leadership continue to pervade school-based implementation efforts. Recommendations centered around the need for school leaders to give teachers the power to re-prioritize how they spend their time as well as providing protected, facilitated time for teachers to collaborate and learn practical skills targeting self-advocacy.

Conclusion

Our findings provide compelling evidence for the use of implementation methodology to strategically target mechanisms of individual behavior change during the process of incorporating new and innovative practices in schools.

Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS)

Background

Strategic implementation leadership is a critical determinant of successful implementation, hypothesized to create a more supportive implementation climate conducive to the adoption and use of evidence-based practices. Implementation leadership behaviors may vary significantly across contexts, necessitating studies that examine the validity of established measurement tools in novel health service delivery sectors. The education sector is the most common site for delivering mental health services to children and adolescents in the USA, but research focused on implementation leadership in schools is in the early phases, and there is a need for adaptation and expansion of instruments in order to tailor to the school context. The current study adapted and validated the School Implementation Leadership Scale (SILS) (based on the Implementation Leadership Scale) in a sample of elementary school personnel from six school districts who were implementing one of two well-established prevention programs for supporting children’s mental health.

Methods

Participants were 441 public school teachers from 52 elementary schools in the Midwest and West Coast of the USA. Participants completed a survey that contained: (1) an adapted and expanded version of the SILS with additional items generated for four existing subscales as well as three new subscales (communication, vision/mission, and availability), and (2) additional tools to evaluate convergent and divergent validity (i.e., measures of general/molar leadership and teaching attitudes). Data underwent (1) examination of item characteristic curves to reduce items and ensure a pragmatic instrument, (2) confirmatory factor analyses to establish structural validity, and (3) evaluation of convergent and divergent validity.

Results

Item reduction analyses resulted in seven subscales of three items each. Results indicated acceptable fit for a seven-factor structural model (CFI = .995, TLI = .99, RMSEA = .07, SRMR = 0.02). Second-order factor loadings were high (λ = .89 to .96), suggesting that the SILS subscales comprise a higher-order implementation leadership factor. All subscales demonstrated good inter-item reliability (α = .91–.96). Convergent and divergent validity results were generally as hypothesized, with moderate to high correlations between SILS subscales and general leadership, moderate correlations with teaching attitudes, and low correlations with school demographics.

Conclusions

Overall, results provided strong structural, convergent, and divergent validity evidence for the 21-item, 7-factor SILS instrument. Implications for the measurement of implementation leadership in schools are discussed, as well as strategies to support leaders to enhance their strategic behaviors related to the implementation of mental health prevention programs (e.g., adaptation of existing leadership-focused implementation strategies).

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01222-7.

Implementation fidelity, attitudes, and influence: a novel approach to classifying implementer behavior

Background

The current study sought to (1) describe a new classification approach for types of implementer behavior and (2) explore the implementer behavior change in response to tailored implementation facilitation based on the classifications.

Methods

A small-scale, cluster-randomized hybrid type III implementation trial was conducted in 38 early care and education classrooms that were part of the Together, We Inspire Smart Eating (WISE) program. WISE focuses on 4 evidence-based practices (EBPs), which are implemented by teachers to promote nutrition. External facilitators (N = 3) used a modified Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE) to complete immersion (i.e., observations) and thematic content analyses of interviews to identify the characteristics of teachers’ behavior at varying levels of implementation fidelity. Three key factors—attitudes toward the innovation, fidelity/adaptations, and influence—were identified that the research team used to classify teachers’ implementation behavior. This process resulted in a novel classification approach. To assess the reliability of applying the classification approach, we assessed the percent agreement between the facilitators. Based on the teachers’ classification, the research team developed a tailored facilitation response. To explore behavior change related to the tailored facilitation, change in fidelity and classification across the school year were evaluated.

Results

The classifications include (1) enthusiastic adopters (positive attitude, meeting fidelity targets, active influence), (2) over-adapting adopters (positive attitude, not meeting fidelity targets, active influence), (3) passive non-adopters (negative attitude, not meeting fidelity targets, passive influence), and (4) active non-adopters (negative attitudes, not meeting fidelity targets, active influence). The average percent agreement among the three facilitators for classification was 75%. Qualitative data support distinct patterns of perceptions across the classifications. A positive shift in classification was observed for 67% of cases between the mid-point and final classification. Finally, we generated an expanded classification approach to consider additional combinations of the three factors beyond those observed in this study.

Conclusions

Data from this study support the ability to apply the classification approach with moderate to high reliability and to use the approach to tailor facilitation toward improved implementation. Findings suggest the potential of our approach for wider application and potential to improve tailoring of implementation strategies such as facilitation.

The Correlation between Organizational (School) Climate and Teacher Job Satisfaction-The Type of Educational Institution Moderating Role

The quality of work and task implementation in educational institutions to a large extent depends on the personnel of those institutions. This is particularly true in the case of teaching pupils with special educational needs. The climate of education and learning and job satisfaction depend on teachers themselves. The purpose of this study was to verify the extent to which the type of educational institution (a mainstream or special school) moderates the correlation between teachers’ subjective evaluation of the organizational climate, their job satisfaction, and work-related emotions. The study involved 214 teachers representing all levels of school education in Poland, i.e., primary, middle, and high schools attended by pupils with special educational needs. Half of the teachers worked in mainstream schools and the other half—in special schools. The study used: The Organizational Climate Questionnaire by Litwin and Stringer; The Satisfaction with Job Scale; and The Work Affect Scale. The results clearly suggest that the teachers’ perception of the organizational climate is a strong predictor of their job satisfaction and work-related emotions. A positive climate is associated with high satisfaction and good emotions (enthusiasm and comfort) while a negative climate co-occurs with low satisfaction and bad emotions (anxiety and depression). The organizational type of institution (mainstream or special school) does not significantly affect the above predictive attribute of the organizational climate.

Investigating strategies to increase general education teachers' adherence to evidence-based social-emotional behavior practices: A meta-analysis of the single-case literature

Educational researchers have produced a variety of evidence-based practices (EBP) to address social, emotional, and behavioral (SEB) needs among students. Yet, these practices are often insufficiently adopted and implemented with fidelity by teachers to produce the beneficial outcomes associated with the EBP, leaving students at risk for developing SEB problems. If ignored, SEB problems can lead to other negative outcomes, such as academic failure. Therefore, implementation strategies (i.e., methods and procedures designed to promote implementation outcomes) are needed to improve teachers' uptake and delivery of EBPs with fidelity. This meta-analysis sought to examine the types and magnitude of effect of implementation strategies that have been designed and tested to improve teacher adherence to SEB EBPs. Included studies (a) used single case experimental designs, (b) employed at least one implementation strategy, (c) targeted general education teachers, and (d) evaluated adherence as a core dimension of fidelity related to the delivery of EBPs. In total, this study included 28 articles and evaluated 122 effect sizes. A total of 15 unique implementation strategies were categorized. Results indicated that, on average, implementation strategies were associated with increases in teacher adherence to EBPs above baseline and group-based pre-implementation trainings alone (g = 2.32, tau = 0.77). Moderator analysis also indicated that larger effects were associated with implementation strategies that used a greater number of unique behavior change techniques (p < .001). Implications and future directions for research and practice regarding use of implementation strategies for general education teachers are discussed.

Factors associated with implementation of a school-based comprehensive program for students with autism

LAY ABSTRACT

The purpose of this study was to identify influences on the implementation of a school-based, comprehensive program for autistic students. Influences from the community context, from within the school organization, and from features of the intervention itself were examined. The study took place in 39 elementary schools in a southeastern state in the United States. Transformational leadership style of the building principal, percentage of white students at the school (which was proposed as a broader community socioeconomic variable), and adequacy of coaching were all associated with implementation.

Assessment of Public Special Education Teachers Training Needs on Evidence-Based Practice for Students with Autism Spectrum Disorders in Spain

Over decades, the concern for the quality of psychoeducational practices for students with autism spectrum disorders has led to study to what extent are evidence-based educational methods disseminated among teachers. The purpose of this cross-sectional study, taking as reference Hsiao and Sorensen’s previous research, was to identify through a survey to what extent social-communication evidence-based practices for these students were provided in teacher education and in-service training programs, in a sample of 108 special education teachers from Spain, and to compare these results with Hsiao and Sorensen’s. Overall, more than 70% of the teachers reported that evidence-based practices in their teacher education programs (87.6%) and in-service training programs (73.6%) were never taught or mentioned incidentally. Finally, a higher percentage of addressing on each practice (i.e., mentioned and discussed or mentioned and taught through direct instruction) is shown in the sample of American teachers compared to the Spaniards, in both training paths.

Using evaluative frameworks to examine the implementation outcomes of a cognitive behavioral therapy program for autistic students with anxiety within public school settings

LAY ABSTRACT

Cognitive behavioral therapy helps to treat anxiety symptoms in autistic youth, but it is difficult for families to access cognitive behavioral therapy in the community. Training school providers to deliver cognitive behavioral therapy may help autistic youth and their families to access these programs. Unfortunately, we do not know how cognitive behavioral therapy programs can be delivered by school providers and how these programs help the autistic students who access them. This study addressed this gap and was part of a larger study that looked at the effectiveness of Facing Your Fears-School-Based in 25 public schools. The study goals were to understand whether Facing Your Fears-School-Based helped students and the factors that made it easy or difficult to deliver Facing Your Fears-School-Based in schools. Thirty providers participated in interviews guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework. Participants shared information that fell into several major categories that included (1) delivering Facing Your Fears-School-Based to many different students; (2) the positive impact of Facing Your Fears-School-Based on students' school participation; and (3) plans to continue using Facing Your Fears-School-Based. School providers also shared that Facing Your Fears-School-Based was easy to use for non-mental health providers and reported adapting Facing Your Fears-School-Based to meet student needs. The results of this study suggest that Facing Your Fears-School-Based may help autistic students and highlight the importance of using mental health programs in schools that are flexible, able to be adapted, and that are able to be used by many different types of school providers.

Organizational Context in General and Special Education: An Exploratory Investigation to Describe the Perspective of School Leaders

Organizational context (e.g., climate, culture, resources) can impede or enhance implementation of evidence-based practices in general education settings or special education settings serving students with autism spectrum disorder. We examined the relations between organizational context and individual (i.e., implementation leadership, administrator- or service provider-role) or school (i.e., enrollment size, public/nonpublic school type) characteristics. Participants were administrative or service providing leaders (n = 34) from 11 schools in one state on the East Coast of the United States. School leaders' average ratings of the organizational context were generally more positive for special education than general education; however, greater culture stress was reported for special education. Correlation analyses indicated being an administrator and implementation leadership were positively associated with implementation climate in both education settings. Being an administrator was also positively associated with cultural effort (i.e., how hard people work towards achieving goals) in special education, but negatively associated with culture stress in general education. In special education, nonpublic schools had better climates (both learning and implementation), but more culture stress. Additionally, school enrollment size was negatively related to available resources and implementation climate in special education. Investigating the similarities and differences in organizational context across educational settings is needed in future research.

Cluster-randomized trial comparing organizationally tailored versus standard approach for integrating an evidence-based cancer control intervention into African American churches

The field of implementation science has devoted increasing attention to optimizing the fit of evidence-based interventions to the organizational settings in which they are delivered. Institutionalization of health promotion into routine organizational operations is one way to achieve this. However, less is known about how to maximize fit and achieve institutionalization, particularly in settings outside of the healthcare system. This paper reports on findings from a parallel cluster-randomized trial that compared an organizationally tailored with a standard (core components only) approach for institutionalizing ("integrating") an evidence-based cancer control intervention into African American churches. Churches randomized to the organizationally tailored condition identified three or more implementation strategies from a menu of 20, with an implementation time frame for each. The primary study outcome was assessed through the Faith-Based Organization Health Integration Inventory, a measure of institutionalization of health promotion activities in church settings, completed by pastors at baseline and 12-month follow-up. Seventeen churches were randomized and 14 were analyzed as 3 did not implement the study protocol. Though the percent increase in total integration score was greater in the tailored condition (N = 9; 18%) than in the standard condition (N = 5; 12%), linear mixed-effect models did not detect a statistically significant group × time interaction. Despite the challenges of integrating health promotion activities outside of healthcare organizations, the current approach shows promise for fostering sustainable health promotion in community settings and merits further study.

Implementation of school-based services for students with autism: Barriers and facilitators across urban and rural districts and phases of implementation

LAY ABSTRACT

The law requires that schools use evidence-based practices to educate students with autism spectrum disorder. However, these practices are often not used, or are not used correctly in school programs. Understanding barriers and facilitators of use of evidence-based practices in schools will help improve the implementation process. This study uses focus groups to characterize how school-based providers representing urban or rural school districts perceive barriers and facilitators for implementing new practices for students with autism spectrum disorder. Guiding questions include the following: (1) Are contextual factors perceived as barriers or facilitators and how do these vary by district location? and (2) What are the key factors impacting implementation across the Exploration, Preparation, Implementation, and Sustainment phases? Focus group participants (n = 33) were service providers to children with autism spectrum disorder from urban- and rural-located school districts. Several personnel-related themes (attitudes and buy-in, knowledge and skills, staffing, and burnout) were shared by participants representing both urban and rural districts. However, some personnel-related themes and organizational factors were unique to rural or urban districts. For example, themes related to system and organizational factors (leadership approval, support and expectations, district structure, competing priorities, time for effective professional development, litigation and due process, and materials and resources) differed between the district locations. This project serves as an initial step in identifying implementation strategies that may improve the use of evidence-based practices in schools.

Cross-cultural adaption and psychometric investigation of the German version of the Evidence Based Practice Attitude Scale (EBPAS-36D)

BACKGROUND

The implementation of evidence-based practice (EBP) in mental health care confers many benefits to patients, and research into factors facilitating the implementation of EBP is needed. As an important factor affecting the implementation of EBP, service providers' attitudes toward EBP emerged. The Evidence-Based Practice Attitude Scale (EBPAS-36) is an instrument with good psychometric characteristics that measures positive and ambivalent attitudes toward EBP. However, a German version is missing. The present study therefore aims to provide a validated German translation of the EBPAS-36.

METHODS

The scale was translated and back-translated as recommended by standard procedures. German psychotherapists were recruited to participate in an online survey. They provided demographic and professional information, completed the EBPAS-36, the Implementation Climate Scale (ICS) and the Intention Scale for Providers (ISP). Standard item and reliability analyses were conducted. Construct validity was evaluated with exploratory (EFA) and confirmatory factor analyses (CFA) in two subsamples (random split). Convergent validity was tested by predicting a high positive correlation of the EBPAS-36D with two scores of attitudes of the ISP and an interest in EBP score. It was tested whether the EBPAS-36D predicts the intention to use EBP.

RESULTS

N = 599 psychotherapists participated in the study. The item analyses showed a mean item difficulty of p_i = 0.64, a mean inter-item correlation of r = 0.18, and a mean item-total correlation of r_itc = 0.40. The internal consistency was very good for the total scale (α = 0.89) and ranged from adequate to very good for the subscales (0.65-0.89), indicating high reliability. The original factor structure showed an acceptable model fit (RMSEA = 0.064 (90% CI = 0.059-0.068); SRMR = 0.0922; AIC = 1400.77), confirming the 12-factor structure of the EBPAS-36. However, a second-order factor structure derived by the EFA had an even better model fit (RMSEA = 0.057 (90% CI = 0.052-0.062); SRMR = 0.0822; AIC = 1274.56). When the EBPAS-36D was entered in a hierarchical regression model with the criterion Intention to use EBP, the EBPAS-36D contributed significantly to the prediction (Change in R² = 0.28, p < 0.001) over and above gender, age and participants' report of ever having worked in a university context.

CONCLUSIONS

The present study confirms good psychometric properties and validity of a German version of the EBPAS-36 in a sample of psychotherapists.

Embedding school cultures and climates that promote evidence-based practice implementation for youth with autism: A qualitative study

LAY ABSTRACT

Schools play a major role in providing services to youth with autism; however, not all schools use evidence-based practices, defined as interventions that are proven to improve youth well-being through rigorous research. School culture and climate are strong predictors of whether or not a school uses evidence-based practices; however, little is known about how principals can create school cultures and climates that support the use of these practices. This study interviewed 32 teachers in elementary schools that implemented three closely related evidence-based practices for youth with autism to better understand how principals create school cultures and climates that support effective services. Analysis of the teachers' responses identified seven strategies principals can use to create school cultures and climates that support the implementation of effective practices for youth with autism. The strategies include the following: (a) support teachers to obtain professional development focused on autism, (b) align performance expectations and evaluations with the needs of students with autism and evidence-based practice delivery, (c) allocate resources to ensure adequate staff, materials, and training are available to implement evidence-based practices, (d) be open and flexible to allow teachers to use the building and resources as needed to meet students' needs, (e) provide direct assistance, feedback, and coaching to troubleshoot challenges or involve outside experts to do so, (f) openly value the work of special education teachers and provide recognition to those who develop expertise in evidence-based practices, and (g) look for opportunities to integrate special and general education teachers and students to foster a truly inclusive climate.

Clinical Supervision in Community Mental Health: Characterizing Supervision as Usual and Exploring Predictors of Supervision Content and Process

Clinical supervision can be leveraged to support implementation of evidence-based practices in community mental health settings, though it has been understudied. This study focuses on 32 supervisors at 23 mental health organizations in Philadelphia. We describe characteristics of supervisors and organizations and explore predictors of supervision content and process. Results highlight a low focus on evidence-based content and low use of active supervision processes. They underscore the need for further attention to the community mental health context when designing supervision-targeted implementation strategies. Future work should assess whether supervision models specific to community mental health are needed.

Making Decision-Making Visible-Teaching the Process of Evaluating Interventions

Significant efforts in the past decades to teach evidence-based practice (EBP) implementation has emphasized increasing knowledge of EBP and developing interventions to support adoption to practice. These efforts have resulted in only limited sustained improvements in the daily use of evidence-based interventions in clinical practice in most health professions. Many new interventions with limited evidence of effectiveness are readily adopted each year-indicating openness to change is not the problem. The selection of an intervention is the outcome of an elaborate and complex cognitive process, which is shaped by how they represent the problem in their mind and is mostly invisible processes to others. Therefore, the complex thinking process that support appropriate adoption of interventions should be taught more explicitly. Making the process visible to clinicians increases the acquisition of the skills required to judiciously select one intervention over others. The purpose of this paper is to provide a review of the selection process and the critical analysis that is required to appropriately decide to trial or not trial new intervention strategies with patients.

Bringing Evidence-Based Interventions into the Schools: An Examination of Organizational Factors and Implementation Outcomes

Although schools are one of the largest providers of behavioral health services for youth, many barriers exist to the implementation of evidence-based interventions in schools. This study used the Stages of Implementation Completion (SIC) to examine school-based implementation outcomes for a computer-assisted cognitive behavioral therapy intervention for anxious youth. Organizational factors and predictors of program startup also were examined. Results indicated that the SIC detected implementation variability in schools and suggested that spending more time completing pre-implementation activities may better prepare schools for active implementation of program delivery. Furthermore, proficiency emerged as a potentially important organizational factor to examine in future school-based implementation research.

Protocol for a hybrid type 2 cluster randomized trial of trauma-focused cognitive behavioral therapy and a pragmatic individual-level implementation strategy

BACKGROUND

More than two-thirds of youth experience trauma during childhood, and up to 1 in 5 of these youth develops posttraumatic stress symptoms that significantly impair their functioning. Although trauma-focused cognitive behavior therapy (TF-CBT) has a strong evidence base, it is rarely adopted, delivered with adequate fidelity, or evaluated in the most common setting where youth access mental health services-schools. Given that individual behavior change is ultimately required for successful implementation, even when organizational factors are firmly in place, focusing on individual-level processes represents a potentially parsimonious approach. Beliefs and Attitudes for Successful Implementation in Schools (BASIS) is a pragmatic, motivationally focused multifaceted strategy that augments training and consultation and is designed to target precise mechanisms of behavior change to produce enhanced implementation and youth clinical outcomes. This study protocol describes a hybrid type 2 effectiveness-implementation trial designed to concurrently evaluate the main effects, mediators, and moderators of both the BASIS implementation strategy on implementation outcomes and TF-CBT on youth mental health outcomes.

METHODS

Using a cluster randomized controlled design, this trial will assign school-based mental health (SMH) clinicians and schools to one of three study arms: (a) enhanced treatment-as-usual (TAU), (b) attention control plus TF-CBT, or (c) BASIS+TF-CBT. With a proposed sample of 120 SMH clinicians who will each recruit 4-6 youth with a history of trauma (480 children), this project will gather data across 12 different time points to address two project aims. Aim 1 will evaluate, relative to an enhanced TAU condition, the effects of TF-CBT on identified mechanisms of change, youth mental health outcomes, and intervention costs and cost-effectiveness. Aim 2 will compare the effects of BASIS against an attention control plus TF-CBT condition on theoretical mechanisms of clinician behavior change and implementation outcomes, as well as examine costs and cost-effectiveness.

DISCUSSION

This study will generate critical knowledge about the effectiveness and cost-effectiveness of BASIS-a pragmatic, theory-driven, and generalizable implementation strategy designed to enhance motivation-to increase the yield of evidence-based practice training and consultation, as well as the effectiveness of TF-CBT in a novel service setting.

TRIAL REGISTRATION

ClinicalTrials.gov registration number NCT04451161 . Registered on June 30, 2020.

Strengthening capacity for implementation of evidence-based practices for autism in schools: The roles of implementation climate, school leadership, and fidelity

Understanding the factors that influence the use of evidence-based practices (EBPs) for autism spectrum disorder (ASD) in schools is critical to the selection of effective implementation strategies to support their sustained use. The current study has 2 aims: (a) evaluate the association between school leadership profiles (undifferentiated and optimal) and fidelity of EBP implementation and (b) examine the role of implementation climate as a mediator in this association. Participants included 56 principals, 90 special education teachers, and 133 classroom staff from 66 elementary schools. Participants completed the Multifactor Leadership Questionnaire and Implementation Climate Scale. Teachers and staff reported on intensity (frequency) of EBP delivery, and the research team rated the accuracy (adherence) to the components of the EBP. Schools were required to implement at least 1 of 3 EBPs for ASD (discrete trial training, pivotal response training, or visual supports). Using structural equation modeling, we found that schools with optimal leadership had higher observed ratings of teacher and staff fidelity for pivotal response training accuracy (p < .05), but not for discrete trial training or visual supports. However, this association became nonsignificant with the introduction of implementation climate into the models. Optimal leadership profiles were linked to more positive teacher/staff-reported implementation climate, compared with undifferentiated profiles (p < .01), but found no association between implementation climate and fidelity. Overall, the results of this study indicate that the role of principal leadership in EBP implementation is complex, which has implications for fostering a conducive organizational implementation context in schools. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Examining the Effects of a Brief, Group-Based Motivational Implementation Strategy on Mechanisms of Teacher Behavior Change

Training and consultation are core implementation strategies used to support the adoption and delivery of evidence-based prevention programs (EBPPs), but are often insufficient alone to effect teacher behavior change. Motivational interviewing (MI) and related behavior change techniques (e.g., strategic education, social influence, implementation planning) delivered in a group format offer promising supplements to training and consultation to improve EBPP implementation. Beliefs and Attitudes for Successful Implementation in Schools for Teachers (BASIS-T) is a theoretically informed, motivational implementation strategy delivered in a group format prior to and immediately after EBPP training. The purpose of this study was to examine the proximal effects of BASIS-T on hypothesized mechanisms of behavior change (e.g., attitudes, subjective norms, intentions to implement) in the context of teachers receiving training and consultation to implement the Good Behavior Game. As part of a pilot trial, 83 elementary school teachers from 9 public elementary schools were randomly assigned (at the school-level to reduce contamination across participants) to a BASIS-T (n = 44) or active comparison control (n = 39) condition, with both conditions receiving Good Behavior Game (GBG) training and consultation. A series of mixed effects models revealed meaningful effects favoring BASIS-T on a number of hypothesized mechanisms of behavior change leading to increased motivation to implement GBG. The implications, limitations, and directions for future research on the use of MI with groups of individuals and other behavior change techniques to increase the yield of training and consultation are discussed.

Partners in School: An Implementation Strategy to Promote Alignment of Evidence-Based Practices Across Home and School for Children with Autism Spectrum Disorder

When parents and teachers align their practices across home and school, it may optimize services for children with autism spectrum disorder (ASD). Partners in School is a multi-faceted implementation strategy designed to improve ASD services in schools. The goal is to increase parents' and teachers' use of evidence-based practices (EBPs) and to align those EBPs across settings. We piloted Partners in School with 49 parent-teacher dyads to assess administration and the factors associated with reported fidelity to the model. Specifically, we measured the number of intervention steps both parents and teachers completed (reported alignment) and the characteristics associated with intervention alignment. Partners in School involves parent-teacher participation in a pre-consultation interview, an in-person consultation meeting, active implementation of the same EBPs in their respective settings, and a post-consultation interview. Parents and teachers also completed surveys pre- and post-consultation. On average, parents and teachers completed approximately five EBP steps on their own in their respective settings (i.e., at home or at school). Of these five steps, parents and teachers both completed three of the same EBPs steps, on average. Different factors were related to reported alignment for parents versus teachers; however, a similarity noted for both parents and teachers was that communication variables were associated with reported alignment. Our findings indicate the important role of communication in aligning stakeholders for ASD service delivery models.

Pivotal response treatment: A study into the relationship between therapist characteristics and fidelity of implementation

In this study, we investigated the relationship between fidelity of pivotal response treatment implementation and therapist characteristics, such as therapist personality, therapist–child relationship, therapist attitude toward evidence-based practices, and therapist experience. We also explored whether child age and autism symptom severity were related to pivotal response treatment fidelity. Participants were 41 level III certified pivotal response treatment therapists who recorded three 10-min pivotal response treatment sessions and completed four questionnaires to measure therapist characteristics. Results indicated that therapists’ attitude toward evidence-based practices, specifically openness to innovation, and therapists’ experience with pivotal response treatment significantly predicted fidelity of pivotal response treatment implementation. Cross-validation methods largely confirmed these findings. Therapist personality, therapist–child relationship, and child characteristics were not significantly related to pivotal response treatment fidelity. Implications for clinical practice and directions for future research are discussed.

A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia

Background

Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static.

Method

Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia’s public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised.

Results

Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048).

Conclusions

System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.

The influence of a blended, theoretically-informed pre-implementation strategy on school-based clinician implementation of an evidence-based trauma intervention

BACKGROUND

Individual-level implementation determinants, such as clinician attitudes, commonly influence the successful adoption of evidence-based practices, but few explicit strategies have been tested with regard to their ability to impact these key mechanisms of change. This paper reports on an initial test of a blended, theoretically informed pre-implementation strategy designed to target malleable individual-level determinants of behavior change. Beliefs and Attitudes for Successful Implementation in Schools (BASIS) is a brief and pragmatic pre-implementation strategy that uses strategic education, social influence techniques, and group-based motivational interviewing to target implementation attitudes, perceived social norms, perceived behavioral control, and behavioral intentions to implement among mental health clinicians working in the education sector.

METHODS

As part of a pilot trial, 25 school mental health clinicians were randomized to BASIS (n = 12) or an attention control placebo (n = 13), with both conditions receiving training and consultation in an evidence-based intervention for youth experiencing trauma (the Cognitive Behavioral Intervention for Trauma in Schools). Theorized mechanisms of change (attitudes, perceived social norms, perceived behavioral control, and behavioral intentions) were assessed at baseline, post-training, and 4-month follow-up. Clinician participation in post-training consultation and intervention adoption were also tracked.

RESULTS

A series of regression models and independent sample t tests indicated that BASIS had significant, medium to large effects on the majority of its proximal mechanisms from baseline to post-training. BASIS was also associated with a greater latency between initial training in the intervention and discontinuation of participation in post-training consultation, with clinicians in the BASIS condition persisting in consultation for an average of 134 days versus 32 days for controls, but this difference was not statistically significant. At 4-month follow-up, most differences in the theorized mechanisms had attenuated, and approximately the same small number of BASIS clinicians adopted the trauma intervention as controls.

CONCLUSION

Findings suggest that the brief BASIS pre-implementation strategy had a significant influence on its proximal mechanisms of change, but that these changes did not persist over time or translate into adoption of the trauma intervention. Implications for theory refinement, revisions to the BASIS protocol, and next steps for research surrounding individual-level implementation strategies are discussed.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03791281 . Registered 31 December 2018-Retrospectively registered.