Individual and organizational characteristics predicting intervention use for children with autism in schools

Partnering with Schools to Adapt a Team Science Intervention: Processes and Challenges

Public schools are a major provider of mental health services for children in the US. Mental and behavioral health services range from universal programming to individualized clinical supports to address student needs. These services in schools are delivered by various professionals including non-teaching and teaching school personnel, school-employed clinicians, and/or contracted community mental health partners. Provision of mental health services requires complex coordination of providers across disciplines, although few professionals have training in multidisciplinary collaboration strategies. Attention to team processes, such as delineating team members' roles, improving communication, and identifying collaboration strategies, may impact the effectiveness of evidence-based mental health service provision in real world settings. One intervention, Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS), has been used in healthcare with positive outcomes and has been adapted to educational settings. The current paper describes the community-partnered subsequent adaptation of TeamSTEPPS for schools. Needs assessment interviews identified challenges, successes, and goals for student mental health. Overarching themes extracted from interviews include limited resources at multiple levels (e.g., financial support, time, and personnel), communication challenges, and poor role clarity. A community advisory board provided guidance during the adaptation and implementation planning process. Adaptations to the intervention included tailoring the intervention to the school context and the development of flexible training plans. In addition, individualized implementation plans were developed with each school partner to mitigate foreseeable barriers to rolling-out TeamSTEPPS. Our team is currently piloting the adapted TeamSTEPPS intervention and implementation strategies in partnership with three school districts.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12310-024-09665-7.

Organizational readiness and implementation fidelity of an early childhood education and care-specific physical activity policy intervention: findings from the Play Active trial

BACKGROUND

Many children do not accumulate sufficient physical activity for good health and development at early childhood education and care (ECEC). This study examined the association between ECEC organizational readiness and implementation fidelity of an ECEC-specific physical activity policy intervention.

METHODS

Play Active aimed to improve the ECEC educator's physical activity practices. We investigated the implementation of Play Active using a Type 1 hybrid study (January 2021-March 2022). Associations between organizational readiness factors and service-level implementation fidelity were examined using linear regressions. Fidelity data were collected from project records, educator surveys and website analytics.

RESULTS

ECEC services with higher levels of organizational commitment and capacity at pre-implementation reported higher fidelity scores compared to services with lower organizational commitment and capacity (all Ps < 0.05). Similarly, services who perceived intervention acceptability and appropriateness at pre-implementation to be high had higher fidelity scores (P < 0.05). Perceived feasibility and organizational efficacy of Play Active were associated with higher but nonsignificant fidelity scores.

CONCLUSIONS

Results indicate that organizational readiness factors may influence the implementation of ECEC-specific physical activity policy interventions. Therefore, strategies to improve organizational readiness should be developed and tested. These findings warrant confirmation in the ECEC and other settings and with other health behavior interventions.

Protocol for a hybrid type 3 effectiveness-implementation trial of a pragmatic individual-level implementation strategy for supporting school-based prevention programming

BACKGROUND

For approximately one in five children who have social, emotional, and behavioral (SEB) challenges, accessible evidence-based prevention practices (EBPPs) are critical. In the USA, schools are the primary setting for children's SEB service delivery. Still, EBPPs are rarely adopted and implemented by front-line educators (e.g., teachers) with sufficient fidelity to see effects. Given that individual behavior change is ultimately required for successful implementation, focusing on individual-level processes holds promise as a parsimonious approach to enhance impact. Beliefs and Attitudes for Successful Implementation in Schools for Teachers (BASIS-T) is a pragmatic, multifaceted pre-implementation strategy targeting volitional and motivational mechanisms of educators' behavior change to enhance implementation and student SEB outcomes. This study protocol describes a hybrid type 3 effectiveness-implementation trial designed to evaluate the main effects, mediators, and moderators of the BASIS-T implementation strategy as applied to Positive Greetings at the Door, a universal school-based EBPP previously demonstrated to reduce student disruptive behavior and increase academic engagement.

METHODS

This project uses a blocked randomized cohort design with an active comparison control (ACC) condition. We will recruit and include approximately 276 teachers from 46 schools randomly assigned to BASIS-T or ACC conditions. Aim 1 will evaluate the main effects of BASIS-T on proximal implementation mechanisms (attitudes, subjective norms, self-efficacy, intentions to implement, and maintenance self-efficacy), implementation outcomes (adoption, reach, fidelity, and sustainment), and child outcomes (SEB, attendance, discipline, achievement). Aim 2 will examine how, for whom, under what conditions, and how efficiently BASIS-T works, specifically by testing whether the effects of BASIS-T on child outcomes are (a) mediated via its putative mechanisms of behavior change, (b) moderated by teacher factors or school contextual factors, and (c) cost-effective.

DISCUSSION

This study will provide a rigorous test of BASIS-T-a pragmatic, theory-driven, and generalizable implementation strategy designed to target theoretically-derived motivational mechanisms-to increase the yield of standard EBPP training and support strategies.

TRIAL REGISTRATION

ClinicalTrials.gov ID: NCT05989568. Registered on May 30, 2023.

The perspective of school leaders on the implementation of evidence-based practices: A mixed methods study

Background

School leaders play an integral role in the use of implementation strategies, which in turn support special education teachers in the implementation of evidence-based practices (EBPs). In this convergent mixed methods study, we explored school leaders' perceptions of the facilitators and barriers to EBP implementation, particularly for students receiving special education, as well as the importance and feasibility of 15 implementation strategies.

Method

School leaders (N  =  22, principals, assistant principals, school psychologists, etc.) participated in a semistructured interview that included three parts-qualitative questions, quantitative ratings of strategies' importance and feasibility, and discussion of the top three implementation strategies. Data strands were analyzed independently and then integrated to generate meta-inferences.

Results

The qualitative data identified facilitators such as access to resources about the intervention (e.g., professional development) and collaboration and teamwork, while barriers centered on lack of school supports, culture/climate, and organizational factors (e.g., lack of communication). The quantitative data indicated that the implementation strategy provide ongoing consultation/coaching was rated as important and feasible. Monitor the progress of the implementation effort was rated as important but less feasible, while conduct educational meetings and change the environment were rated as feasible, but less important. Build partnerships to support implementation was rated as less important and feasible. There was convergence and divergence in mixed methods findings.

Conclusion

This study underscores the critical need to increase school leaders' knowledge and skills related to implementation science to better leverage implementation strategies that address the confluence of relevant implementation determinants.

Implementation Science in School-Based, Universal-Level Intervention Research: A Scoping Review

PURPOSE

The aim of this study was to map the use of implementation science frameworks, models, and theories in intervention research targeting learning needs in the classroom.

METHOD

A scoping review was conducted. Electronic database and manual searches were conducted. Two reviewers independently completed screening, data extraction, and quality appraisal. Qualitative content analysis was undertaken using Nilsen's taxonomy and the domains of the Consolidated Framework for Implementation Research (CFIR). The data were further analyzed using the CFIR valence and strength rating scales.

RESULTS

Included papers (n = 22) used a diverse sample of implementation science frameworks and models. Most studies used determinant frameworks to guide data collection and analysis. Few studies used implementation science theory. Most studies were mixed methods (n = 11), published since 2019 (n = 20), and conducted in North America (n = 15). Over half of the interventions targeted social, emotional, and mental health (n = 13). A complex interplay of inner setting factors was identified as having a strong influence on implementation. Teachers' knowledge and beliefs, adaptability, and complexity of interventions were also identified as important considerations when conducting research in this context.

CONCLUSIONS

Early engagement with stakeholders in education is recommended when designing universal level speech, language, and communication interventions for use in the ordinary classroom. Adaptive, hybrid designs that test both implementation strategies and effectiveness of interventions may be warranted given the influence of inner setting factors on implementation outcomes.

Applicability of the model of inclusive education in early childhood education: a case study

Despite the development of policies and research supporting it, inclusion remains a challenge in contemporary education. We have developed a theoretical model for implementing inclusive education, thereby supporting early childhood education quality. It is necessary to establish the applicability of this model in order to apply it to improve the practices for adopting inclusive education. We conducted a case study, which showed that all levels and key characteristics of the theoretical model were also relevant in practice. However, as a result of the case study, the features describing the key characteristics were modified compared with the initial model. Additionally, the case study revealed that some of the features did not appear in practice. Those undetected features were mostly related to understanding the concept of inclusive education and the philosophy of inclusion. There appeared a need for a clearer understanding of inclusion on both the institutional and state level. The implementation of inclusive education does not in itself always increase inclusion or reduce exclusion. Therefore, when implementing inclusive education, it is necessary to think carefully about what is being done to allow all children to be meaningfully involved in the same classroom and by their teachers.

Concordance of multiple informant assessment of school-based social skills intervention and association with child outcomes: Results from a randomized trial

Background

Fidelity, or the degree to which an intervention is implemented as designed, is essential for effective implementation. There has been a growing emphasis on assessing fidelity of evidence-based practices for autistic children in schools. Fidelity measurement should be multidimensional and focus on core intervention components and assess their link with program outcomes. This study evaluated the relation between intervention fidelity ratings from multiple sources, tested the relation between fidelity ratings and child outcomes, and determined the relations between core intervention components and child outcomes in a study of an evidence-based psychosocial intervention designed to promote inclusion of autistic children at school, Remaking Recess.

Method

This study extends from a larger randomized controlled trial examining the effect of implementation support on Remaking Recess fidelity and child outcomes. Schools were randomized to receive the intervention or the intervention plus implementation support. Observers, intervention coaches, and school personnel completed fidelity measures to rate completion and quality of intervention delivery. A measure of peer engagement served as the child outcome. Pearson correlation coefficients were calculated to determine concordance between raters. Two sets of hierarchical linear models were conducted using fidelity indices as predictors of peer engagement.

Results

Coach- and self-rated completion and quality scores, observer- and self-rated quality scores, and observer- and coach-rated quality fidelity scores were significantly correlated. Higher observer-rated completion and quality fidelity scores were predictors of higher peer engagement scores. No single intervention component emerged as a significant predictor of peer engagement.

Conclusions

This study demonstrates the importance of using a multidimensional approach for measuring fidelity, testing the link between fidelity and child outcomes, and examining how core intervention components may be associated with child outcomes. Future research should clarify how to improve multi-informant reports to provide "good enough" ratings of fidelity that provide meaningful information about outcomes in community settings.

Plain Language Summary

Fidelity is defined as how closely an intervention is administered in the way the creators intended. Fidelity is important because it allows researchers to determine what exactly is leading to changes. In recent years, there has been an interest in examining fidelity of interventions for autistic children who receive services in school. This study looked at the relationship between fidelity ratings from multiple individuals, the relationship between fidelity and child outcomes, and the relationship between individual intervention component and child changes in a study of Remaking Recess, an intervention for autistic children at school. Schools were randomly selected to receive the intervention only or the intervention plus implementation support from the research team. Observers, intervention coaches, and individuals delivering the intervention themselves completed fidelity measures. Child engagement with peers was measured before and after the intervention. Several measures of self-, coach-, and observer-report fidelity were associated with each other. Higher observer-reported fidelity was associated with higher child peer engagement scores. No single intervention step was linked to child peer engagement and both treatment groups had similar outcomes in terms of fidelity. This study shows the importance of having multiple raters assess different parts of intervention fidelity, looking at the link between fidelity and child outcomes, and seeing how individual intervention steps may be related to outcomes. Future research should aim to find out which types of fidelity ratings are "good enough" to lead to positive changes following treatment so that those aspects can be used and targeted in the future.

Individual and organizational factors as predictors of early evidence-based practice adoption in Michigan high schools: Baseline data from an implementation trial

Background

Adolescents increasingly access mental health services in schools. School mental health professionals (SPs; school counselors, social workers, etc.) can offer evidence-based mental health practices (EBPs) in schools, which may address access gaps and improve clinical outcomes. Although some studies have assessed factors associated with EBP adoption in schools, additional research focusing on SP- and school-level factors is warranted to support EBP implementation as SPs' mental health delivery grows.

Methods

Baseline data were collected from SPs at Michigan high schools participating in a statewide trial to implement SP-delivered cognitive behavioral therapy (CBT) to students. Models examined factors associated with attitudes about EBPs, implementation climate, and implementation leadership, and their associations with CBT knowledge, training attendance, and pre-training CBT delivery.

Results

One hundred ninety-eight SPs at 107 schools (87%) completed a baseline survey. The mean Evidence-Based Practice Attitude Scale (EBPAS) total score was 2.9, and school-aggregated mean scores of the Implementation Climate Scale (ICS) and Implementation Leadership Scale (ILS) were 1.83 and 1.77, respectively, all on a scale ranging from 0 (low) to 4 (high). ICS and ILS scores were lower than typically reported in clinical settings, while EBPAS scores were higher. School characteristics were not significantly associated with EBPAS, ICS, or ILS scores, but scores did differ by SP role. Higher EBPAS scores were associated with more CBT knowledge (average marginal effect for 1 SD change [AME] = 0.15 points) and a higher probability of training completion (AME = 8 percentage points). Higher ICS scores were associated with a higher probability of pre-training CBT delivery (AME = 6 percentage points), and higher ILS scores were associated with higher probability of training completion (AME = 10 percentage points).

Conclusions

Our findings suggest that SPs' attitudes toward EBPs and organizational support were positively associated with early signs of implementation success. As schools increasingly fill the adolescent mental healthcare access gap, efforts to strengthen both provider attitudes toward EBP and strategic organizational factors supporting EBP delivery will be key to encouraging EBP uptake in schools.

Plain Language Summary

Schools are an important setting in which adolescents receive mental healthcare. We need to better understand how to implement evidence-based practices (EBPs) in this setting to improve student mental health. This study examined the attitudes and perceptions of school professionals (SPs) as key contributors to the implementation of a particular EBP, the delivery of cognitive behavioral therapy (CBT) in schools. The study found that implementation climate and leadership scores in participating schools were lower than scores typically reported in clinical settings, while scores for SP attitudes about EBP adoption were higher than typical scores in clinical settings. Results further suggest that SPs with more positive attitudes toward EBPs are more knowledgeable of CBT and more likely to complete a 1-day CBT training. We also found that higher implementation climate scores were associated with SPs reporting pre-training CBT delivery (although this association was not statistically significant), and more implementation leadership was associated with SPs completing the CBT training. These findings suggest that SP attitudes toward EBPs and organizational support in schools are positively associated with early signs of implementation success. Early, low-intensity efforts to (1) improve SP attitudes about mental health EBPs, and (2) increase schools' support for implementation may scaffold more intensive implementation efforts in schools down the road.

Proactive versus Reactive: Strategies in the Implementation of School-based Services for Students with ASD

Autism spectrum disorder (ASD) is increasingly prevalent, intervention costs are high, and long-term outcomes are poor. Proactive implementation of evidence-based practices (EBPs; Steinbrenner et al., Evidence-Based Practices for Children, Youth, and Young Adults with Autism, 2020) through an assessment or planning process can lead to more effective services (Rubin et al., Admin Policy Mental Health Mental Health Serv Res 43(6), 1023-1028, 2016). The objective of this study is (a) to identify factors associated with the use of proactive versus reactive strategies for school-based services for students with ASD and (b) to examine school personnel perceptions of the use of proactive versus reactive strategies. Data were from a larger project in which 6 semi-structured focus groups were conducted to understand school personnel perceptions of how school districts implement new programs for ASD. Transcripts were coded using an iterative coding and review process. In the present study, primary themes were identified and classified as proactive or reactive. Participants reported both proactive and reactive allocation of district resources for intervention implementation. Reactive implementation was associated with litigation or due process, escalated student behavior, and non-public school placement, whereas proactive implementation was associated with ASD-specific programming, grants, personnel experience with EBPs, and prospective needs assessment. Participants perceived the reactive strategies as disadvantageous, and yet these strategies were sometimes still employed. Understanding the role of proactive and reactive strategies and the factors that influence their use could enable more effective planning for EBP use to improve both cost savings and student outcomes. This study is an important first step to explore resource allocation for school-based services for students with ASD.

Organizational Context in General and Special Education: An Exploratory Investigation to Describe the Perspective of School Leaders

Organizational context (e.g., climate, culture, resources) can impede or enhance implementation of evidence-based practices in general education settings or special education settings serving students with autism spectrum disorder. We examined the relations between organizational context and individual (i.e., implementation leadership, administrator- or service provider-role) or school (i.e., enrollment size, public/nonpublic school type) characteristics. Participants were administrative or service providing leaders (n = 34) from 11 schools in one state on the East Coast of the United States. School leaders' average ratings of the organizational context were generally more positive for special education than general education; however, greater culture stress was reported for special education. Correlation analyses indicated being an administrator and implementation leadership were positively associated with implementation climate in both education settings. Being an administrator was also positively associated with cultural effort (i.e., how hard people work towards achieving goals) in special education, but negatively associated with culture stress in general education. In special education, nonpublic schools had better climates (both learning and implementation), but more culture stress. Additionally, school enrollment size was negatively related to available resources and implementation climate in special education. Investigating the similarities and differences in organizational context across educational settings is needed in future research.

Protocol for a hybrid type 2 cluster randomized trial of trauma-focused cognitive behavioral therapy and a pragmatic individual-level implementation strategy

BACKGROUND

More than two-thirds of youth experience trauma during childhood, and up to 1 in 5 of these youth develops posttraumatic stress symptoms that significantly impair their functioning. Although trauma-focused cognitive behavior therapy (TF-CBT) has a strong evidence base, it is rarely adopted, delivered with adequate fidelity, or evaluated in the most common setting where youth access mental health services-schools. Given that individual behavior change is ultimately required for successful implementation, even when organizational factors are firmly in place, focusing on individual-level processes represents a potentially parsimonious approach. Beliefs and Attitudes for Successful Implementation in Schools (BASIS) is a pragmatic, motivationally focused multifaceted strategy that augments training and consultation and is designed to target precise mechanisms of behavior change to produce enhanced implementation and youth clinical outcomes. This study protocol describes a hybrid type 2 effectiveness-implementation trial designed to concurrently evaluate the main effects, mediators, and moderators of both the BASIS implementation strategy on implementation outcomes and TF-CBT on youth mental health outcomes.

METHODS

Using a cluster randomized controlled design, this trial will assign school-based mental health (SMH) clinicians and schools to one of three study arms: (a) enhanced treatment-as-usual (TAU), (b) attention control plus TF-CBT, or (c) BASIS+TF-CBT. With a proposed sample of 120 SMH clinicians who will each recruit 4-6 youth with a history of trauma (480 children), this project will gather data across 12 different time points to address two project aims. Aim 1 will evaluate, relative to an enhanced TAU condition, the effects of TF-CBT on identified mechanisms of change, youth mental health outcomes, and intervention costs and cost-effectiveness. Aim 2 will compare the effects of BASIS against an attention control plus TF-CBT condition on theoretical mechanisms of clinician behavior change and implementation outcomes, as well as examine costs and cost-effectiveness.

DISCUSSION

This study will generate critical knowledge about the effectiveness and cost-effectiveness of BASIS-a pragmatic, theory-driven, and generalizable implementation strategy designed to enhance motivation-to increase the yield of evidence-based practice training and consultation, as well as the effectiveness of TF-CBT in a novel service setting.

TRIAL REGISTRATION

ClinicalTrials.gov registration number NCT04451161 . Registered on June 30, 2020.