Preliminary effectiveness of the ACT SMART implementation toolkit: facilitating evidence-based practice implementation in community-based autism organizations

Reported quality indicators and implementation outcomes of community partnership in autism intervention research: A systematic review

There is minimal research on the quality of community partnerships in studies of interventions for autistic children. However, building high quality community engagement in autism intervention research may improve implementation outcomes. This systematic review examined studies that report community partnership in autism intervention research. A total of 135 articles were identified and 11 of these articles were included in the final review. Community partnership data were extracted using indicators from the conceptual framework for assessing research-practice partnerships (RPP; Henrick et al., Henrick et al., Assessing research-practice partnerships: Five dimensions of effectiveness, William T. Grant Foundation, 2017) and implementation outcomes data were extracted using the taxonomy of distinct implementation outcomes (Proctor et al., Administration and Policy in Mental Health and Mental Health Services Research, 38:65-76, 2011). Quality of studies were appraised using JBIs critical appraisal tools (Munn et al., JBI Evidence Synthesis, 18:2127-2133, 2020). RPP indicators and implementation outcomes were variably reported across studies. RPP indicators and implementation outcomes more likely to be reported were related to building trust, cultivating partnership relationships, conducting rigorous research to inform action, acceptability, and feasibility. RPP indicators and implementation outcomes less likely to be reported were related to building capacity to engage in partnership work, sustainability, cost, and penetration. Together, these results may suggest the need for increased sustainability and capacity building efforts in partnerships and increased guidelines for reporting outcomes.

Fidelity to the ACT SMART Toolkit: an instrumental case study of implementation strategy fidelity

BACKGROUND

Evidence-based practices (EBPs) are shown to improve a variety of outcomes for autistic children. However, EBPs often are mis-implemented or not implemented in community-based settings where many autistic children receive usual care services. A blended implementation process and capacity-building implementation strategy, developed to facilitate the adoption and implementation of EBPs for autism spectrum disorder (ASD) in community-based settings, is the Autism Community Toolkit: Systems to Measure and Adopt Research-based Treatments (ACT SMART Toolkit). Based on an adapted Exploration, Adoption decision, Preparation, Implementation, Sustainment (EPIS) Framework, the multi-phased ACT SMART Toolkit is comprised of (a) implementation facilitation, (b) agency-based implementation teams, and (c) a web-based interface. In this instrumental case study, we developed and utilized a method to evaluate fidelity to the ACT SMART Toolkit. This study responds to the need for implementation strategy fidelity evaluation methods and may provide evidence supporting the use of the ACT SMART Toolkit.

METHODS

We used an instrumental case study approach to assess fidelity to the ACT SMART Toolkit during its pilot study with six ASD community agencies located in southern California. We assessed adherence, dose, and implementation team responsiveness for each phase and activity of the toolkit at both an aggregate and individual agency level.

RESULTS

Overall, we found that adherence, dose, and implementation team responsiveness to the ACT SMART Toolkit were high, with some variability by EPIS phase and specific activity as well as by ASD community agency. At the aggregate level, adherence and dose were rated notably lowest during the preparation phase of the toolkit, which is a more activity-intensive phase of the toolkit.

CONCLUSIONS

This evaluation of fidelity to the ACT SMART Toolkit, utilizing an instrumental case study design, demonstrated the potential for the strategy to be used with fidelity in ASD community-based agencies. Findings related to the variability of implementation strategy fidelity in the present study may also inform future adaptations to the toolkit and point to broader trends of how implementation strategy fidelity may vary by content and context.