Implementation Science, Behavior Analysis, and Supporting Evidence-based Practices for Individuals with Autism

Efficacy of Community-Delivered PEERS® for Adolescents: Increases in Social Skills and Decreases in Social Anxiety and Loneliness

PURPOSE

PEERS® for Adolescents is an evidence-based social skills training program developed for individuals with autism spectrum disorder (ASD), which is now widely implemented by community providers in clinics and schools. However, majority of past efficacy studies on PEERS® were conducted in controlled research settings, with limited information about its effectiveness when delivered in the community. We sought to examine the effects of PEERS® on social functioning and mental health outcomes when delivered in an outpatient autism specialty clinic.

METHODS

Clinical data from 45 adolescents with social challenges (age range: 11-18 years old; 31.1% female assigned at birth) were extracted for secondary analyses. Paired t-tests were performed to examine the pre- to post-intervention changes in social and mental health outcomes. Correlations between pre- and post-change scores of outcome measures were examined.

RESULTS

Self-reported social skills knowledge, caregiver-reported social skills (measured by the Social Skills Improvement Systems) and the number of get-togethers hosted, increased significantly from pre- to post-intervention. Additionally, caregiver-reported anxiety and self-reported loneliness significantly decreased from pre- to post-intervention. Exploratory analyses showed that increases in caregiver-reported social skills were associated with decreases in self-reported loneliness.

CONCLUSIONS

Our findings provide evidence supporting the efficacy of PEERS® for improving social knowledge and skills of adolescents with social challenges when delivered in the community. The current study also showed the potential benefit of PEERS® for improving adolescent mental health.

Alliance for Scientific Autism Intervention: System Components and Outcome Data from High-Quality Service Delivery Organizations

Promoting excellence in autism intervention is arguably more urgent than ever for the field of applied behavior analysis. To fulfill this objective, autism agencies must operate from validated program systems and do so with fidelity. Program components include, but are not limited to, staff training and evaluation of clinical skills, functional personnel roles designed to promote positive outcomes for those served, and professional staff-communication-skill repertoires. Data on client outcomes must be tied to implementation of core program variables; and, contingencies between the data on client outcomes and staff performance must exist. Furthermore, these contingencies must be yoked across members of the organization to ensure a sustainable and effective program model. Finally, data on consumer satisfaction must be collected and used to evaluate program components and agency practices. Members of the Alliance for Scientific Autism Intervention have implemented key program-wide systems based upon the work of McClannahan and Krantz Journal of Applied Behavior Analysis, 26, 589-596 (1993) for decades and across various agency cultures. Data collected by six independent educational agencies on client outcomes, program implementation, and consumer feedback for a 10-year time span demonstrate the sustainability of the model and support the importance of key organizational systems and the relationship between implementation of the model and high-quality outcomes for individuals with autism.

Community-based care for autistic youth: community providers' reported use of treatment practices in the United States

Introduction

To illustrate the landscape of community-based care for autistic youth in the United States, we identified transdisciplinary psychosocial intervention practice sets that community providers report utilizing to care for this population, and examined characteristics associated with provider-reported utilization.

Methods

The Usual Care for Autism Study (UCAS) Survey assessed provider demographics and provider-reported use of transdisciplinary practices for common ASD co-occurring problems: social difficulties, externalizing behaviors, and anxiety. Community practitioners (N = 701) from allied health, behavioral, education, medical, mental health and other disciplines who treat or work with autistic youth (7-22 years) participated.

Results

Exploratory factor analysis yielded four factors: Consequence-Based Strategies (CBS), Cognitive-Behavioral and Therapy Strategies (CBTS), Antecedent-Based Strategies (ABS), and Teaching Strategies (TS). Providers across disciplines reported utilizing ABS more often than other sets. Providers from behavioral disciplines, with less than 4-year or Master degrees, or with more experience reported the most use of ABS, CBS and CBTS. Medical and behavioral providers reported the most use of TS. Setting and child characteristics were associated with practice set use, indicating variability by disability and client socioeconomic status.

Discussion

Findings reflect the complexity and inconsistency of the service landscape for autistic youth across the U.S. Only by understanding the service landscape and predictors of practice utilization, can researchers, policymakers, provider groups, and the autistic community facilitate effective implementation strategy development and use to ultimately improve community-based care.

Fifteen Years and Counting: The Dissemination Impact of Behavior Analysis in Practice

When Behavior Analysis in Practice (BAP) was founded 15 years ago, questions were raised about whether a practitioner-focused journal was really needed to complement our field's well-established applied research periodicals. Like research journals, BAP publishes primary research reports for which scholarly citations are one measure of impact. Unlike most research journals, it also was intended to achieve dissemination impact, which implies influence on people who may not conduct research or leave behind citations. Using altmetric data as an objective measure of dissemination impact, we present evidence that BAP is becoming a leader in this domain among applied behavior analysis journals, and thus appears to be accomplishing exactly what it was designed to. We recommend explicitly relying on dissemination impact data to help shape the journal's future development.

Qualitative Investigation into the Perception towards Compassionate Parenting among Parents of Autistic Children: Cross-Cultural Comparison between the UK and The Netherlands

Parenting a child with autism spectrum disorder is believed to present challenges that lead to increased levels of stress, as well as a reduction in the quality of the relationship between parent and child. This study aims to investigate parental perceptions toward a compassionate parenting style of parenting to better understand how this style may influence relationships and quality of life in parents. Parents from the United Kingdom (six parents) and the Netherlands (five parents) were invited take part in semi-structured interviews, whereby the data collected were subsequently analysed using thematic analysis. Data from both British and Dutch groups were overall similar to each other. Four themes were identified from the aggregated data: (a) "Parents believe compassionate parenting is important" (parents believed that compassion is an essential element of their parenting style and improves situational outcomes); (b) "Compassionate parenting de-escalates stressful situations" (compassionate parenting reduces stress and improves the quality of life); (c) "High pressure situations as a threat to practice compassion" (challenges and limitations to compassionate parenting style); and (d) "Greater public and professional awareness of autistic behaviours" (the general public and professional services often lack awareness to recognise autistic behaviour traits). Results are consistent with research examining the perceptions of parents of neurotypical children, in that a more compassionate approach to parenting is valued, as it is believed to create a greater connection with the child. Our findings inform researchers and educators as to what parents of children with ASD find useful, important, and worthwhile. Future research needs to investigate how compassionate parenting impacts autistic children's quality of life.

Practice Makes Proficient: Evaluation of Implementation Fidelity Following COMPASS Consultation Training

Objective

To test a training package for COMPASS, a multi-level consultation and coaching intervention for improved educational outcomes of students with ASD.

Method

Using a Hybrid Type 3 design with emphasis on implementation and a multidimensional approach to evaluating implementation outcomes, we tested a training package with community-based consultant trainees (CTs) unfamiliar with COMPASS and evaluated acceptance, appropriateness, feasibility, and fidelity from multiple sources (trainees, teachers, and parents).

Results

Results confirm that COMPASS-naïve CTs can be successfully trained. At least one feedback session was needed to achieve proficiency. Initial fidelity ratings between researchers and CTs were disparate suggesting self-report may not be adequate. Four feedback opportunities were required to achieve proficiency in writing intervention plans, an activity particularly challenging for CTs. Teachers and parents perceived COMPASS as acceptable, appropriate, and feasible. CTs knowledge of EBPs increased significantly following training; however, positive attitudes toward EBPs did not.

Conclusions

The implementation outcomes suggest that the training package was effective for training CTs; however, additional practice with writing intervention plans is warranted.

The development of the conversation skills assessment tool

Background and aims

Having a conversation with someone or even more within a group of people is complex. We are never taught at school how to do it, which implies we consider having a conversation as something simple and straightforward. Instead, we just learn from observing others. Some people are great conversationalists - it comes naturally to them - while others struggle. Some people may not fully understand how the process works, how turn-taking happens, don't understand visual cues such as body language and facial expressions, and fail to comprehend that some topics may be appropriate or inappropriate. This can be the case for both neurotypical and neurodivergent people. The Conversation skills Assessment Tool has been developed in this first instance to help in assessing and examining conversation skills in an intervention with young autistic adults on a virtual platform (a virtual world). This paper will present the evolution of the new measure through the exploratory phase, the development phase and finally a detailed account of the inter-rater reliability process.

Methods

The intervention associated with this study was carried out though a multiple baseline design with 3 autistic participants (in their early 20 s) and took place over 4 phases (15-17 sessions). The sessions involved semi-structured conversations in face-to-face (phases 1 and 4) and virtual (phases 2 and 3) settings and were videotaped with the participants' consent. Twelve of those were used by this study in the development process through iterative inter-rater reliability stages between two coding teams.

Results

Evaluation of the Conversation skills Assessment Tool tool revealed the potential benefit of implementing interventions with measures that more objectively and concretely (e.g., by noting frequencies) assess observable behaviours that are associated with having positive conversations with others. Beyond this, it is anticipated that Conversation skills Assessment Tool can emerge as a tool capable of not only accounting for the environment an interaction takes place in (e.g., professional, casual), but also offers beneficial feedback for both autistic students and other populations (e.g., young children, English language learners).

Conclusions

This measure has the potential to offer quantifiable and trackable guidance to people who have difficulties conversing. The authors do not wish to perpetuate an ableist social construct of what is a 'good' conversation, nor do they suggest that conversation skills training is useful solely for people with communication and/or socialization difficulties. Rather, they hope that Conversation skills Assessment Tool can be adopted more broadly to give both neurotypical and neurodivergent people a better understanding of how to communicate more effectively with others, while also becoming more aware and accepting of differing conversational styles.

Implications

Because of its ability to track (or self-monitor) one's development of conversational skills over time, Conversation skills Assessment Tool could serve as an educative tool in early childhood education. It can be used by occupational/speech therapists and other professionals and also used to self-monitor one's development of conversational skills. Conversation skills Assessment Tool was developed to assess conversation skills on a one-to-one basis; therefore, another iteration of Conversation skills Assessment Tool would have to look at group conversations.

Implementation readiness for evidence-based autism practices in school systems

Background

The increase in the number of autistic children being identified has led to increased demand on public schools to provide high-quality services. Effectively scaling up evidence-based practice (EBP) use for autistic students is challenging, given the complicated organization of special education. Teachers have significant challenges implementing autism EBP with fidelity. Factors such as implementation leadership and climate and attitudes toward EBP are linked to successful EBP use and may vary at different levels of the education system. Examining mechanisms of successful implementation is a critical step to support scale-up.

Method

In this observational study, conducted from September 2018 to March 2020, California school personnel (n = 2273) at multiple levels of the system completed surveys related to implementation climate, leadership, and attitudes toward EBP. Data were collected throughout California at the Special Education Local Plan Areas, County Office of Education, and district and school levels from educators and administrators working in public schools supporting autistic students. Multi-level modeling was conducted to characterize implementation readiness.

Results

Overall, implementation climate and leadership scores are low across levels with regional levels rated more positively than districts or schools. Attitudes toward EBP were moderate, with those working in schools having the poorest ratings and specialists/trainers and related service providers (e.g., speech-language pathologists) having the highest ratings.

Conclusions

Outcomes provide a unique opportunity to compare implementation factors across organizational levels with a large, statewide sample. These data provide guidance for developing implementation interventions at multiple levels of the education system to increase readiness for effective scale-up of autism EBP in schools. Personnel and leaders at different organizational levels may need differentiated training targeting improved implementation climate and leadership. Personnel within districts and schools may experience a particular benefit from leadership support for EBP implementation.

Bringing Assessment-to-Instruction (A2i) Technology to Scale: Exploring the Process from Development to Implementation

Bringing effective, research-based literacy interventions into the classroom is challenging, especially given the cultural and linguistic diversity of today's classrooms. We examined the promise of Assessment-to-Instruction (A2i) technology redesigned to be used at scale to support teachers' implementation of the individualized student instruction (ISI) intervention from kindergarten through third grade. In seven randomized controlled trials, A2i and ISI have demonstrated efficacy. However, the research version of A2i was not scalable. In order to bring A2i to scale in schools serving linguistically diverse students, we carried out the current study across two phases. This study represents both an exploration of what it takes to bring an educational intervention to scale (Phase 1) and a quasi-experiment on the literacy outcomes of learners whose teachers used the technology (Phase 2). We integrated assessments of vocabulary, word decoding, and reading comprehension; revised the A2i algorithms to account for the constellation of skills English learners (ELs) bring to the classroom; updated the user interfaces and added new graphic features; and improved bandwidth and stability of the technology. Findings were mixed, including several non-significant results, a marginally significant intent-to-treat effect on word reading in kindergarten and first grade for English monolingual students and ELs, and one significant interaction effect, which suggested ELs and students with less developed reading skills in second and third grade benefitted most from the intervention. With some caution, we conclude that A2i demonstrates potential to be used at scale and promise of effectiveness for improving code-focused skills for diverse learners.

Predictors of Attrition in a Randomized Trial of a Social Communication Intervention for Infant-Toddlers at Risk for Autism

Addressing factors that make it more likely for families to attrite from early intervention trials will allow researchers to ensure that families reap the full benefits of participation. This study was an analysis of 78 children (Mage = 18.38 months, SD = 5.78) at risk for autism participating in a university-based randomized controlled trial of two 8-week long early intervention programs. Overall, attrition through 8-weeks was low, approximately 13%, however by the one-year follow-up attrition rates were approximately 50%. The most consistent predictor of attrition was the distance that families had to travel to the university. These data highlight the importance of providing services and support (e.g., financial and logistic) during follow-up to families to maximize their participation. Clincaltrials.gov Identifier: NCT01874327, 6/11/2013.

Patient Outcomes After Applied Behavior Analysis for Autism Spectrum Disorder

OBJECTIVES

The purpose of this study was to examine patterns of service receipt and patient outcomes for children receiving applied behavior analysis (ABA) for autism spectrum disorder (ASD) in an integrated health care system in which commercially insured children were covered by a state autism mandate.

METHODS

This retrospective, observational study used a random sample of children with ASD (3-17 yrs) who were members of a large integrated health care system in Southern California and referred for ABA between January 2016 and November 2018. From the 4145 children referred, a random stratified sample of 334 was selected to extract data from clinical reports over 24 months of services. The primary outcome measures were time in ABA and child adaptive behavior.

RESULTS

Thirteen percent of the sample never received ABA after referral. Of those who were referred for ABA, 66% initiated ABA and remained in services for 12 months, whereas less than half (46%) remained in services for 24 months. Having a history of special education was associated with longer time spent in ABA, whereas having a single parent was associated with discontinuation of ABA. A minority of children received a full ABA dose (28%), but the lowest functioning children still experienced clinically significant adaptive behavior gains after 24 months of ABA (p = 0.02).

CONCLUSION

In a health system implementation of ABA for children with ASD, there were high rates of ABA discontinuation and low ABA dosing. These challenges may diminish the potential benefits of ABA, even in a context in which there is mandated commercial insurance coverage.

Educational Interventions for Children and Youth with Autism: A 40-Year Perspective

Commemorating the 40 th anniversary of the Diagnostic and Statistical Manual (DSM) III, the purpose of this commentary is to describe school-based and school-relevant interventions and instructional approaches for children and youth with autism that have been developed and employed during that time period. The commentary begins with a brief description of foundational research that provides an historical context. Research themes shaped by science, ethics, social policy, and the changes in the DSM provide an organization for describing the evolution of intervention and instructional practices over the four previous decades. The commentary concludes with a discussion of school-contextual variables that influence implementation and the promise of the "iSciences" for closing the research to practice gap in the future.

How to Be RAD: Repeated Acquisition Design Features that Enhance Internal and External Validity

The Repeated Acquisition Design (RAD) is a type of single-case research design (SCRD) that involves repeated and rapid measurement of irreversible discrete skills or behaviors through pre-and postintervention probes across different sets of stimuli. Researchers interested in the study of learning in animals and humans have used the RAD because of its sensitivity to detect immediate changes in rate or accuracy. Despite its strengths, critics of the RAD have cautioned against its use due to reasonable threats to internal validity like pretest effects, history, and maturation. Furthermore, many methodologists and researchers have neglected the RAD in their SCRD standards (e.g., What Works Clearinghouse [WWC], 2020; Horner et al., 2005). Unless given guidance to address threats to internal validity, researchers may avoid the design altogether or continue to use a weak version of the RAD. Therefore, we propose a set of 15 quality RAD indicators, comprising foundational elements that should be present in all RAD studies and additional features that enhance causal inference and external validity. We review contemporary RAD use and describe how the additional features strengthen the rigor of RAD studies. We end the article with suggested guidelines for interpreting effects and the strength of the evidence generated by RAD studies. We invite researchers to use these initial guidelines as a jumping off point for a more RAD future.

From Someone Who May Cause Trouble to Someone You Can Play With: Stakeholders' Perspectives on Preschool Program Quality for Autistic Children

In Sweden, young autistic children typically attend community-based preschool programs, which may not be adapted to their needs. In the current study, stakeholders to autistic children receiving Early Intensive Behavioral Intervention were interviewed following a quasi-randomized study (#NCT03634761) aimed at improving the preschool program quality using the Swedish version of the Autism Program Environment Rating Scale (APERS). Stakeholders provided their perceptions and experiences concerning key factors for high quality preschool programs as well as well as their experiences of the abovementioned APERS study. Applying thematic analysis, stakeholder groups differed in what they emphasized, but all highlighted staff’s competence, children’s inclusion and participation, collaboration, and the learning environment as key program areas that had been positively influenced by the APERS-based intervention.

A Multi-Site Feasibility Assessment of Implementing a Best-Practices Meet-And-Greet Intervention in Animal Shelters in the United States

Animal shelters must incorporate empirically validated programs to increase life-saving measures; however, altering existing protocols is often a challenge. The current study assessed the feasibility of nine animal shelters within the United States to replicate a validated procedure for introducing an adoptable dog with a potential adopter (i.e., "meet-and-greet") following an educational session. Each of the shelters were first entered into the "baseline" condition, where introduction between adoptable dogs and potential adopters were as usual. After a varying number of months, each shelter entered into the "experimental" phase, where staff and volunteers were taught best practices for a meet-and-greet using lecture, demonstration, and role-play. Data on the likelihood of adoption following a meet-and-greet were collected with automated equipment installed in meet-and-greet areas. Data on feasibility and treatment integrity were collected with questionnaires administered to volunteers and staff followed by a focus group. We found that a single educational session was insufficient to alter the meet-and-greet protocol; challenges included not remembering the procedure, opposing opinions of volunteers and staff, lack of resources, and a procedural drift effect in which the protocol was significantly altered across time. In turn, no animal shelters increased their dog adoptions in the "experimental" phase. New research is needed to develop effective educational programs to encourage animal shelters to incorporate empirical findings into their protocols.