Iterative redesign of a caregiver-mediated intervention for use in educational settings

Short report: Examining race, ethnicity, sex, and gender among autistic youth and their educators who participated in school-based research

LAY ABSTRACT

Schools are the main provider of behavioral health services in the United States (US). Conducting research in schools may help increase access for autistic youth, but there is limited research about who participates in school-based autism research. The researchers examined data spanning 14 years of participation in school-based autism research. All data were collected in the US. Researchers found that in this sample, more than half of the youth identified as racial/ethnic minoritized (REM) youth. However, demographics of sampled youth were significantly different than expected, based on publicly available school-based population demographics. More youth identified as White non-Hispanic than would be expected in those schools. Majority of the sampled educators also identified as White non-Hispanic. High percentages of autistic REM youth suggest school-based autism research may be an effective way of providing services to more diverse populations. However, sampled schools were also not representative of a US national sample, with sampled schools having fewer White non-Hispanic youth than would be expected from US national statistics. Complexity around representation of schools sampled and demographics of service environments suggests a need for more research.

Applying a User-Centered Design Framework to Develop a Remote Research Assessment Protocol for a Randomized Clinical Trial for Toddlers with Early Autism Characteristics

COVID-19 required many research teams to shift from in-person to remote assessments, which posed both procedural and theoretical challenges. While research has explored the utility of remote assessments for autism diagnosis from the perspective of families and clinicians, less is known about their application in clinical trials. This paper describes the development of a remote research assessment protocol for a randomized clinical trial focusing on the implementation of reciprocal imitation teaching (RIT) with toddlers in Part C early intervention. This project spans two phases. For Phase 1, our team developed and documented a series of steps utilizing user-centered design (UCD) strategies (e.g., recruiting potential users, creating a prototype, engaging in iterative development) for the purpose of redesigning an assessment protocol for a remote environment. For Phase 2, we examined preliminary outcomes of the redesign process. Primary end users (assessors) rated post-redesign usability and acceptability, while acceptability was examined using attrition data from secondary end users (family participants). Preliminary fidelity of implementation was also examined. The iterative redesign process allowed the research team to refine aspects of the assessment that ultimately led to promising preliminary ratings of usability, acceptability, and feasibility, as well as high fidelity. Preliminary data suggest that the redesigned assessment appears to be an acceptable, feasible, and usable tool for autism clinical trial research and that assessors can use it with fidelity. Further research is needed to examine the reliability and validity of the assessment, as well as implementation characteristics on a larger scale.

Formative Evaluation of a Comprehensive Self-Management Intervention for Irritable Bowel Syndrome, Comorbid Anxiety, and Depression: Mixed Methods Study

BACKGROUND

Irritable bowel syndrome (IBS) is a disorder of the gut-brain interaction that is associated with abdominal pain, altered bowel patterns, and reduced quality of life. Up to 50% of patients with IBS also report anxiety or depressive symptoms. Although effective self-management interventions exist for individuals with IBS, few have been effectively implemented, and most do not consider the unique needs of patients with comorbid IBS and anxiety or depression.

OBJECTIVE

This study aimed to determine the anticipated acceptability, appropriateness, feasibility, and usability of a comprehensive self-management intervention using an implementation science and human-centered design approach among individuals with comorbid IBS and anxiety or depression and health care providers.

METHODS

A convergent mixed methods design was used to elicit feedback on the comprehensive self-management intervention outline and content to identify refinement needs before testing. Patients with IBS and moderate to severe anxiety or depression and health care providers were purposefully sampled from primary care and gastroenterology settings. Participants completed semistructured interviews and surveys on anticipated acceptability, appropriateness, feasibility, and usability.

RESULTS

Patient participants (n=12) were on average 36.8 (SD 12.2) years of age, and 42% (5/12) were currently receiving psychological therapy. Health care providers (n=14) were from primary care (n=7) and gastroenterology (n=7) settings. The mean usability scores (out of 100) were 52.5 (SD 14.5) for patients and 45.6 (SD 11.6) for providers. For patients and providers, qualitative data expanded the quantitative findings for acceptability and appropriateness. Acceptability findings were the comprehensive nature of the intervention and discussion of the gut-brain interaction. For appropriateness, participants reported that the intervention provided structure, accountability, and support. Feasibility was confirmed for patients, but there was a divergence of findings between quantitative and qualitative measures for providers. Patients focused on intervention feasibility, while providers focused on implementation feasibility in the clinic. Identified usability issues to address before implementation included the intervention delivery format, length, and lack of integration into health care settings that, if not addressed, may limit the reach of the intervention.

CONCLUSIONS

Patients and health care providers found the intervention acceptable and appropriate. Several feasibility and usability issues were identified, including intervention delivery methods, length of intervention, and the best methods to implement in the clinic setting. The next steps are to refine the intervention to address the identified issues and test in a pilot study whether addressing usability issues leads to the anticipated improvements in implementation and uptake.

Implementation Science in School-Based, Universal-Level Intervention Research: A Scoping Review

PURPOSE

The aim of this study was to map the use of implementation science frameworks, models, and theories in intervention research targeting learning needs in the classroom.

METHOD

A scoping review was conducted. Electronic database and manual searches were conducted. Two reviewers independently completed screening, data extraction, and quality appraisal. Qualitative content analysis was undertaken using Nilsen's taxonomy and the domains of the Consolidated Framework for Implementation Research (CFIR). The data were further analyzed using the CFIR valence and strength rating scales.

RESULTS

Included papers (n = 22) used a diverse sample of implementation science frameworks and models. Most studies used determinant frameworks to guide data collection and analysis. Few studies used implementation science theory. Most studies were mixed methods (n = 11), published since 2019 (n = 20), and conducted in North America (n = 15). Over half of the interventions targeted social, emotional, and mental health (n = 13). A complex interplay of inner setting factors was identified as having a strong influence on implementation. Teachers' knowledge and beliefs, adaptability, and complexity of interventions were also identified as important considerations when conducting research in this context.

CONCLUSIONS

Early engagement with stakeholders in education is recommended when designing universal level speech, language, and communication interventions for use in the ordinary classroom. Adaptive, hybrid designs that test both implementation strategies and effectiveness of interventions may be warranted given the influence of inner setting factors on implementation outcomes.

"On an island by myself": implications for the inclusion of autistic students in self-contained classrooms in public elementary schools

Introduction

Autistic students have limited access to inclusive classes and activities in their schools. Principals and special education teachers who directly teach and administer programs for autistic elementary students can offer critical insight into factors, such as educators' attitudes, that may impact inclusive opportunities in schools. These attitudes may serve as barriers to or facilitators of promoting an inclusive school setting.

Methods

Semi-structured interviews with 26 elementary school principals and 26 special education teachers explored their experiences implementing evidence-based practices for autistic students (pivotal response training, discrete trial training, and visual schedules) in 26 self-contained classrooms in the United States. Autism-specific culture and inclusion emerged as a theme, which was analyzed for this paper.

Results

An inductive approach to thematic analysis revealed principals' and special education teachers' perspectives regarding the "autism-specific culture" in the school, including attitudes towards and inclusion of autistic students in self-contained classrooms in the broader school environment. Analysis of text related to "autism-specific culture" detailed aspects of inclusion, factors (i.e., barriers and facilitators) affecting inclusion, principals' and special education teachers' attitudes towards autistic students placed in self-contained classrooms, attitudes of other school staff towards teachers in self-contained classrooms, and recommendations to support an inclusive school environment for autistic students.

Discussion

Results suggest that valuing "equal" access to classes and activities for autistic students in self-contained classrooms may not be sufficient for promoting an inclusive school environment, Educators may benefit from targeted strategies to facilitate inclusion. Strategies range from supporting educators' attitudes and knowledge of autism to shifting physical aspects of the school environment (e.g., location of classrooms). Additional implications for supporting the true inclusion (i.e., inclusion that goes beyond physical inclusion) involves of autistic students in self-contained classrooms schools are discussed.

Behavioral Parent Training via Telehealth for Autistic Children: Further Exploration of Feasibility During the COVID-19 Pandemic

Objectives

Many autistic children exhibit challenging and disruptive behaviors that can present challenges for both children and their families by interfering with acquisition of adaptive skills and affecting family and peer relationships. Behavioral parent training (BPT) is an evidence-based approach to reducing autistic children's disruptive behavior, but many families face a number of barriers to accessing BPT, such as availability of BPT in their community, and transportation and scheduling challenges. Therefore, we sought to explore the feasibility and promise of effectiveness of adapting an established BPT program to a telehealth format during the COVID-19 pandemic.

Methods

A feasibility trial of BPT via telehealth was conducted with fourteen parents of autistic children.

Results

Parents and clinicians were able to implement BPT via telehealth with a high degree of fidelity, and parents rated both BPT and the telehealth format favorably. The program also showed promise of effectiveness in reducing autistic children's disruptive behavior, improving their adaptive skills, as well as reducing parents' stress, and improving parents' sense of parenting competence.

Conclusions

The findings replicate and extend findings from previous studies, further demonstrating the promise of telehealth as a viable alternative format for delivering BPT. We also explore implications for future research, including the opportunity for more thorough evaluation of the effectiveness of BPT via telehealth.

Usability Issues in Evidence-Based Psychosocial Interventions and Implementation Strategies: Cross-project Analysis

BACKGROUND

People often prefer evidence-based psychosocial interventions (EBPIs) for mental health care; however, these interventions frequently remain unavailable to people in nonspecialty or integrated settings, such as primary care and schools. Previous research has suggested that usability, a concept from human-centered design, could support an understanding of the barriers to and facilitators of the successful adoption of EBPIs and support the redesign of EBPIs and implementation strategies.

OBJECTIVE

This study aimed to identify and categorize usability issues in EBPIs and their implementation strategies.

METHODS

We adapted a usability issue analysis and reporting format from a human-centered design. A total of 13 projects supported by the National Institute of Mental Health-funded Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness Center at the University of Washington used this format to describe usability issues for EBPIs and implementation strategies with which they were working. Center researchers used iterative affinity diagramming and coding processes to identify usability issue categories. On the basis of these categories and the underlying issues, we propose heuristics for the design or redesign of EBPIs and implementation strategies.

RESULTS

The 13 projects reported a total of 90 usability issues, which we categorized into 12 categories, including complex and/or cognitively overwhelming, required time exceeding available time, incompatibility with interventionist preference or practice, incompatibility with existing workflow, insufficient customization to clients/recipients, intervention buy-in (value), interventionist buy-in (trust), overreliance on technology, requires unavailable infrastructure, inadequate scaffolding for client/recipient, inadequate training and scaffolding for interventionists, and lack of support for necessary communication. These issues range from minor inconveniences that affect a few interventionists or recipients to severe issues that prevent all interventionists or recipients in a setting from completing part or all of the intervention. We propose 12 corresponding heuristics to guide EBPIs and implementation strategy designers in preventing and addressing these usability issues.

CONCLUSIONS

Usability issues were prevalent in the studied EBPIs and implementation strategies. We recommend using the lens of usability evaluation to understand and address barriers to the effective use and reach of EBPIs and implementation strategies.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/14990.

Human-centered design methods to achieve preparation phase goals in the multiphase optimization strategy framework

Background

The public health impact of behavioral and biobehavioral interventions to prevent and treat mental health and substance use problems hinges on developing methods to strategically maximize their effectiveness, affordability, scalability, and efficiency.

Methods

The multiphase optimization strategy (MOST) is an innovative, principled framework that guides the development of multicomponent interventions. Each phase of MOST (Preparation, Optimization, Evaluation) has explicit goals and a range of appropriate research methods to achieve them. Methods for attaining Optimization and Evaluation phase goals are well-developed. However, methods used in the Preparation phase are often highly researcher-specific, and concrete ways to achieve Preparation phase goals are a priority area for further development.

Results

We propose that the discover, design, build, and test (DDBT) framework provides a theory-driven and methods-rich roadmap for achieving the goals of the Preparation phase of MOST, including specifying the conceptual model, identifying and testing candidate intervention components, and defining the optimization objective. The DDBT framework capitalizes on strategies from the field of human-centered design and implementation science to drive its data collection methods.

Conclusions

MOST and DDBT share many conceptual features, including an explicit focus on implementation determinants, being iterative and flexible, and designing interventions for the greatest public health impact. The proposed synthesized DDBT/MOST approach integrates DDBT into the Preparation phase of MOST thereby providing a framework for rigorous and efficient intervention development research to bolster the success of intervention optimization.

Plain Language Summary

1. What is already known about the topic? Optimizing behavioral interventions to balance effectiveness with affordability, scalability, and efficiency requires a significant investment in intervention development.2. What does this paper add? This paper provides a structured approach to integrating human-centered design principles into the Preparation phase of the multiphase optimization strategy (MOST).3. What are the implications for practice, research, or policy? The proposed synthesized model provides a framework for rigorous and efficient intervention development research in the Preparation phase of MOST that will ensure the success of intervention optimization and contribute to improving public health impact of mental health and substance use interventions.