Extensions of open science for applied behavior analysis: Preregistration for single-case experimental designs

Open science practices are designed to enhance the utility, integrity, and credibility of scientific research. This article highlights how preregistration in open science practice can be leveraged to enhance the rigor and transparency of single-case experimental designs within an applied behavior analysis framework. We provide an overview of the benefits of preregistration including increased transparency, reduced risk of researcher bias, and improved replicability, and we review the specific contexts under which these practices most benefit the proposed framework. We discuss potential concerns with and unique considerations for preregistering experiments that use single-case designs, with practical guidance for researchers who are seeking to preregister their studies. We present a checklist as a tool for researchers in applied behavior analysis to use for preregistration and provide recommendations for our field to strengthen the contingencies for open science practices that include preregistration.

Efficacy of competing stimulus assessments: A summary of 35 consecutively encountered cases

Competing stimulus assessments (CSAs) are designed to identify stimuli that reduce challenging behavior through competition with its maintaining reinforcers. Recently, Haddock and Hagopian (2020) found that over 92% of CSAs described in published studies identified at least one high-competition stimulus (i.e., a stimulus correlated with at least an 80% reduction in challenging behavior). The current study describes the outcomes of CSAs in a retrospective consecutive controlled case series study of 35 cases (individuals) admitted to an inpatient setting. Findings on the limited relation between the level of stimulus engagement and reductions in challenging behavior were replicated; however, the efficacy of CSAs was lower (only 47% of CSAs were successful in identifying one or more high-competition stimuli). Discrepant findings across studies on the efficacy of CSAs are discussed in terms of differences in the sample participants and how outcomes are reported, which vary depending on the study's research questions.

"Unvoluntary" Movement Disorders: Distinguishing between Tics, Akathisia, Restless Legs, and Stereotypies

Tics, stereotypies, akathisia, and restless legs fall at different places on the spectrum of discrete, unwanted and potentially disabling motor routines. Unlike tremor, chorea, myoclonus, or dystonia, this subgroup of abnormal movements is characterized by the subject's variable ability to inhibit or release undesired motor patterns on demand. Though it may be sometimes clinically challenging, it is crucial to distinguish these "unvoluntary" motor behaviors because secondary causes and management approaches differ substantially. To this end, physicians must consider the degree of repetitiveness of the movements, the existence of volitional control, and the association with sensory symptoms, or cognitive-ideational antecedent. This review aims to summarize the current existing knowledge on phenomenology, diagnosis, and treatment of tics, stereotypies, akathisia, and restless leg syndrome.

Agreement between visual inspection and objective analysis methods: A replication and extension

Behavior analysts typically rely on visual inspection of single‐case experimental designs to make treatment decisions. However, visual inspection is subjective, which has led to the development of supplemental objective methods such as the conservative dual‐criteria method. To replicate and extend a study conducted by Wolfe et al. (2018) on the topic, we examined agreement between the visual inspection of five raters, the conservative dual‐criteria method, and a machine‐learning algorithm (i.e., the support vector classifier) on 198 AB graphs extracted from clinical data. The results indicated that average agreement between the 3 methods was generally consistent. Mean interrater agreement was 84%, whereas raters agreed with the conservative dual‐criteria method and the support vector classifier on 84% and 85% of graphs, respectively. Our results indicate that both objective methods produce results consistent with visual inspection, which may support their future use.

Structured visual analysis of single-case experimental design data: Developments and technological advancements

Visual analysis is the primary method used to interpret single-case experimental design (SCED) data in applied behavior analysis. Research shows that agreement between visual analysts can be suboptimal at times. To address the inconsistent interpretations of SCED data, recent structured visual-analysis technological advancements have been developed. To assess the extent to which structured visual analysis is used to guide or supplement applied behavior analysts' interpretation of SCED graphs, a systematic review between the years of 2015 to 2020 in the Journal of Applied Behavior Analysis was conducted. Findings showed that despite recent efforts to develop structured visual-analysis tools and criteria, these methods are rarely used to analyze SCED data. An overview of structured visual-analysis tools is shared, their utility is delineated, common characteristics are brought to light, and future directions for both research and their clinical use are highlighted.

Contingency-based delay to reinforcement following functional communication training for autistic individuals: A multilevel meta-analysis

LAY ABSTRACT

Functional communication training, an intervention for challenging behavior rooted in principles of applied behavior analysis, has copious empirical support dating back to the mid-1980s for autistic individuals. Recently, there has been a concerted effort to thin reinforcement delivery during functional communication training using contingency-based delays that, in turn, are designed to enhance practicality and feasibility while not compromising efficacy. In this synthesis, we meta-analyzed the literature base with the goal of investigating both combined and across type effectiveness of contingency-based delays. We also aimed to investigate moderating variables that might impact intervention outcomes. Findings showed that contingency-based delays were effective for individuals with an autism spectrum disorder diagnosis and most effective when the delay incorporated some form of positive reinforcement. In addition, differential reinforcement of alternative-based delays was overall more effective when compared to differential reinforcement of other behavior-based delays. Noteworthy moderating variables found to impact contingency-based delay efficacy included the intervention dosage and the topography of behavior. We discuss these findings and highlight directions where additional empirical research is warranted to improve our understanding about contingency-based delays for individuals diagnosed with autism spectrum disorder.

Machine learning to analyze single-case graphs: A comparison to visual inspection

Behavior analysts commonly use visual inspection to analyze single‐case graphs, but studies on its reliability have produced mixed results. To examine this issue, we compared the Type I error rate and power of visual inspection with a novel approach—machine learning. Five expert visual raters analyzed 1,024 simulated AB graphs, which differed on number of points per phase, autocorrelation, trend, variability, and effect size. The ratings were compared to those obtained by the conservative dual‐criteria method and two models derived from machine learning. On average, visual raters agreed with each other on only 75% of graphs. In contrast, both models derived from machine learning showed the best balance between Type I error rate and power while producing more consistent results across different graph characteristics. The results suggest that machine learning may support researchers and practitioners in making fewer errors when analyzing single‐case graphs, but replications remain necessary.

Meta-analyses and effect sizes in applied behavior analysis: A review and discussion

For more than four decades, researchers have used meta-analyses to synthesize data from multiple experimental studies often to draw conclusions that are not supported by individual studies. More recently, single-case experimental design (SCED) researchers have adopted meta-analysis techniques to answer research questions with data gleaned from SCED experiments. Meta-analyses enable researchers to answer questions regarding intervention efficacy, generality, and condition boundaries. Here we discuss meta-analysis techniques, the rationale for their adaptation with SCED studies, and current indices used to quantify the effect of SCED data in applied behavior analysis.

A Decade Review of Two Potential Analysis Altering Variables in Graph Construction

In single-case research designs (SCDs) to determine a functional relation a time-series graph is constructed. Preliminary evidence suggest the approach used to scale the vertical axis and the data points per x- to y-axis ratio (DPPXYR) impact visual analysts' decisions. We conducted a systematic review to evaluate time-series graphs published in the last decade in four premier journals in the field of autism. We included 348 articles including 2675 graphs. We identified large variation across and within types of SCDs when evaluating the lengths of the vertical and horizontal axis using the y:x ratio and the DPPXYR, with few adhering to current recommendations. A majority of graphs used an appropriate method to scale the vertical axis that would not increase Type I error rates.