A Primer on Single-Case Research Designs: Contemporary Use and Analysis

How to assess and take into account trend in single-case experimental design data

One of the data features that are expected to be assessed when analyzing single-case experimental designs (SCED) data is trend. The current text deals with four different questions that applied researchers can ask themselves when assessing trend and especially when dealing with improving baseline trend: (a) What options exist for assessing the presence of trend?; (b) Once assessed, what criterion can be followed for deciding whether it is necessary to control for baseline trend?; (c) What strategy can be followed for controlling for baseline trend?; and (d) How to proceed in case there is baseline trend only in some A-B comparisons? Several options are reviewed for each of these questions in the context of real data, and tentative recommendations are provided. A new user-friendly website is developed to implement the options for fitting a trend line and a criterion for selecting a specific technique for that purpose. Trend-related and more general data analytical recommendations are provided for applied researchers.Trial registration: ClinicalTrials.gov identifier: NCT04560777.

Type I error rates and power of two randomization test procedures for the changing criterion design

Single-case experimental design (SCED) data can be analyzed following different approaches. One of the first historically proposed options is randomizations tests, benefiting from the inclusion of randomization in the design: a desirable methodological feature. Randomization tests have become more feasible with the availability of computational resources, and such tests have been proposed for all major types of SCEDs: multiple-baseline, reversal/withdrawal, alternating treatments, and changing criterion designs. The focus of the current text is on the last of these, given that they have not been the subject of any previous simulation study. Specifically, we estimate type I error rates and statistical power for two different randomization procedures applicable to changing criterion designs: the phase change moment randomization and the blocked alternating criterion randomization. We include different series lengths, number of phases, levels of autocorrelation, and random variability. The results suggest that type I error rates are generally controlled and that sufficient power can be achieved with as few as 28-30 measurements for independent data, although more measurements are needed in case of positive autocorrelation. The presence of a reversal to a previous criterion level is beneficial. R code is provided for carrying out randomization tests following the two randomization procedures.

Testing delayed, gradual, and temporary treatment effects in randomized single-case experiments: A general response function framework

Randomization tests represent a class of significance tests to assess the statistical significance of treatment effects in randomized single-case experiments. Most applications of single-case randomization tests concern simple treatment effects: immediate, abrupt, and permanent changes in the level of the outcome variable. However, researchers are confronted with delayed, gradual, and temporary treatment effects; in general, with "response functions" that are markedly different from single-step functions. We here introduce a general framework that allows specifying a test statistic for a randomization test based on predicted response functions that is sensitive to a wide variety of data patterns beyond immediate and sustained changes in level: different latencies (degrees of delay) of effect, abrupt versus gradual effects, and different durations of the effect (permanent or temporary). There may be reasonable expectations regarding the kind of effect (abrupt or gradual), entailing a different focal data feature (e.g., level or slope). However, the exact amount of latency and the exact duration of a temporary effect may not be known a priori, justifying an exploratory approach studying the effect of specifying different latencies or delayed effects and different durations for temporary effects. We provide illustrations of the proposal with real data, and we present a user-friendly freely available web application implementing it.

Approaches for Treating Multiply Controlled Problem Behavior

A portion of problem behavior is maintained by multiple reinforcement contingencies. Treating multiply controlled problem behavior may be more complex than treating problem behavior maintained by a single contingency. Several approaches for addressing multiply controlled problem behavior have been described in the literature. The purpose of this review is to provide practitioners with an overview of function-based approaches for addressing multiply controlled problem behavior. In particular, we present guidelines for functional analysis and treatment. We also describe strengths and limitations of published treatment approaches and discuss strategies for mitigating these limitations. Finally, we describe areas for future research.

Does the choice of a linear trend-assessment technique matter in the context of single-case data?

Trend is one of the data aspects that is an object of assessment in the context of single-case experimental designs. This assessment can be performed both visually and quantitatively. Given that trend, just like other relevant data features such as level, immediacy, or overlap does not have a single operative definition, a comparison among the existing alternatives is necessary. Previous studies have included illustrations of differences between trend-line fitting techniques using real data. In the current study, I carry out a simulation to study the degree to which different trend-line fitting techniques lead to different degrees of bias, mean square error, and statistical power for a variety of quantifications that entail trend lines. The simulation involves generating both continuous and count data, for several phase lengths, degrees of autocorrelation, and effect sizes (change in level and change in slope). The results suggest that, in general, ordinary least squares estimation performs well in terms of relative bias and mean square error. Especially, a quantification of slope change is associated with better statistical results than quantifying an average difference between conditions on the basis of a projected baseline trend. In contrast, the performance of the split-middle (bisplit) technique is less than optimal.

A Visual Aid and Objective Rule Encompassing the Data Features of Visual Analysis

Visual analysis of single-case research is commonly described as a gold standard, but it is often unreliable. Thus, an objective tool for applying visual analysis is necessary, as an alternative to the Conservative Dual Criterion, which presents some drawbacks. The proposed free web-based tool enables assessing change in trend and level between two adjacent phases, while taking data variability into account. The application of the tool results in (a) a dichotomous decision regarding the presence or absence of an immediate effect, a progressive or delayed effect, or an overall effect and (b) a quantification of overlap. The proposal is evaluated by applying it to both real and simulated data, obtaining favorable results. The visual aid and the objective rules are expected to make visual analysis more consistent, but they are not intended as a substitute for the analysts' judgment, as a formal test of statistical significance, or as a tool for assessing social validity.

Single-Case Data Analysis: A Practitioner Guide for Accurate and Reliable Decisions

Practitioners frequently use single-case data for decision-making related to behavioral programming and progress monitoring. Visual analysis is an important and primary tool for reporting results of graphed single-case data because it provides immediate, contextualized information. Criticisms exist concerning the objectivity and reliability of the visual analysis process. When practitioners are equipped with knowledge about single-case designs, including threats and safeguards to internal validity, they can make technically accurate conclusions and reliable data-based decisions with relative ease. This paper summarizes single-case experimental design and considerations for professionals to improve the accuracy and reliability of judgments made from single-case data. This paper can also help practitioners to appropriately incorporate single-case research design applications in their practice.

A Randomized Controlled Trial for Audiovisual Multisensory Perception in Autistic Youth

Differences in audiovisual integration are commonly observed in autism. Temporal binding windows (TBWs) of audiovisual speech can be trained (i.e., narrowed) in non-autistic adults; this study evaluated a computer-based perceptual training in autistic youth and assessed whether treatment outcomes varied according to individual characteristics. Thirty autistic youth aged 8-21 were randomly assigned to a brief perceptual training (n = 15) or a control condition (n = 15). At post-test, the perceptual training group did not differ, on average, on TBWs for trained and untrained stimuli and perception of the McGurk illusion compared to the control group. The training benefited youth with higher language and nonverbal IQ scores; the training caused widened TBWs in youth with co-occurring cognitive and language impairments.

Waiting for baseline stability in single-case designs: Is it worth the time and effort?

Researchers and practitioners often use single-case designs (SCDs), or n-of-1 trials, to develop and validate novel treatments. Standards and guidelines have been published to provide guidance as to how to implement SCDs, but many of their recommendations are not derived from the research literature. For example, one of these recommendations suggests that researchers and practitioners should wait for baseline stability prior to introducing an independent variable. However, this recommendation is not strongly supported by empirical evidence. To address this issue, we used Monte Carlo simulations to generate graphs with fixed, response-guided, and random baseline lengths while manipulating trend and variability. Then, our analyses compared the type I error rate and power produced by two methods of analysis: the conservative dual-criteria method (a structured visual aid) and a support vector classifier (a model derived from machine learning). The conservative dual-criteria method produced fewer errors when using response-guided decision-making (i.e., waiting for stability) and random baseline lengths. In contrast, waiting for stability did not reduce decision-making errors with the support vector classifier. Our findings question the necessity of waiting for baseline stability when using SCDs with machine learning, but the study must be replicated with other designs and graph parameters that change over time to support our results.

Speech-Language Pathology Interventions for Children With Executive Function Deficits: A Systematic Literature Review

PURPOSE

The purpose of this synthesis was to systematically review the research and guidance for school-based speech-language pathologists (SLPs) who provide intervention to children with developmental executive function (EF) deficits, particularly those children with co-occurring developmental language disorder (DLD).

METHOD

We conducted a structured search of four major electronic databases, as well as a manual review of references and journals, which yielded 4,571 nonduplicate articles. We screened first titles and abstracts and then full texts to identify peer-reviewed articles, dissertations, and theses containing research or guidance for SLPs' interventions for children with co-occurring DLD and EF deficits; this process yielded 27 articles for analysis. We categorized these studies by type of publication and synthesized their contents to assess the evidence base for EF interventions in children with DLD and to evaluate the guidance for SLP-implemented direct and indirect interventions.

RESULTS

A small body of research explores the efficacy of SLPs' intervention for children with co-occurring DLD and EF deficits, generally finding modest but inconsistent effects of cognitive interventions and strategy training to improve language outcomes. Meanwhile, nonempirical articles (e.g., tutorials) offer guidance to SLPs to support students with EF deficits through direct and indirect services.

CONCLUSIONS

A growing body of literature equips SLPs with the principles and strategies of EF intervention. Many of these articles are sourced from literature about children with EF deficits or attention-deficit/hyperactivity disorder, but few empirical studies measure the efficacy of these interventions for children with co-occurring DLD.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21401901.

Defining and assessing immediacy in single-case experimental designs

Immediacy is one of six data aspects (alongside level, trend, variability, overlap, and consistency) that has to be accounted for when visually analyzing single-case data. Given that it is one of the aspects that has received considerably less attention than other data aspects, the current text offers a review of the proposed conceptual definitions of immediacy (i.e., what it refers to) and also of the suggested operational definitions (i.e., how exactly is it assessed and/or quantified). Provided that a variety of conceptual and operational definitions is identified, we propose following a sensitivity analysis using a randomization test for assessing immediate effects in single-case experimental designs, by identifying when changes were most clear. In such a sensitivity analysis, the immediate effects are tested for multiple possible intervention points and for different possible operational definitions. Robust immediate effects can be detected if the results for the different operational definitions converge.

A concise review of recent advancements in the graphical training of behavior analysts

This Concise Review is an appraisal of contemporary research on teaching single-case experimental design (SCED) graphical creation published between 2017-2021. Recent work on SCED graphical creation is summarized and areas for future research are highlighted.

A proposal for the assessment of replication of effects in single-case experimental designs

In science in general and in the context of single‐case experimental designs, replication of the effects of the intervention within and/or across participants or experiments is crucial for establishing causality and for assessing the generality of the intervention effect. Specific developments and proposals for assessing whether an effect has been replicated or not (or to what extent) are scarce, in the general context of behavioral sciences, and practically null in the single‐case experimental designs context. We propose an extension of the modified Brinley plot for assessing how many of the effects replicate. To make this assessment possible, a definition of replication is suggested, on the basis of expert judgment, rather than on statistical criteria. The definition of replication and its graphical representation are justified, presenting their strengths and limitations, and illustrated with real data. A user‐friendly software is made available for obtaining automatically the graphical representation.

A practitioner's guide to conducting and analysing embedded randomized single-case experimental designs

Single-case experimental designs (SCEDs) are a class of experimental designs suited for answering research questions at an individual level. The main designs available in SCED research are phase designs, multiple baseline designs, alternation designs, and changing criterion designs. Embedded designs, also referred to as combination or hybrid designs, consist of one of these basic designs forms embedded in another design (e.g., a changing criterion design embedded in a multiple baseline design). Systematic reviews of SCEDs have repeatedly indicated that embedded designs are frequently used in applied SCED research. In spite of their popularity, specific recommendations on the conduct and analysis of embedded SCED designs are lacking to date. The purpose of the present article is therefore to provide guidance to applied researchers wishing to conduct embedded SCED designs in terms of design options, design requirements, randomization, and data analysis.

Effects of Acceptance and Commitment Training Plus Behavior Parent Training on Parental Implementation of Autism Treatment

The objective of this study was to examine the effects of acceptance and commitment training (ACT) plus behavior parent training (BPT), when delivered via telehealth, on parental implementation of behavioral strategies, experiential avoidance (EA), and stress. The study also examined the subsequent effects on the parents’ autistic children’s behaviors. A multiple baseline design was implemented across four parent-child dyads who participated in the online training. The findings showed that ACT+BPT resulted in parental implementation reaching and maintaining high levels. The training also decreased EA and stress in three parents. Moreover, the parents’ ratings of their children’s challenging behaviors decreased. However, such a trend was not as clearly depicted by direct measures of the children’s behaviors. A social validity interview revealed parents found ACT beneficial in assisting them to learn and use the BPT strategies. Implications and suggestions for future research are discussed.

Building Prealgebra Fluency Through a Self-Managed Practice Intervention: Order of Operations

Behavioral fluency refers to a combination of accuracy and speed that enables students to function proficiently in the learning environment. The present study investigated the effects of a self-managed frequency-building intervention on the behavioral fluency of a critical prealgebra skill in four 6th-grade students. The intervention involved students having access to the PEMDAS (parentheses, exponents, multiplication, division, addition, and subtraction) mnemonic during frequency building. Using an alternating-treatments design, the first experimental condition presented the intervention as three 1-min practice trials with 30 s of feedback delivered immediately after each frequency-building trial ended. The second condition offered one 3-min practice trial with 90 s of feedback once the trial ended. A baseline condition (no practice) had the students engage in a 1-min timed trial with no feedback. The alternating-treatments design demonstrated that three of the four students produced a superior performance within the two intervention conditions when compared to baseline. However, the results did not conclusively show that one frequency-building intervention was superior to the other.

Precision Teaching and Behavior Dynamics

Precision teaching (PT) comprises a sophisticated measurement and decision-making system aimed at helping its users foster superior outcomes for the clients or students they serve. The longevity of PT has led to many discoveries surrounding measurement, performance, learning, and behavior change. The contributions to knowledge derive from a range of studies. Some of the research uses single-case experimental designs, whereas the majority employs an approach called behavior dynamics. The use of behavior dynamics distinguishes PT from behavior analysis. Behavior dynamics does not seek to uncover functional relations. Nevertheless, behavior dynamics represents a sound approach to conduct research, generate reliable information, and engender knowledge.

Hybrid Telepractice Delivery of Enhanced Milieu Teaching: Effects on Caregiver Implementation and Child Communication

Purpose This study investigated the effects of enhanced milieu teaching (EMT) on caregiver implementation of language support strategies and child communication skills using a hybrid telepractice service delivery model. Method Four caregivers and children with language delays aged 18-27 months participated in a multiple baseline across behaviors single-case research design. The therapist provided EMT to each caregiver-child dyad using a hybrid telepractice service delivery model with approximately 40% in-person and 60% telepractice sessions. Caregivers were taught to use five EMT language support strategies: matched turns, target talk, expansions, time delays, and milieu teaching episodes. Caregiver instruction followed the Teach-Model-Coach-Review approach. Caregiver outcomes were two measures of EMT implementation fidelity, accuracy, and frequency of EMT strategy use. Accuracy was measured by the percentage of spoken turns in which caregivers used each strategy correctly. Frequency was measured by the number of spoken turns in which caregivers used each strategy correctly. Child outcomes were number of communication acts, weighted count of communication acts, and number of different words. Results There was a functional relation between the intervention and the accuracy of EMT strategy use for all four dyads, and the frequency of strategy use for three dyads. Caregiver use of EMT strategies maintained for 6 weeks post-intervention. After caregivers learned EMT strategies, gradual increases in the number of communication acts, weighted count of communication acts, and number of different words occurred for three children. Conclusion Results demonstrate the preliminary efficacy of using a hybrid telepractice service delivery model to teach caregivers EMT language support strategies. Supplemental Material https://doi.org/10.23641/asha.14977605.

Optimal number of baseline sessions before changing phases within single-case experimental designs

Recommendations vary considerably for the minimum or optimal number of baseline sessions to conduct within single-case experimental design clinical analyses or research studies. We examined the optimal number of baseline sessions that produced minimal bias. First, we examined the relation between the number of baseline sessions and the degree of bias in calculating estimates of treatment effect size. As the number of baseline sessions increased, the bias in effect size estimates decreased, r = -0.36, p < 0.001. s, we examined what would be the minimum number of baseline sessions associated with varying levels of bias. Bias of approximately ten percent was associated with four to five baseline sessions. Bias of about five percent was associated with six to seven baseline sessions. Third, we examined the relation between standard deviation and varying levels of bias. As the number of baseline sessions increases, the standard deviation for the phase decreased, r = -0.89, p < 0.001. Fourth, we examined what value of standard deviation in the baseline phase was associated with equal to or more than five versus ten percent bias. When considering five or ten percent bias, the optimal level of standard deviation was 0.59 or less.

A brief acceptance and commitment intervention for work-related stress and burnout amongst frontline homelessness staff: A single case experimental design series

PURPOSE

Recent intervention research for burnout amongst those working in health and social care contexts has found acceptance and commitment therapy (ACT) interventions to be of use but has provided less clarity on the role of psychological flexibility (a key ACT construct). This study further evaluated the usefulness of ACT for burnout and work-engagement and assessed the role of psychological flexibility in contributing to therapeutic change.

PROCEDURE

A nonconcurrent multiple-baseline across-participants single-case experimental design was used. Four participants were recruited from a homelessness organization in the East Midlands, England. The ACT-intervention was split into three modules to reflect the three aspects of the ACT triflex, and the sequence of delivery was randomized for each participant in order to test the relationship between these aspects.

FINDINGS

Support was found for the ACT intervention reducing exhaustion and increasing work-engagement. Psychological Flexibility increased in all participants and was temporally related to increases in other outcome variables in some instances. Delivery of the intervention focussed on any given aspect of the ACT triflex could increase different domains of psychological flexibility.

IMPLICATIONS

This study adds to the growing body of research in favour of ACT interventions for burnout and adds to the understanding of psychological flexibility as a mediating variable.

Assessing Consistency in Single-Case Data Features Using Modified Brinley Plots

The current text deals with the assessment of consistency of data features from experimentally similar phases and consistency of effects in single-case experimental designs. Although consistency is frequently mentioned as a critical feature, few quantifications have been proposed so far: namely, under the acronyms CONDAP (consistency of data patterns in similar phases) and CONEFF (consistency of effects). Whereas CONDAP allows assessing the consistency of data patterns, the proposals made here focus on the consistency of data features such as level, trend, and variability, as represented by summary measures (mean, ordinary least squares slope, and standard deviation, respectively). The assessment of consistency of effect is also made in terms of these three data features, while also including the study of the consistency of an immediate effect (if expected). The summary measures are represented as points on a modified Brinley plot and their similarity is assessed via quantifications of distance. Both absolute and relative measures of consistency are proposed: the former expressed in the same measurement units as the outcome variable and the latter as a percentage. Illustrations with real data sets (multiple baseline, ABAB, and alternating treatments designs) show the wide applicability of the proposals. We developed a user-friendly website to offer both the graphical representations and the quantifications.

A comparison of single-case evaluation tools applied to functional communication training with augmentative and alternative communication supports for students with developmental disabilities

BACKGROUND

Students with developmental disabilities frequently present with both limited vocal speech and challenging behavior. Functional communication training (FCT) with augmentative and alternative communication (AAC) supports, is a commonly recommended intervention to reduce challenging behavior for these students, while also increasing appropriate communication.

AIMS

Current research on this topic has not applied multiple evaluation tools, despite the recent suggestion to do so. Further, there are limited studies in the field of special education that have (a) applied multiple evaluation tools and (b) compared the results of the tools.

METHOD

In the current review, we applied three evaluation tools to intervention studies examining the use of FCT with AAC supports in school-based settings to determine the current level of scientific support for this intervention. We identified 38 studies, which contained 59 single-case designs (SCDs). Next, we compared the methodological rigor and/or quality, outcome scores, and Evidence-Based Practice (EBP) ratings provided by the three evaluation tools.

RESULTS AND CONCLUSION

Our results yielded inconsistent methodological rigor and/or quality, participant outcome measures, and EBP classifications between the evaluation tools. No two evaluation tools completely aligned. Limitations and future research are discussed.

What's in a Click? The Efficacy of Conditioned Reinforcement in Applied Animal Training: A Systematic Review and Meta-Analysis

Simple Summary

Conditioned reinforcement, for example, clicker training, has become increasingly popular in recent decades. Hence, questions about the effectiveness of the conditioned reinforcer have become prominent in the animal training arena. This article summarizes the scientific literature on conditioned reinforcement in applied animal training settings (e.g., homes). It was found that dogs and horses were the most frequently studied animals. Clickers and food were the most often used training stimuli. Effect size analysis found a medium effect of clicker training. The literature reviewed here shows that conditioned reinforcement is an effective approach to change animal behavior; however, sizable information potentially related to its effectiveness was not clearly reported in the studies (e.g., food preferences). Although this review fills in a gap in the literature, it also points to the need for more research to further the understanding of conditioned reinforcement phenomena.

Abstract

A conditioned reinforcer is a stimulus that acquired its effectiveness to increase and maintain a target behavior on the basis of the individual’s history—e.g., pairings with other reinforcers. This systematic review synthesized findings on conditioned reinforcement in the applied animal training field. Thirty-four studies were included in the review and six studies were eligible for a meta-analysis on the effectiveness of behavioral interventions that implemented conditioned reinforcement (e.g., clicks, spoken word, or whistles paired with food). The majority of studies investigated conditioned reinforcement with dogs (47%, n = 16) and horses (30%, n = 10) implementing click–food pairings. All other species (cats, cattle, fish, goats, and monkeys) were equally distributed across types of conditioned (e.g., clicker or spoken word) and unconditioned reinforcers (e.g., food, water, or tactile). A meta-analysis on the effectiveness of conditioned reinforcement in behavioral interventions found a medium summary effect size (Tau-U 0.77; CI_95% = [0.53, 0.89]), when comparing baseline, where no training was done, and treatment levels. Moderators of conditioned reinforcement effectiveness were species (e.g., horses) and research design (e.g., multiple-baseline designs). The small number of intervention-focused studies available limits the present findings and highlights the need for more systematic research into the effectiveness of conditioned reinforcement across species.

Assessing consistency of effects when applying multilevel models to single-case data

In the context of single-case experimental designs, replication is crucial. On the one hand, the replication of the basic effect within a study is necessary for demonstrating experimental control. On the other hand, replication across studies is required for establishing the generality of the intervention effect. Moreover, the "replicability crisis" presents a more general context further emphasizing the need for assessing consistency in replications. In the current text, we focus on replication of effects within a study, and we specifically discuss the consistency of effects. Our proposal for assessing the consistency of effects refers to one of the promising data analytical techniques, multilevel models, also known as hierarchical linear models or mixed effects models. One option is to check, for each case in a multiple-baseline design, whether the confidence interval for the individual treatment effect excludes zero. This is relevant for assessing whether the effect is replicated as being non-null. However, we consider that it is more relevant and informative to assess, for each case, whether the confidence interval for the random effects includes zero (i.e., whether the fixed effect estimate is a plausible value for each individual effect). This is relevant for assessing whether the effect is consistent in size, with the additional requirement that the fixed effect itself is different from zero. The proposal for assessing consistency is illustrated with real data and is implemented in free user-friendly software.

Using Principles from Applied Behaviour Analysis to Address an Undesired Behaviour: Functional Analysis and Treatment of Jumping Up in Companion Dogs

The aim of this study was to investigate the feasibility and effectiveness of procedures successfully used in human related applied behaviour analysis practices to the field of clinical animal behaviour. Experiment 1 involved functional analyses to identify the reinforcement contingencies maintaining jumping up behaviour in five dogs. Experiment 2 comprised teaching dog owners a noncontingent reinforcement intervention (i.e., time-based reinforcement) via behavioural skills training. Single-case experimental methods were implemented in both experiments. The results of Experiment 1 showed that access to a tangible (dogs D01, D02, D03, and D04) and owner attention (dog D05) were reliably maintaining the jumping up behaviour. Experiment 2 demonstrated that noncontingent reinforcement effectively reduced jumping in three out of four dogs (Tau -0.59, CI 90% [-1-0.15], p = 0.026, Tau -1, CI 90% [-1--0.55], p = 0.0003, and Tau -0.32, CI 90% [-0.76-0.11], p = 0.22 for dyads D01, D02, and D05, respectively), and that behavioural skills training was successful in teaching owners to perform a dog training intervention with high fidelity. Although the results are promising, more canine-related research into functional analysis and noncontingent reinforcement, as well as implementation of behavioural skills training with animal caregivers, is needed.