Replicability and randomization test logic in behavior analysis

Testing delayed, gradual, and temporary treatment effects in randomized single-case experiments: A general response function framework

Randomization tests represent a class of significance tests to assess the statistical significance of treatment effects in randomized single-case experiments. Most applications of single-case randomization tests concern simple treatment effects: immediate, abrupt, and permanent changes in the level of the outcome variable. However, researchers are confronted with delayed, gradual, and temporary treatment effects; in general, with "response functions" that are markedly different from single-step functions. We here introduce a general framework that allows specifying a test statistic for a randomization test based on predicted response functions that is sensitive to a wide variety of data patterns beyond immediate and sustained changes in level: different latencies (degrees of delay) of effect, abrupt versus gradual effects, and different durations of the effect (permanent or temporary). There may be reasonable expectations regarding the kind of effect (abrupt or gradual), entailing a different focal data feature (e.g., level or slope). However, the exact amount of latency and the exact duration of a temporary effect may not be known a priori, justifying an exploratory approach studying the effect of specifying different latencies or delayed effects and different durations for temporary effects. We provide illustrations of the proposal with real data, and we present a user-friendly freely available web application implementing it.

From data through discount rates to the area under the curve

The rate of discounting future goods is a crucial factor in intertemporal trade-offs, upon which depends not only individual well-being but also that of our planet: How much privation now for a temperate future for our grandchildren? What is the best way to measure how the value of future goods decreases with its delay? The most accurate discount functions involve several covarying parameters, making interpretation equivocal. A universal and robust measure is the area under the discount curve, the AuC. The AuC of a hyperbolic discount function is a logarithmic function of the discount rate, k. The same integral also approximates the area under a hyperboloid function. A simple technique converts each datum into estimates of the discount rate, eliminating rogue data points in the process. These trimmed estimates are converted into areas and tested against data, where they succeed at predicting the AuC and its relation to log(k).

The effects of response disequilibrium on social media use: A laboratory analogue

Response disequilibrium is the perturbation of unconstrained behavior with a contingency. For example, the imposition of advertisements before or after viewing TikTok videos. The purpose of this laboratory analogue was to determine the effects of two such response disequilibrium conditions: (1) which required participants to view 5 s increments of advertisements to access 2 s increments of TikTok videos and (2) which required participants to view 5 s increments of TikTok videos to access 15 s increments of advertisements. The disequilibrium condition in (1) is called a response deficit due to the restricted access to TikTok videos relative to baseline while (2) is called a response excess due to the overabundance of advertisements relative to baseline. Additionally, participants had access to a third activity that was freely available throughout. Participants could browse images of TikTok video thumbnails while in deficit and excess. As predicted, participants increased their viewing of advertisements when TikTok was in deficit and decreased their viewing of TikTok when advertisements were in excess. Furthermore, some participants substituted TikTok with browsing during the excess of advertisements. This investigation has implications for a behavioral economic analysis of social media use and the contingency management of time spent on these platforms.

Defining and assessing immediacy in single-case experimental designs

Immediacy is one of six data aspects (alongside level, trend, variability, overlap, and consistency) that has to be accounted for when visually analyzing single-case data. Given that it is one of the aspects that has received considerably less attention than other data aspects, the current text offers a review of the proposed conceptual definitions of immediacy (i.e., what it refers to) and also of the suggested operational definitions (i.e., how exactly is it assessed and/or quantified). Provided that a variety of conceptual and operational definitions is identified, we propose following a sensitivity analysis using a randomization test for assessing immediate effects in single-case experimental designs, by identifying when changes were most clear. In such a sensitivity analysis, the immediate effects are tested for multiple possible intervention points and for different possible operational definitions. Robust immediate effects can be detected if the results for the different operational definitions converge.

Meta-Analytic Methods to Detect Publication Bias in Behavior Science Research

Publication bias is an issue of great concern across a range of scientific fields. Although less documented in the behavior science fields, there is a need to explore viable methods for evaluating publication bias, in particular for studies based on single-case experimental design logic. Although publication bias is often detected by examining differences between meta-analytic effect sizes for published and grey studies, difficulties identifying the extent of grey studies within a particular research corpus present several challenges. We describe in this article several meta-analytic techniques for examining publication bias when published and grey literature are available as well as alternative meta-analytic techniques when grey literature is inaccessible. Although the majority of these methods have primarily been applied to meta-analyses of group design studies, our aim is to provide preliminary guidance for behavior scientists who might use or adapt these techniques for evaluating publication bias. We provide sample data sets and R scripts to follow along with the statistical analysis in hope that an increased understanding of publication bias and respective techniques will help researchers understand the extent to which it is a problem in behavior science research.

A practitioner's guide to conducting and analysing embedded randomized single-case experimental designs

Single-case experimental designs (SCEDs) are a class of experimental designs suited for answering research questions at an individual level. The main designs available in SCED research are phase designs, multiple baseline designs, alternation designs, and changing criterion designs. Embedded designs, also referred to as combination or hybrid designs, consist of one of these basic designs forms embedded in another design (e.g., a changing criterion design embedded in a multiple baseline design). Systematic reviews of SCEDs have repeatedly indicated that embedded designs are frequently used in applied SCED research. In spite of their popularity, specific recommendations on the conduct and analysis of embedded SCED designs are lacking to date. The purpose of the present article is therefore to provide guidance to applied researchers wishing to conduct embedded SCED designs in terms of design options, design requirements, randomization, and data analysis.

Meta-analyses and effect sizes in applied behavior analysis: A review and discussion

For more than four decades, researchers have used meta-analyses to synthesize data from multiple experimental studies often to draw conclusions that are not supported by individual studies. More recently, single-case experimental design (SCED) researchers have adopted meta-analysis techniques to answer research questions with data gleaned from SCED experiments. Meta-analyses enable researchers to answer questions regarding intervention efficacy, generality, and condition boundaries. Here we discuss meta-analysis techniques, the rationale for their adaptation with SCED studies, and current indices used to quantify the effect of SCED data in applied behavior analysis.

The Effects of Obligatory and Preferential Frames on Delay Discounting

Human decision making is partly determined by the verbal stimuli involved in a choice. Verbal stimuli that may be particularly relevant to human decision making are the words should and like, whereby should is presumably associated with what one ought to choose, and like is presumably associated with what one prefers to choose. The current study examined the potential effects of should and like on decisions in a monetary delay-discounting task. Eighty-three participants were recruited from Amazon's Mechanical Turk and were randomly assigned to a sequence of 2 conditions-should and like-in a repeated-measures experimental design. Based on condition assignment, the questions "Which should you choose?" and "Which would you like to choose?" appeared above each monetary option and its respective delay. Overall, participants demonstrated significantly lower levels of discounting in the should condition when compared to the like condition. However, this effect was much less consistent for participants exposed to the should condition prior to the like condition. The results of the current investigation indicate that the use of the words should and like constitutes separate classes of verbal stimuli that we refer to as obligatory and preferential frames. The effect of obligatory and preferential frames on delay discounting may be relevant to the prediction and control of decision making in social contexts.