Bayesian Time-Series Models in Single Case Experimental Designs: A Tutorial for Trauma Researchers

From the Individual to the Group: Using Idiographic Analyses and Two-Stage Random Effects Meta-Analysis to Obtain Population Level Inferences for within-Person Processes

In psychology, the use of portable technology and wearable devices to ease participant burden in data collection is on the rise. This creates increased interest in collecting real-time or near real-time data from individuals within their natural environments. As a result, vast amounts of observational time series data are generated. Often, motivation for collecting this data hinges on understanding within-person processes that underlie psychological phenomena. Motivated by the body of Dr. Peter Molenaar's life work calling for analytical approaches that consider potential heterogeneity and non-ergodicity, the focus of this paper is on using idiographic analyses to generate population inferences for within-person processes. Meta-analysis techniques using one-stage and two-stage random effects meta-analysis as implemented in single-case experimental designs are presented. The case for preferring a two-stage approach for meta-analysis of single-subject observational time series data is made and demonstrated using an empirical example. This provides a novel implementation of the methodology as prior implementations focus on applications to short time series with experimental designs. Inspired by Dr. Molenaar's work, we describe how an approach, two-stage random effects meta-analysis (2SRE-MA), aligns with recent calls to consider idiographic approaches when making population-level inferences regarding within-person processes.

The BEHAVE application as a tool to monitor inclusive interventions for subjects with neurodevelopmental disorders

In the last few years, many educational and therapeutic interventions for young people with neurodevelopmental disorders are based on systematic monitoring of the outcomes. These interventions are typically conducted using single-case experimental designs, (SCEDs) a set of methods aimed at testing the effect of an intervention on a single subject or a small number of subjects. In SCEDs, an effective process of decision-making needs accurate, precise, and reliable data but also that caregivers and health professionals can gather information with minimal effort. The use of Information Communication Technologies in SCEDs can support the process of data collection and analysis, facilitating the collection of accurate and reliable data, providing reports accessible also by non-experts, and promoting interactions and sharing among clinicians, educators, and caregivers. The present paper introduces the BEHAVE application, a web-based highly customizable application, designed to implement SCEDs, supporting both data collection and automatic analysis of the datasets. Moreover, the paper will describe two case studies of kindergarten children with neurodevelopmental disorders, highlighting how the BEHAVE application supported the entire process, from data collection in multiple contexts to decision-making based on the analysis provided by the system. In particular, the paper describes the case studies of Carlo and Dario, two children with severe language and communication impairments, and the inclusive education interventions carried out to maximize their participation in a typical home and school setting increasing their mand repertoire. Results revealed an increase in the mand repertoire in both children who become able to generalize the outcomes to multiple life contexts. The active participation of the caregivers played a crucial role in the ability of children to use the learned skills in settings different from the ones they were learned in.

Defining and assessing immediacy in single-case experimental designs

Immediacy is one of six data aspects (alongside level, trend, variability, overlap, and consistency) that has to be accounted for when visually analyzing single-case data. Given that it is one of the aspects that has received considerably less attention than other data aspects, the current text offers a review of the proposed conceptual definitions of immediacy (i.e., what it refers to) and also of the suggested operational definitions (i.e., how exactly is it assessed and/or quantified). Provided that a variety of conceptual and operational definitions is identified, we propose following a sensitivity analysis using a randomization test for assessing immediate effects in single-case experimental designs, by identifying when changes were most clear. In such a sensitivity analysis, the immediate effects are tested for multiple possible intervention points and for different possible operational definitions. Robust immediate effects can be detected if the results for the different operational definitions converge.

Accurate models vs. accurate estimates: A simulation study of Bayesian single-case experimental designs

Although statistical practices to evaluate intervention effects in single-case experimental design (SCEDs) have gained prominence in recent times, models are yet to incorporate and investigate all their analytic complexities. Most of these statistical models incorporate slopes and autocorrelations, both of which contribute to trend in the data. The question that arises is whether in SCED data that show trend, there is indeterminacy between estimating slope and autocorrelation, because both contribute to trend, and the data have a limited number of observations. Using Monte Carlo simulation, we compared the performance of four Bayesian change-point models: (a) intercepts only (IO), (b) slopes but no autocorrelations (SI), (c) autocorrelations but no slopes (NS), and (d) both autocorrelations and slopes (SA). Weakly informative priors were used to remain agnostic about the parameters. Coverage rates showed that for the SA model, either the slope effect size or the autocorrelation credible interval almost always erroneously contained 0, and the type II errors were prohibitively large. Considering the 0-coverage and coverage rates of slope effect size, intercept effect size, mean relative bias, and second-phase intercept relative bias, the SI model outperformed all other models. Therefore, it is recommended that researchers favor the SI model over the other three models. Research studies that develop slope effect sizes for SCEDs should consider the performance of the statistic by taking into account coverage and 0-coverage rates. These helped uncover patterns that were not realized in other simulation studies. We underline the need for investigating the use of informative priors in SCEDs.

Comparing the Bayesian Unknown Change-Point Model and Simulation Modeling Analysis to Analyze Single Case Experimental Designs

Recently, there has been an increased interest in developing statistical methodologies for analyzing single case experimental design (SCED) data to supplement visual analysis. Some of these are simulation-driven such as Bayesian methods because Bayesian methods can compensate for small sample sizes, which is a main challenge of SCEDs. Two simulation-driven approaches: Bayesian unknown change-point model (BUCP) and simulation modeling analysis (SMA) were compared in the present study for three real datasets that exhibit "clear" immediacy, "unclear" immediacy, and delayed effects. Although SMA estimates can be used to answer some aspects of functional relationship between the independent and the outcome variables, they cannot address immediacy or provide an effect size estimate that considers autocorrelation as required by the What Works Clearinghouse (WWC) Standards. BUCP overcomes these drawbacks of SMA. In final analysis, it is recommended that both visual and statistical analyses be conducted for a thorough analysis of SCEDs.