Multilevel meta-analysis of multiple regression coefficients from single-case experimental studies

Harnessing Available Evidence in Single-Case Experimental Studies: The Use of Multilevel Meta-Analysis

The use of multilevel models to combine and compare the results of multiple single-case experimental design (SCED) studies has been proposed about two decades ago. Since then, the number of multilevel meta-analyses of SCED studies steadily increased, together with the complexity of multilevel models used. At the same time, many studies were done to empirically evaluate the approach in a variety of situations, and to study how the flexibility of multilevel models can be employed to account for many complexities that often are encountered in SCED research, such as autocorrelation, linear and nonlinear time trends, specific designs, external event effects, multiple outcomes, and heterogeneity. In this paper, we give a state-of-the-art of the multilevel approach, by making an overview of basic and more extended models, summarizing simulation results, and discussing some remaining issues.

Single-case design meta-analyses in education and psychology: a systematic review of methodology

Meta-analysis is of increasing importance as this quantitative synthesis technique has the potential to summarize a tremendous amount of research evidence, which can help making evidence-based decisions in policy, practice, and theory. This paper examines the single-case meta-analyses within the Education and Psychology fields. The amount of methodological studies related to the meta-analysis of Single-Case Experimental Designs (SCEDs) is increasing rapidly, especially in these fields. This underscores the necessity of a succinct summary to help methodologists identify areas for further development in Education and Psychology research. It also aids applied researchers and research synthesists in discerning when to use meta-analytic techniques for SCED studies based on criteria such as bias, mean squared error, 95% confidence intervals, Type I error rates, and statistical power. Based on the summary of empirical evidence from 18 reports identified through a systematic search procedure, information related to meta-analytic techniques, data generation and analysis models, design conditions, statistical properties, conditions under which the meta-analytic technique is appropriate, and the study purpose(s) were extracted. The results indicate that three-level hierarchical linear modeling is the most empirically validated SCED meta-analytic technique, and parameter bias is the most prominent statistical property investigated. A large number of primary studies (more than 30) and at least 20 measurement occasions per participant are recommended for usage of SCED meta-analysis in Education and Psychology fields.

Advanced methods and implementations for the meta-analyses of animal models: Current practices and future recommendations

Meta-analytic techniques have been widely used to synthesize data from animal models of human diseases and conditions, but these analyses often face two statistical challenges due to complex nature of animal data (e.g., multiple effect sizes and multiple species): statistical dependency and confounding heterogeneity. These challenges can lead to unreliable and less informative evidence, which hinders the translation of findings from animal to human studies. We present a literature survey of meta-analysis using animal models (animal meta-analysis), showing that these issues are not adequately addressed in current practice. To address these challenges, we propose a meta-analytic framework based on multilevel (linear mixed-effects) models. Through conceptualization, formulations, and worked examples, we illustrate how this framework can appropriately address these issues while allowing for testing new questions. Additionally, we introduce other advanced techniques such as multivariate models, robust variance estimation, and meta-analysis of emergent effect sizes, which can deliver robust inferences and novel biological insights. We also provide a tutorial with annotated R code to demonstrate the implementation of these techniques.

Network Meta-Analysis for Single-Case Design Studies: An Illustration

Single-case designs (SCDs) are used to evaluate the effects of interventions on individual participants. By repeatedly measuring participants under different conditions, SCD studies focus on individual effects rather than on group summaries. The main limitation of SCDs remains its generalisability to wider populations, reducing the relevance of their findings for practice and policy making. With this limitation in mind, methodological developments for synthesising SCD data from different studies that investigate the same research question have intensified in the past decades (e.g. multilevel modelling). However, these techniques are restricted to comparing two interventions at a time and can only incorporate evidence from studies that directly compare the two treatments of interest. These limitations could be addressed by using network meta-analysis that incorporates both direct and indirect evidence to simultaneously compare multiple interventions. Despite its potential, network meta-analytical techniques have yet to be applied to SCD data. Thus, in this paper, we argue that network meta-analysis can be a valuable tool to synthesise SCD data. We demonstrate the use of network meta-analysis in SCD data using a real dataset, and we conclude by reflecting on the challenges that SCD researchers might face when applying network meta-analysis methods to their data.

Contingency-based delay to reinforcement following functional communication training for autistic individuals: A multilevel meta-analysis

LAY ABSTRACT

Functional communication training, an intervention for challenging behavior rooted in principles of applied behavior analysis, has copious empirical support dating back to the mid-1980s for autistic individuals. Recently, there has been a concerted effort to thin reinforcement delivery during functional communication training using contingency-based delays that, in turn, are designed to enhance practicality and feasibility while not compromising efficacy. In this synthesis, we meta-analyzed the literature base with the goal of investigating both combined and across type effectiveness of contingency-based delays. We also aimed to investigate moderating variables that might impact intervention outcomes. Findings showed that contingency-based delays were effective for individuals with an autism spectrum disorder diagnosis and most effective when the delay incorporated some form of positive reinforcement. In addition, differential reinforcement of alternative-based delays was overall more effective when compared to differential reinforcement of other behavior-based delays. Noteworthy moderating variables found to impact contingency-based delay efficacy included the intervention dosage and the topography of behavior. We discuss these findings and highlight directions where additional empirical research is warranted to improve our understanding about contingency-based delays for individuals diagnosed with autism spectrum disorder.