Handling missing data in randomization tests for single-case experiments: A simulation study

A Quantitative Systematic Literature Review of Combination Punishment Literature: Progress Over the Last Decade

This review evaluated single-case experimental design research that examined challenging behavior interventions utilizing punishment elements. Thirty articles published between 2013 and 2022 met study inclusion criteria. Study quality was also assessed. Through multiple levels of analysis (e.g., descriptive statistics, non-parametric statistics), we examined (a) participant and study trends, (b) differential outcomes related to temporal reinforcement approaches (antecedent, consequent, or combined reinforcement) applied alongside punishment element(s), (c) differential outcomes related to the punishment type (negative, positive) applied alongside reinforcement, and (d) effect sizes associated with study rigor across peer-reviewed and gray literature. Our results may tentatively suggest that, for certain situations, concurrently applying punishment with antecedent reinforcement approaches may coincide with significantly larger effect sizes compared to combined temporal reinforcement approaches, while positive punishment applied concurrently with reinforcement may coincide with larger but non-significant intervention effects. Most featured articles met rigor criteria, but larger effects were seen in peer-reviewed literature.

Neuromuscular electrical stimulation to combat cognitive aging in people with spinal cord injury: protocol for a single case experimental design study

INTRODUCTION

Individuals with spinal cord injury (SCI) can experience accelerated cognitive aging. Myokines (factors released from muscle cells during contractions), such as brain-derived neurotrophic factor (BDNF), are thought to have beneficial effects on cognition. Neuromuscular electrical stimulation (NMES) was shown to elicit a large release of myokines. However, the effects of NMES on cognitive function have not been studied.

OBJECTIVE

To present the study protocol for a clinical trial evaluating the effects of NMES aimed at improving cognition and BDNF.

METHODS

A replicated randomized three-phases single-case experimental design (SCED) with sequential multiple baseline time series and a single-armed prospective trial will be conducted with 15 adults with chronic SCI (> 12 months after injury) above L1 neurological level undergoing 30-min quadriceps NMES, 3 days per week for 12 weeks.

MAIN STUDY ENDPOINTS

Primary endpoint is cognitive performance (assessed by a smartphone test) conducted three times per week during the baseline phase with random duration of 3 to 8 weeks, the intervention phase of 12 weeks, and the follow-up phase of 3 weeks after a no measurement rest period of 12 weeks. Secondary endpoints are changes in BDNF levels and cognitive performance measured before the baseline period, before and after intervention and after a 12 weeks follow-up.

CONCLUSION

This will be the first study investigating the effects of 12 weeks NMES on both cognition and BDNF levels in individuals with SCI. The SCED results provide information on individual treatment effect courses which may direct future research.

TRIAL REGISTRATION

ClinicalTrials.gov (NCT05822297, 12/01/2023).

A Description of Missing Data in Single-Case Experimental Designs Studies and an Evaluation of Single Imputation Methods

Missing data is inevitable in single-case experimental designs (SCEDs) studies due to repeated measures over a period of time. Despite this fact, SCEDs implementers such as researchers, teachers, clinicians, and school psychologists usually ignore missing data in their studies. Performing analyses without considering missing data in an intervention study using SCEDs or a meta-analysis study including SCEDs studies in a topic can lead to biased results and affect the validity of individual or overall results. In addition, missingness can undermine the generalizability of SCEDs studies. Considering these drawbacks, this study aims to give descriptive and advisory information to SCEDs practitioners and researchers about missing data in single-case data. To accomplish this task, the study presents information about missing data mechanisms, item level and unit level missing data, planned missing data designs, drawbacks of ignoring missing data in SCEDs, and missing data handling methods. Since single imputation methods among missing data handling methods do not require complicated statistical knowledge, are easy to use, and hence are more likely to be used by practitioners and researchers, the present study evaluates single imputation methods in terms of intervention effect sizes and missing data rates by using a real and hypothetical data sample. This study encourages SCEDs implementers, and also meta-analysts to use some of the single imputation methods to increase the generalizability and validity of the study results in case they encounter missing data in their studies.

How to assess and take into account trend in single-case experimental design data

One of the data features that are expected to be assessed when analyzing single-case experimental designs (SCED) data is trend. The current text deals with four different questions that applied researchers can ask themselves when assessing trend and especially when dealing with improving baseline trend: (a) What options exist for assessing the presence of trend?; (b) Once assessed, what criterion can be followed for deciding whether it is necessary to control for baseline trend?; (c) What strategy can be followed for controlling for baseline trend?; and (d) How to proceed in case there is baseline trend only in some A-B comparisons? Several options are reviewed for each of these questions in the context of real data, and tentative recommendations are provided. A new user-friendly website is developed to implement the options for fitting a trend line and a criterion for selecting a specific technique for that purpose. Trend-related and more general data analytical recommendations are provided for applied researchers.Trial registration: ClinicalTrials.gov identifier: NCT04560777.

Characteristics of Missing Data in Single-Case Experimental Designs: An Investigation of Published Data

Single-case experimental designs (SCEDs) have grown in popularity in the fields such as education, psychology, medicine, and rehabilitation. Although SCEDs are valid experimental designs for determining evidence-based practices, they encounter some challenges in analyses of data. One of these challenges, missing data, is likely to be occurred frequently in SCEDs research due to repeated measurements over time. Since missing data is a critical factor that can weaken the validity and generalizability of a study, it is important to determine the characteristics of missing data in SCEDs, which are especially conducted with a small number of participants. In this regard, this study aimed to describe missing data features in SCEDs studies in detail. To accomplish this goal, 465 published SCEDs studies within the recent 5 years in six journals were included in the investigation. The overall results showed that the prevalence of missing data among SCEDs articles in at least one phase, as at least one data point, was approximately 30%. In addition, the results indicated that the missing data rates were above 10% within most studies where missing data occurred. Although missing data is so common in SCEDs research, only a handful of studies (5%) have handled missing data; however, their methods are traditional. In analyzing SCEDs data, several methods are proposed considering missing data ratios in the literature. Therefore, missing data rates determined in this study results can shed light on the analyses of SCEDs data with proper methods by improving the validity and generalizability of study results.

Genetic overlap for ten cardiovascular diseases: A comprehensive gene-centric pleiotropic association analysis and Mendelian randomization study

Recent studies suggest that pleiotropic effects may explain the genetic architecture of cardiovascular diseases (CVDs). We conducted a comprehensive gene-centric pleiotropic association analysis for ten CVDs using genome-wide association study (GWAS) summary statistics to identify pleiotropic genes and pathways that may underlie multiple CVDs. We found shared genetic mechanisms underlying the pathophysiology of CVDs, with over two-thirds of the diseases exhibiting common genes and single-nucleotide polymorphisms (SNPs). Significant positive genetic correlations were observed in more than half of paired CVDs. Additionally, we investigated the pleiotropic genes shared between different CVDs, as well as their functional pathways and distribution in different tissues. Moreover, six hub genes, including ALDH2, XPO1, HSPA1L, ESR2, WDR12, and RAB1A, as well as 26 targeted potential drugs, were identified. Our study provides further evidence for the pleiotropic effects of genetic variants on CVDs and highlights the importance of considering pleiotropy in genetic association studies.

Associations between adiposity, diabetes, lifestyle factors and the risk of gliomas

Background

Despite numerous observational studies linking adiposity, diabetes, and lifestyle factors with gliomas, the causal associations between them remain uncertain.

Methods

This study aimed to use two-sample Mendelian randomization (MR) analysis to investigate whether these associations are causal. Specifically, independent genetic variants in body mass index (BMI), waist circumference (WC), type 2 diabetes (T2D), smoking, alcohol, and coffee consumption were extracted from the published genome-wide association studies (GWASs) with genome-wide significance. The corresponding summary-level data for gliomas were available from a GWAS of 1,856 cases and 4,955 controls of European descent from the GliomaScan consortium. Additionally, glioma pathogenesis-related protein 1 data were used for validation, and Radial MR analysis was conducted to examine the potential outlier single-nucleotide polymorphisms (SNPs).

Results

One standard deviation (SD) increase in BMI had an odds ratio (OR) of 1.392 (95% confidence interval (CI), 0.935-2.071) for gliomas, while one SD increase in WC had an OR of 0.967 (95% CI, 0.547-1.710). For T2D, a one-unit increase in log-transformed OR had an OR of 0.923 (95% CI, 0.754-1.129). The prevalence of smoking initiation had an OR of 1.703 (95% CI, 0.871-3.326) for gliomas, while the prevalence of alcohol intake frequency had an OR of 0.806 (95% CI, 0.361-1.083), and the prevalence of coffee intake had an OR of 0.268 (95% CI, 0.033-2.140) for gliomas.

Conclusion

This study provides evidence that adiposity, T2D, smoking, alcohol drinking, and coffee intake do not play causal roles in the development of gliomas. The findings highlight the importance of reconsidering causal relationships in epidemiological research to better understand the risk factors and prevention strategies for gliomas.

Emotionally focused couple therapy in cancer survivor couples with marital and sexual problems: a replicated single-case experimental design

Objective

The current research examined the effect of Emotionally Focused Couples Therapy (EFCT) on perceived intimacy, affect, and dyadic connection in cancer survivor couples with relationship challenges.

Method

In this longitudinal replicated single-case study, positive and negative affect, intimacy, partner responsiveness, and expression of attachment-based emotional needs were reported every 3 days before and during treatment. Thirteen couples, with one partner having survived colorectal cancer or breast cancer, participated for the full duration of the study. Statistical analysis of the data was performed using randomization tests, piecewise regression, and multilevel analyses.

Results

Adherence to the therapeutic protocol was tested and found adequate. Compared with baseline, significant positive effects on affect variables were found during the therapeutic process. Positive affect increased and negative affect decreased. Partner responsiveness, perceived intimacy, and the expression of attachment-based emotional needs improved, but only in the later phase of treatment. Results at the group level were statistically significant, whereas effects at the individual level were not.

Discussion

This study found positive group-level effects of EFCT on affect and dyadic outcome measures in cancer survivors. The positive results warrant further research, including randomized clinical trials, to replicate these effects of EFCT in cancer survivor couples experiencing marital and sexual problems.

The randomized marker method for single-case randomization tests: Handling data missing at random and data missing not at random

Single-case experiments are frequently plagued by missing data problems. In a recent study, the randomized marker method was found to be valid and powerful for single-case randomization tests when the missing data were missing completely at random. However, in real-life experiments, it is difficult for researchers to ascertain the missing data mechanism. For analyzing such experiments, it is essential that the missing data handling method is valid and powerful for various missing data mechanisms. Hence, we examined the performance of the randomized marker method for data that are missing at random and data that are missing not at random. In addition, we compared the randomized marker method with multiple imputation, because the latter is often considered the gold standard among imputation techniques. To compare and evaluate these two methods under various simulation conditions, we calculated the type I error rate and statistical power in single-case randomization tests using these two methods of handling missing data and compared them to the type I error rate and statistical power using complete datasets. The results indicate that while multiple imputation presents an advantage in the presence of strongly correlated covariate data, the randomized marker method remains valid and results in sufficient statistical power for most of the missing data conditions simulated in this study.

Individual Patterns and Temporal Trajectories of Changes in Fear and Pain during Exposure In Vivo: A Multiple Single-Case Experimental Design in Patients with Chronic Pain

Exposure in vivo (EXP) is an effective treatment to reduce pain-related fear and disability in chronic pain populations. Yet, it remains unclear how reductions in fear and pain relate to each other. This single-case experimental design study attempted to identify patterns in the individual responses to EXP and to unravel temporal trajectories of fear and pain. Daily diaries were completed before, during and after EXP. Multilevel modelling analyses were performed to evaluate the overall effect. Temporal effects were scrutinized by individual regression analyses and determination of the time to reach a minimal clinically important difference. Furthermore, individual graphs were visually inspected for potential patterns. Twenty patients with chronic low back pain and complex regional pain syndrome type I were included. On a group level, both fear and pain were reduced following EXP. Individually, fear was significantly reduced in 65% of the patients, while pain in only 20%. A decrease in fear was seen mostly in the first weeks, while pain levels reduced later or remained unchanged. Daily measurements provided rich data on temporal trajectories of reductions in fear and pain. Overall, reductions in fear preceded pain relief and seemed to be essential to achieve pain reductions.

Methods for Modeling Autocorrelation and Handling Missing Data in Mediation Analysis in Single Case Experimental Designs (SCEDs)

Single-Case Experimental Designs (SCEDs) are increasingly recognized as a valuable alternative to group designs. Mediation analysis is useful in SCEDs contexts because it informs researchers about the underlying mechanism through which an intervention influences the outcome. However, methods for conducting mediation analysis in SCEDs have only recently been proposed. Furthermore, repeated measures of a target behavior present the challenges of autocorrelation and missing data. This paper aims to extend methods for estimating indirect effects in piecewise regression analysis in SCEDs by (1) evaluating three methods for modeling autocorrelation, namely, Newey-West (NW) estimation, feasible generalized least squares (FGLS) estimation, and explicit modeling of an autoregressive structure of order one (AR(1)) in the error terms and (2) evaluating multiple imputation in the presence of data that are missing completely at random. FGLS and AR(1) outperformed NW and OLS estimation in terms of efficiency, Type I error rates, and coverage, while OLS was superior to the methods in terms of power for larger samples. The performance of all methods is consistent across 0% and 20% missing data conditions. 50% missing data led to unsatisfactory power and biased estimates. In light of these findings, we provide recommendations for applied researchers.

A practitioner's guide to conducting and analysing embedded randomized single-case experimental designs

Single-case experimental designs (SCEDs) are a class of experimental designs suited for answering research questions at an individual level. The main designs available in SCED research are phase designs, multiple baseline designs, alternation designs, and changing criterion designs. Embedded designs, also referred to as combination or hybrid designs, consist of one of these basic designs forms embedded in another design (e.g., a changing criterion design embedded in a multiple baseline design). Systematic reviews of SCEDs have repeatedly indicated that embedded designs are frequently used in applied SCED research. In spite of their popularity, specific recommendations on the conduct and analysis of embedded SCED designs are lacking to date. The purpose of the present article is therefore to provide guidance to applied researchers wishing to conduct embedded SCED designs in terms of design options, design requirements, randomization, and data analysis.

Single-case design standards: An update and proposed upgrades

In this paper, we provide a critique focused on the What Works Clearinghouse (WWC) Standards for Single-Case Research Design (Standards 4.1). Specifically, we (a) recommend the use of visual-analysis to verify a single-case intervention study's design standards and to examine the study's operational issues, (b) identify limitations of the design-comparable effect-size measure and discuss related statistical matters, (c) review the applicability and practicality of Standards 4.1 to single-case designs (SCDs), and (d) recommend inclusion of content pertaining to diversity, equity, and inclusion in future standards. Within the historical context of the WWC Pilot Standards for Single-Case Design (1.0), we suggest that Standards 4.1 may best serve as standards for meta-analyses of SCDs but will need to make clear distinctions among the various types of SCD studies that are included in any research synthesis. In this regard, we argue for transparency in SCD studies that meet design standards and those that do not meet design standards in any meta-analysis emanating from the WWC. The intent of these recommendations is to advance the science of SCD research both in research synthesis and in promoting evidence-based practices.

Different randomized multiple-baseline models for different situations: A practical guide for single-case intervention researchers

Single-case researchers often implement multiple-baseline designs as their preferred methodology for intervention evaluations. Recent writings and empirical investigations have argued in favor of incorporating various forms of randomization into such designs for the purpose of elevating the intervention study's internal validity and scientific credibility. In this article, we consider a variety of randomized multiple-baseline designs and associated randomization statistical tests, along with their potential strengths and limitations. In what amounts to a practical guide, we refer school psychology researchers to these versatile randomization procedures for planning and executing their intervention studies.

Assessing Intervention Effects in the Presence of Missing Scores

Due to repeated observations of an outcome behavior in N-of-1 or single-case design (SCD) intervention studies, the occurrence of missing scores is inevitable in such studies. Approximately 21% of SCD articles published in five reputable journals between 2015 and 2019 exhibited evidence of missing scores. Missing rates varied by designs, with the highest rate (24%) found in multiple baseline/probe designs. Missing scores cause difficulties in data analysis. And inappropriate treatments of missing scores lead to consequences that threaten internal validity and weaken generalizability of intervention effects reported in SCD research. In this paper, we comprehensively review nine methods for treating missing SCD data: the available data method, six single imputations, and two model-based methods. The strengths, weaknesses, assumptions, and examples of these methods are summarized. The available data method and three single imputation methods are further demonstrated in assessing an intervention effect at the class and students’ levels. Assessment results are interpreted in terms of effect sizes, statistical significances, and visual analysis of data. Differences in results among the four methods are noted and discussed. The extensive review of problems caused by missing scores and possible treatments should empower researchers and practitioners to account for missing scores effectively and to support evidence-based interventions vigorously. The paper concludes with a discussion of contingencies for implementing the nine methods and practical strategies for managing missing scores in single-case intervention studies.