A Gradual Effects Model for Single-Case Designs

How to assess and take into account trend in single-case experimental design data

One of the data features that are expected to be assessed when analyzing single-case experimental designs (SCED) data is trend. The current text deals with four different questions that applied researchers can ask themselves when assessing trend and especially when dealing with improving baseline trend: (a) What options exist for assessing the presence of trend?; (b) Once assessed, what criterion can be followed for deciding whether it is necessary to control for baseline trend?; (c) What strategy can be followed for controlling for baseline trend?; and (d) How to proceed in case there is baseline trend only in some A-B comparisons? Several options are reviewed for each of these questions in the context of real data, and tentative recommendations are provided. A new user-friendly website is developed to implement the options for fitting a trend line and a criterion for selecting a specific technique for that purpose. Trend-related and more general data analytical recommendations are provided for applied researchers.Trial registration: ClinicalTrials.gov identifier: NCT04560777.

Testing delayed, gradual, and temporary treatment effects in randomized single-case experiments: A general response function framework

Randomization tests represent a class of significance tests to assess the statistical significance of treatment effects in randomized single-case experiments. Most applications of single-case randomization tests concern simple treatment effects: immediate, abrupt, and permanent changes in the level of the outcome variable. However, researchers are confronted with delayed, gradual, and temporary treatment effects; in general, with "response functions" that are markedly different from single-step functions. We here introduce a general framework that allows specifying a test statistic for a randomization test based on predicted response functions that is sensitive to a wide variety of data patterns beyond immediate and sustained changes in level: different latencies (degrees of delay) of effect, abrupt versus gradual effects, and different durations of the effect (permanent or temporary). There may be reasonable expectations regarding the kind of effect (abrupt or gradual), entailing a different focal data feature (e.g., level or slope). However, the exact amount of latency and the exact duration of a temporary effect may not be known a priori, justifying an exploratory approach studying the effect of specifying different latencies or delayed effects and different durations for temporary effects. We provide illustrations of the proposal with real data, and we present a user-friendly freely available web application implementing it.

Typicality of Level Change (TLC) as an Additional Effect Measure to NAP and Tau-U in Single Case Research

Single case research is a viable way to obtain evidence for social and psychological interventions on an individual level. Across single case research studies various analysis strategies are employed, varying from visual analysis to the calculation of effect sizes. To calculate effect sizes in studies with few measurements per time period (<40 data points with a minimum of five data points in each phase), non-parametric indices such as Nonoverlap of All Pairs (NAP) and Tau-U are recommended. However, both indices have restrictions. This article discusses the restrictions of NAP and Tau-U and presents the description, calculation, and benefits of an additional effect size, called the Typicality of Level Change (TLC) index. In comparison to NAP and Tau-U, the TLC index is more aligned to visual analysis, not restricted by a ceiling effect, and does not overcompensate for problematic trends in data. The TLC index is also sensitive to the typicality of an effect. TLC is an important addition to ease the restrictions of current nonoverlap methods when comparing effect sizes between cases and studies.

Assessing differential land use impacts on soil quality: A method based on log-response ratios and polygonal projections

Soil quality indices (SQI) used for assessing soil degradation are often developed using additive scoring functions. However, these SQI may lack reference values for interpreting their outputs and the capacity to differentiate changes in specific variables. To overcome these limitations, this study introduces SQI using Log Response Ratios (LRR) as measures of size effects caused by land use in physical, chemical, and microbiological soil quality indicators. LRR vectors projected 2D polygons with condensed change measures along their perimeters. This method was tested in andosols within the southeastern region of Antioquia, Colombia. These soils were subjected to contrasting stages of degradation determined by the extent of A-horizon removal due to land use practices. This study shows that mining and agriculture have detrimental effects on soil organic carbon and water contents, and that size effects vary significantly between land uses (p < 0.05). Microbiological features also exhibit distinct size effects, such as populations of culturable mesophilic bacteria and fungi, microbial basal respiration, spore density of arbuscular mycorrhizal fungi (AMF), their diversity, and total glomalin-related soil proteins (p < 0.05). The SQI proposed exhibited a negative correlation with SQI computed from scoring additive functions either considering the entire dataset (R2 = 0.87) or a minimum dataset (R2 = 0.90). This approach underscores the utility of using LRR geometrical analysis to assess global soil quality differences among land uses (p < 0.01), offering a visual, quantifiable representation of the effects of each land use over specific soil quality indicators.

Does the choice of a linear trend-assessment technique matter in the context of single-case data?

Trend is one of the data aspects that is an object of assessment in the context of single-case experimental designs. This assessment can be performed both visually and quantitatively. Given that trend, just like other relevant data features such as level, immediacy, or overlap does not have a single operative definition, a comparison among the existing alternatives is necessary. Previous studies have included illustrations of differences between trend-line fitting techniques using real data. In the current study, I carry out a simulation to study the degree to which different trend-line fitting techniques lead to different degrees of bias, mean square error, and statistical power for a variety of quantifications that entail trend lines. The simulation involves generating both continuous and count data, for several phase lengths, degrees of autocorrelation, and effect sizes (change in level and change in slope). The results suggest that, in general, ordinary least squares estimation performs well in terms of relative bias and mean square error. Especially, a quantification of slope change is associated with better statistical results than quantifying an average difference between conditions on the basis of a projected baseline trend. In contrast, the performance of the split-middle (bisplit) technique is less than optimal.

A Visual Aid and Objective Rule Encompassing the Data Features of Visual Analysis

Visual analysis of single-case research is commonly described as a gold standard, but it is often unreliable. Thus, an objective tool for applying visual analysis is necessary, as an alternative to the Conservative Dual Criterion, which presents some drawbacks. The proposed free web-based tool enables assessing change in trend and level between two adjacent phases, while taking data variability into account. The application of the tool results in (a) a dichotomous decision regarding the presence or absence of an immediate effect, a progressive or delayed effect, or an overall effect and (b) a quantification of overlap. The proposal is evaluated by applying it to both real and simulated data, obtaining favorable results. The visual aid and the objective rules are expected to make visual analysis more consistent, but they are not intended as a substitute for the analysts' judgment, as a formal test of statistical significance, or as a tool for assessing social validity.

An Examination of Measurement Procedures and Characteristics of Baseline Outcome Data in Single-Case Research

There has been growing interest in using statistical methods to analyze data and estimate effect size indices from studies that use single-case designs (SCDs), as a complement to traditional visual inspection methods. The validity of a statistical method rests on whether its assumptions are plausible representations of the process by which the data were collected, yet there is evidence that some assumptions-particularly regarding normality of error distributions-may be inappropriate for single-case data. To develop more appropriate modeling assumptions and statistical methods, researchers must attend to the features of real SCD data. In this study, we examine several features of SCDs with behavioral outcome measures in order to inform development of statistical methods. Drawing on a corpus of over 300 studies, including approximately 1,800 cases, from seven systematic reviews that cover a range of interventions and outcome constructs, we report the distribution of study designs, distribution of outcome measurement procedures, and features of baseline outcome data distributions for the most common types of measurements used in single-case research. We discuss implications for the development of more realistic assumptions regarding outcome distributions in SCD studies, as well as the design of Monte Carlo simulation studies evaluating the performance of statistical analysis techniques for SCD data.

Reproducibility in Small-N Treatment Research: A Tutorial Using Examples From Aphasiology

PURPOSE

Small-N studies are the dominant study design supporting evidence-based interventions in communication science and disorders, including treatments for aphasia and related disorders. However, there is little guidance for conducting reproducible analyses or selecting appropriate effect sizes in small-N studies, which has implications for scientific review, rigor, and replication. This tutorial aims to (a) demonstrate how to conduct reproducible analyses using effect sizes common to research in aphasia and related disorders and (b) provide a conceptual discussion to improve the reader's understanding of these effect sizes.

METHOD

We provide a tutorial on reproducible analyses of small-N designs in the statistical programming language R using published data from Wambaugh et al. (2017). In addition, we discuss the strengths, weaknesses, reporting requirements, and impact of experimental design decisions on effect sizes common to this body of research.

RESULTS

Reproducible code demonstrates implementation and comparison of within-case standardized mean difference, proportion of maximal gain, tau-U, and frequentist and Bayesian mixed-effects models. Data, code, and an interactive web application are available as a resource for researchers, clinicians, and students.

CONCLUSIONS

Pursuing reproducible research is key to promoting transparency in small-N treatment research. Researchers and clinicians must understand the properties of common effect size measures to make informed decisions in order to select ideal effect size measures and act as informed consumers of small-N studies. Together, a commitment to reproducibility and a keen understanding of effect sizes can improve the scientific rigor and synthesis of the evidence supporting clinical services in aphasiology and in communication sciences and disorders more broadly. Supplemental Material and Open Science Form: https://doi.org/10.23641/asha.21699476.

Meta-Analyses of Reading Intervention Studies Including Students With Learning Disabilities: A Methodological Review

Learning disabilities (LD) may affect a range of academic skills but are most often observed in reading. Researchers and policymakers increasingly recommend addressing reading difficulties encountered by students with LD using evidence-based practices, or interventions validated through multiple, high-quality research studies. A valuable tool in identifying evidence-based practices is the meta-analysis, which entails statistically aggregating the results obtained through primary studies. Specific methods used in meta-analyses have the potential to influence their findings, with ramifications for research and practice. This review assessed the methodological features of the systematic reviews and analytic procedures featured in meta-analyses of reading intervention studies that included students with LD written between 2000 and 2020. Identified articles (N = 23) suggest that meta-analyses have become more prevalent and transparent over time, notwithstanding issues related to publication bias and the opacity of coding procedures. A discussion of implications follows a description of results.

Metabarcoding Reveals Impact of Different Land Uses on Fungal Diversity in the South-Eastern Region of Antioquia, Colombia

Changes in soil fungal communities caused by land use have not been sufficiently studied in South American Andosols, which are considered key food production areas. Since fungal communities play an important role in soil functionality, this study analysed 26 soil samples of Andosols collected from locations devoted to conservation, agriculture and mining activities in Antioquia, Colombia, to establish differences between fungal communities as indicators of soil biodiversity loss using Illumina MiSeq metabarcoding on nuclear ribosomal ITS2 region. A non-metric multidimensional scaling allowed to explore driver factors of changes in fungal communities, while the significance of these variations was assessed by PERMANOVA. Furthermore, the effect size of land use over relevant taxa was quantified. Our results suggest a good coverage of fungal diversity with a detection of 353,312 high-quality ITS2 sequences. We found strong correlations of Shannon and Fisher indexes with dissimilarities on fungal communities (r = 0.94). These correlations allow grouping soil samples according to land use. Variations in temperature, air humidity and organic matter content lead to changes in abundances of relevant orders (Wallemiales and Trichosporonales). The study highlights specific sensitivities of fungal biodiversity features in tropical Andosols, which may serve as a basis for robust assessments of soil quality in the region.

Defining and assessing immediacy in single-case experimental designs

Immediacy is one of six data aspects (alongside level, trend, variability, overlap, and consistency) that has to be accounted for when visually analyzing single-case data. Given that it is one of the aspects that has received considerably less attention than other data aspects, the current text offers a review of the proposed conceptual definitions of immediacy (i.e., what it refers to) and also of the suggested operational definitions (i.e., how exactly is it assessed and/or quantified). Provided that a variety of conceptual and operational definitions is identified, we propose following a sensitivity analysis using a randomization test for assessing immediate effects in single-case experimental designs, by identifying when changes were most clear. In such a sensitivity analysis, the immediate effects are tested for multiple possible intervention points and for different possible operational definitions. Robust immediate effects can be detected if the results for the different operational definitions converge.

Interpreting treatment effect size from single case experimental design data: a preliminary analysis of differential effects of treatments designed to increase or decrease behaviour

BACKGROUND

Estimates of treatment effect size from single case experimental design (SCED) data may be impacted by the direction for treatment effects (i.e. ascending or descending slope for the dependent variable). Estimating effect sizes for treatments designed to decrease behaviour are potentially more restricted because the intended direction for treatment is zero (i.e. an absolute basal). Conversely, effect sizes for interventions that increase behaviour are less restricted due to a relatively unconstrained ceiling from a pure measurement standpoint (i.e. no absolute ceiling). That is, treatments that increase behaviour have a broader range of possible effect size values as the ceiling is only limited by demand characteristics and the learners' skills and motivation to exhibit the behaviour.

METHOD

The current study represents a preliminary analysis of the mean and range of SCED effect sizes for treatments designed to either increase or decrease target behaviour. A within-case Cohen's d measure that was developed for SCED data was used to estimate treatment effect sizes.

RESULTS

Results indicated that the mean and range of effect size values for treatments that increased behaviour were significantly greater compared with treatments that decreased behaviour.

CONCLUSIONS

Results are discussed in terms of developing standards, or best practices, specific to interpreting effect size values and meeting quality control requirements for inclusion of the data set in future SCED meta-analytic studies estimating treatment effect size. Specifically, preliminary results suggest that benchmarks for low, medium and high SCED effect size values need to be developed separately for treatments that increase or decrease levels of the dependent variable.

Performance criteria-based effect size (PCES) measurement of single-case experimental designs: A real-world data study

Visual analysis and nonoverlap-based effect sizes are predominantly used in analyzing single case experimental designs (SCEDs). Although they are popular analytical methods for SCEDs, they have certain limitations. In this study, a new effect size calculation model for SCEDs, named performance criteria-based effect size (PCES), is proposed considering the limitations of 4 nonoverlap-based effect size measures, widely accepted in the literature and that blend well with visual analysis. In the field test of PCES, actual data from published studies were utilized, and the relations between PCES, visual analysis, and the 4 nonoverlap-based methods were examined. In determining the data to be used in the field test, 1,052 tiers (AB phases) were identified from 6 journals. The results revealed a weak or moderate relation between PCES and nonoverlap-based methods due to its focus on performance criteria. Although PCES has some weaknesses, it promises to eliminate the causes that may create issues in nonoverlap-based methods, using quantitative data to determine socially important changes in behavior and to complement visual analysis.

Reduced human activity in shallow reefs during the COVID-19 pandemic increases fish evenness

The COVID-19 pandemic provides a rare opportunity to examine effects of people on natural systems and processes. Here, we collected fish diversity data from coral reefs at the Israeli Gulf of Aqaba during and after the COVID-19 lockdown. We examined beach entrances to the reef, nearby shallow reefs and deeper areas exposed mostly to divers. We found that the lockdown elicited a behavioral response that resulted in elevated species richness at designated reef entrances, predominantly influenced by increased evenness without changes to total abundances. This effect was observed both at the local scale and when several beach entrances were aggregated together. Consequently, non-extractive human activities may have substantial short-term impacts on fish diversity. Our insights could help designate guidelines to manage visitor impacts on coral reefs and aid in their prolonged persistence.

Five direct behavior rating multi-item scales: Sensitivity to the effects of classroom interventions

Direct Behavior Rating (DBR) is a tool designed for the assessment of behavioral changes over time. Unlike methods for summative evaluations, the development of progress monitoring tools requires evaluation of sensitivity to change. The present study aimed to evaluate this psychometric feature of five newly developed DBR Multi-Item Scales (DBR-MIS). Teachers identified students with behaviors interfering with their learning or the learning of others and implemented a Daily Report Card (DRC) intervention in the classroom settings for two months. The analyses were performed on 31 AB single case studies. Change metrics were calculated at an individual level by using Tau-U_{A vs. B + trend B} and Hedges' g and at a scale-level by using Mixed Effect Meta-Analysis, Hierarchical Linear Models (HLMs), and Between-Case Standardized Mean Difference (BC-SMD). HLMs were estimated considering both fixed and random effects of intervention and linear trend within the intervention phase. The results supported sensitivity to change for three DBR-MIS (i.e., Academic Engagement, Organizational Skills, and Disruptive Behavior), and the relative magnitudes were consistent across the metrics. Sensitivity to change of DBR-MIS Interpersonal Skills received moderate support. Conversely, empirical evidence was not provided for sensitivity to change of DBR-MIS Oppositional Behavior. Particular emphasis was placed on the intervention trend in that responses to behavioral interventions might occur gradually or require consistency over time in order to be observed by raters. Implications for the use of the new DBR-MIS in the context of progress monitoring of social-emotional behaviors are discussed.

Assessing the Benefits of Forested Riparian Zones: A Qualitative Index of Riparian Integrity Is Positively Associated with Ecological Status in European Streams

Developing a general, predictive understanding of ecological systems requires knowing how much structural and functional relationships can cross scales and contexts. Here, we introduce the CROSSLINK project that investigates the role of forested riparian buffers in modified European landscapes by measuring a wide range of ecosystem attributes in stream-riparian networks. CROSSLINK involves replicated field measurements in four case-study basins with varying levels of human development: Norway (Oslo Fjord), Sweden (Lake Mälaren), Belgium (Zwalm River), and Romania (Argeş River). Nested within these case-study basins include multiple, independent stream-site pairs with a forested riparian buffer and unbuffered section located upstream, as well as headwater and downstream sites to show cumulative land-use impacts. CROSSLINK applies existing and bespoke methods to describe habitat conditions, biodiversity, and ecosystem functioning in aquatic and terrestrial habitats. Here, we summarize the approaches used, detail protocols in supplementary materials, and explain how data is applied in an optimization framework to better manage tradeoffs in multifunctional landscapes. We then present results demonstrating the range of riparian conditions present in our case-study basins and how these environmental states influence stream ecological integrity with the commonly used macroinvertebrate Average Score Per Taxon (ASPT) index. We demonstrate that a qualitative index of riparian integrity can be positively associated with stream ecological status. This introduction to the CROSSLINK project shows the potential for our replicated study with its panoply of ecosystem attributes to help guide management decisions regarding the use of forested riparian buffers in human-impacted landscapes. This knowledge is highly relevant in a time of rapid environmental change where freshwater biodiversity is increasingly under pressure from a range of human impacts that include habitat loss, pollution, and climate change.

Power of a randomization test in a single case multiple baseline AB design

A randomization test can be used to statistically test hypotheses in multiple baseline designs to complement the commonly used visual inspection analysis. A crossed factor simulation study was performed to investigate the power of a randomization test in an multiple baseline design. The results show that the degree of autocorrelation of the observations, the number of participants, the effect size, the overlap of possible start moments of the intervention between participants, the ratio of the number of measurements in the baseline- and intervention phase, a gradually emerging effect, and the number of measurements had strong main effects on the power. The two-way interactions between number of participants and effect size, and between the number of measurements and the number of start moments of the intervention also had a large effect. An online tool was developed to calculate the power of a multiple baseline design given several design characteristics.

A Gradual Effects Model for Single-Case Designs

Single-case designs are a class of repeated measures experiments used to evaluate the effects of interventions for small or specialized populations, such as individuals with low-incidence disabilities. There has been growing interest in systematic reviews and syntheses of evidence from single-case designs, but there remains a need to further develop appropriate statistical models and effect sizes for data from the designs. We propose a novel model for single-case data that exhibit nonlinear time trends created by an intervention that produces gradual effects, which build up and dissipate over time. The model expresses a structural relationship between a pattern of treatment assignment and an outcome variable, making it appropriate for both treatment reversal and multiple baseline designs. It is formulated as a generalized linear model so that it can be applied to outcomes measured as frequency counts or proportions, both of which are commonly used in single-case research, while providing readily interpretable effect size estimates such as log response ratios or log odds ratios. We demonstrate the gradual effects model by applying it to data from a single-case study and examine the performance of proposed estimation methods in a Monte Carlo simulation of frequency count data.