How Many Tiers Do We Need? Type I Errors and Power in Multiple Baseline Designs

Assessment of executive function impairments in adults with acquired brain injury across single-case experimental design: A scoping review of primary outcomes

The use of single-case experimental design (SCED) to evaluate cognitive remediation is growing. SCEDs require rigorous methodology and appropriate choice of primary outcomes. To review primary outcomes that assess executive function impairments in patients with acquired brain injury (ABI). A scoping review was conducted using the Arksey and O'Malley framework and the PRISMA extension for scoping review (PRISMA-ScR). Five databases were searched resulting in the inclusion of twenty-one studies. Primary outcomes were extracted and classified according to the type of measure, ecological setting and sources of possible bias. A wide variety of primary outcomes were identified; the majority of which evaluated behavioural performance during a real-life activity or in a real-life simulated setting. The most frequent bias observed across outcomes was the coaching effect. The findings of this scoping review highlight the importance of selecting appropriate outcomes for repeated measures in SCED studies. Direct observation of the target behaviour is a potential gold standard for assessing the effectiveness of a cognitive intervention.

The impact of a hospital-based special care unit on behavioural and psychological symptoms in older people living with dementia

BACKGROUND

Hospital patients with behavioural and psychological symptoms of dementia (BPSD) are vulnerable to a range of adverse outcomes. Hospital-based Special Care Units (SCUs) are secure dementia-enabling environments providing specialised gerontological care. Due to a scarcity of research, their value remains unconfirmed.

OBJECTIVE

To compare hospital based SCU management of BPSD with standard care.

DESIGN

Single-case multiple baseline design.

SETTING AND PARTICIPANTS

One-hundred admissions to an 8-bed SCU over 2 years in a large Australian public hospital.

METHODS

Repeated measures of BPSD severity were undertaken prospectively by specialist dementia nurses for patients admitted to a general ward (standard care) and transferred to the SCU. Demographic and other clinical data, including diagnoses, medication use, and care-related outcomes were obtained from medical records retrospectively. Analysis used multilevel models to regress BPSD scores onto care-setting outcomes, adjusting for time and other factors.

RESULTS

When receiving standard care, patients' BPSD severity was 6.8 (95% CI 6.04-7.64) points higher for aggression, 15.6 (95% CI 13.90-17.42) points higher for the neuropsychiatric inventory, and 5.8 (95% CI 5.14-6.50) points higher for non-aggressive agitation compared to SCU. Patients receiving standard care also experienced increased odds for patient-to-nurse violence (OR 2.61, 95% CI 1.67-4.09), security callouts (OR 5.39 95% CI 3.40-8.52), physical restraint (OR 17.20, 95% CI 7.94-37.25) and antipsychotic administration (OR 3.41, 95% CI 1.60-7.24).

CONCLUSION

Clinically significant reductions in BPSD and psychotropic administration were associated with SCU care relative to standard ward care. These results suggest more robust investigation of hospital SCUs, and dementia-enabling design are warranted.

The Transtheoretical model's processes of change in the heart of a physical activity intervention: A series of n-of-1

The objectives were a) to test whether a Processes of Change (POC)-personalized Transtheoretical model (TTM)-based intervention could increase physical activity (PA) among inactive adults, and b) to examine whether the intervention increased the level of TTM theoretical constructs. The following hypotheses were formulated: 1) PA levels will be significantly higher during and after the intervention in comparison to baseline measures; 2) the level of targeted POCs will increase during the intervention; 3) non targeted POCs will stay stable, and 4) self-efficacy and decisional balance levels will increase during the intervention. A series of N-of-1 with A (1 to 2-week)-B(10-week)-A'(2-week) design were conducted with 12 inactive adults. Behavioral counselors used behavior change techniques to target TTM constructs and supervise PA. Interventions were individualized based on the 5 POCs with the lowest pre-intervention level. Device-based and subjective PA along with TTM measures were collected weekly online. PA data were analyzed with piecewise linear models. A visual analysis was run to examine the TTM constructs. Device, self-reported and TTM data were available for five, seven and five participants, respectively. A significant self-reported PA increase for six participants was found during the phase B and A2. A significant device-measured PA increase was observed in two participants during the study. A substantial increase of targeted POC from baseline for all participants with available data was observed. This study provides the first evidence of behavioral and psychological effects of a POC-personalized TTM-based intervention in inactive adults.

Single-case experimental designs for child neurological rehabilitation and developmental disability research

Single-case experimental designs (SCEDs) are a group of methodologies of growing interest, aiming to test the effectiveness of an intervention at the single-participant level, using a rigorous and prospective methodology. SCEDs may promote flexibility on how we design research protocols and inform clinical decision-making, especially for personalized outcome measures, inclusion of families with challenging needs, measurement of children's progress in relation to parental implementation of interventions, and focus on personal goals. Design options for SCEDs are discussed in relation to an expected on/off effect of the intervention (e.g. school/environmental adaptation, assistive technology devices) or, alternatively, on an expected carry-on/maintenance of effects (interventions aiming to develop or restore a function). Randomization in multiple-baseline designs and 'power' calculations are explained. The most frequent reasons for not detecting an intervention effect in SCEDs are also presented, especially in relation to baseline length, trend, and instability. The use of SCEDs on the front and back ends of randomized controlled trials is discussed.

Defining and assessing immediacy in single-case experimental designs

Immediacy is one of six data aspects (alongside level, trend, variability, overlap, and consistency) that has to be accounted for when visually analyzing single-case data. Given that it is one of the aspects that has received considerably less attention than other data aspects, the current text offers a review of the proposed conceptual definitions of immediacy (i.e., what it refers to) and also of the suggested operational definitions (i.e., how exactly is it assessed and/or quantified). Provided that a variety of conceptual and operational definitions is identified, we propose following a sensitivity analysis using a randomization test for assessing immediate effects in single-case experimental designs, by identifying when changes were most clear. In such a sensitivity analysis, the immediate effects are tested for multiple possible intervention points and for different possible operational definitions. Robust immediate effects can be detected if the results for the different operational definitions converge.

Effects of adding video feedback to emotionally focused therapy supervision: A concurrent multiple-baseline across subjects design

Emotionally focused therapy (EFT) is an empirically supported intervention for relationship distress with an established model of supervision. This study examined whether incorporating video feedback (VF) software into EFT supervision would improve therapists' level of development compared to traditional EFT supervision in a university training clinic. A concurrent multiple-baseline across subjects design, along with a thematic analysis of qualitative data, were used in this proof-of-concept study of the new supervision component. Overall, quantitative findings suggested that using VF in EFT supervision resulted in some improvement to therapists' development, while qualitative findings showed that all participants supported the incorporation of VF into EFT supervision. Future research on VF is needed to provide additional insight into the use of video review supervision.

Performance criteria-based effect size (PCES) measurement of single-case experimental designs: A real-world data study

Visual analysis and nonoverlap-based effect sizes are predominantly used in analyzing single case experimental designs (SCEDs). Although they are popular analytical methods for SCEDs, they have certain limitations. In this study, a new effect size calculation model for SCEDs, named performance criteria-based effect size (PCES), is proposed considering the limitations of 4 nonoverlap-based effect size measures, widely accepted in the literature and that blend well with visual analysis. In the field test of PCES, actual data from published studies were utilized, and the relations between PCES, visual analysis, and the 4 nonoverlap-based methods were examined. In determining the data to be used in the field test, 1,052 tiers (AB phases) were identified from 6 journals. The results revealed a weak or moderate relation between PCES and nonoverlap-based methods due to its focus on performance criteria. Although PCES has some weaknesses, it promises to eliminate the causes that may create issues in nonoverlap-based methods, using quantitative data to determine socially important changes in behavior and to complement visual analysis.

A proposal for the assessment of replication of effects in single-case experimental designs

In science in general and in the context of single‐case experimental designs, replication of the effects of the intervention within and/or across participants or experiments is crucial for establishing causality and for assessing the generality of the intervention effect. Specific developments and proposals for assessing whether an effect has been replicated or not (or to what extent) are scarce, in the general context of behavioral sciences, and practically null in the single‐case experimental designs context. We propose an extension of the modified Brinley plot for assessing how many of the effects replicate. To make this assessment possible, a definition of replication is suggested, on the basis of expert judgment, rather than on statistical criteria. The definition of replication and its graphical representation are justified, presenting their strengths and limitations, and illustrated with real data. A user‐friendly software is made available for obtaining automatically the graphical representation.

Threats to Internal Validity in Multiple-Baseline Design Variations

Multiple baseline designs—both concurrent and nonconcurrent—are the predominant experimental design in modern applied behavior analytic research and are increasingly employed in other disciplines. In the past, there was significant controversy regarding the relative rigor of concurrent and nonconcurrent multiple baseline designs. The consensus in recent textbooks and methodological papers is that nonconcurrent designs are less rigorous than concurrent designs because of their presumed limited ability to address the threat of coincidental events (i.e., history). This skepticism of nonconcurrent designs stems from an emphasis on the importance of across-tier comparisons and relatively low importance placed on replicated within-tier comparisons for addressing threats to internal validity and establishing experimental control. In this article, we argue that the primary reliance on across-tier comparisons and the resulting deprecation of nonconcurrent designs are not well-justified. In this article, we first define multiple baseline designs, describe common threats to internal validity, and delineate the two bases for controlling these threats. Second, we briefly summarize historical methodological writing and current textbook treatment of these designs. Third, we explore how concurrent and nonconcurrent multiple baselines address each of the main threats to internal validity. Finally, we make recommendations for more rigorous use, reporting, and evaluation of multiple baseline designs.

The No Worries Trial: Efficacy of Online Dialectical Behaviour Therapy Skills Training for Chronic Pain (iDBT-Pain) Using A Single Case Experimental Design

Emotion dysregulation frequently co-occurs with chronic pain, which in turn leads to heightened emotional and physical suffering. This cycle of association has prompted a recommendation for psychological treatment of chronic pain to target mechanisms for emotion regulation. The current trial addressed this need by investigating a new internet-delivered treatment incorporating emotional skills training from dialectical behavioural therapy (DBT). Using a single-case experimental design that is suited to heterogeneous populations and can demonstrate efficacy with a small sample, three participants with chronic pain were recruited. Participants received four weeks of online DBT skills training (iDBT-Pain intervention) which incorporated one-on-one sessions over Zoom and a web app. Results revealed compelling evidence for the intervention on the primary outcome of emotion dysregulation and were promising for the secondary outcome of pain intensity. Improvement was also identified on pre-and post-measures of depression, coping behaviours, sleep problems, wellbeing, and harm avoidance, indicating that the intervention may positively influence other factors related to chronic pain. Overall, the trial provides preliminary efficacy for the intervention to improve chronic pain. However, we recommend further investigation of the iDBT-Pain intervention, either in single case trials, which when conducted with scientific rigour may be aggregated to derive nomothetic conclusions, or in a group-comparison trial to compare with usual modes of treatment. Perspective: This trial advances understanding of emotion-focused treatment for chronic pain and provides evidence for a viable new technological treatment. Importantly, as an internet-delivered approach, the iDBT-Pain intervention is accessible to those with restricted mobility and remote communities where there are often limited psychological services for people with chronic pain. Trial registration: The trial was registered on the Australian and New Zealand Clinical Trials Registry (ACTRN12620000604909).