An R package for single-case randomization tests

Type I error rates and power of two randomization test procedures for the changing criterion design

Single-case experimental design (SCED) data can be analyzed following different approaches. One of the first historically proposed options is randomizations tests, benefiting from the inclusion of randomization in the design: a desirable methodological feature. Randomization tests have become more feasible with the availability of computational resources, and such tests have been proposed for all major types of SCEDs: multiple-baseline, reversal/withdrawal, alternating treatments, and changing criterion designs. The focus of the current text is on the last of these, given that they have not been the subject of any previous simulation study. Specifically, we estimate type I error rates and statistical power for two different randomization procedures applicable to changing criterion designs: the phase change moment randomization and the blocked alternating criterion randomization. We include different series lengths, number of phases, levels of autocorrelation, and random variability. The results suggest that type I error rates are generally controlled and that sufficient power can be achieved with as few as 28-30 measurements for independent data, although more measurements are needed in case of positive autocorrelation. The presence of a reversal to a previous criterion level is beneficial. R code is provided for carrying out randomization tests following the two randomization procedures.

Evaluating the impact of movement tracking feedback on engagement with home exercise programmes of children with cerebral palsy using a new therapy app: a protocol for a mixed-methods single-case experimental design with alternating treatments

INTRODUCTION

Children with cerebral palsy (CP) are prescribed home exercise programmes (HEPs) to increase the frequency of movement practice, yet adherence to HEPs can be low. This paper outlines the protocol for a single-case experimental design (SCED) with alternating treatments, using a new home therapy exercise application, Bootle Boot Camp (BBCamp), offered with and without movement tracking feedback. This study will explore the impact of feedback on engagement, movement quality, lower limb function and family experiences to help understand how technology-supported HEPs should be translated and the added value, if any, of movement tracking technology.

METHODS AND ANALYSIS

In this explanatory sequential mixed-methods study using a SCED, 16 children with CP (aged 6-12 years, Gross Motor Function Classification System levels I-II) will set lower limb goals and be prescribed an individualised HEP by their physiotherapist to complete using BBCamp on their home television equipped with a three-dimensional camera-computer system. Children will complete four weekly exercise sessions over 6 weeks. Children will be randomised to 1 of 16 alternating treatment schedules where BBCamp will provide or withhold feedback during the first 4 weeks. The version of BBCamp that results in the most therapeutic benefit will be continued for 2 final weeks. Goals will be re-evaluated and families interviewed. The primary outcome is adherence (proportion of prescribed exercise repetitions attempted) as a measure of behavioural engagement. Secondary outcomes are affective and cognitive engagement (smiley face ratings), exercise fidelity, lower limb function, goal achievement and participant experiences. SCED data will be analysed using visual and statistical methods. Quantitative and qualitative data will be integrated using joint displays.

ETHICS AND DISSEMINATION

Ethical approval was obtained from the Research Ethics Boards at Bloorview Research Institute and the University of Toronto. Results will be distributed through peer-reviewed journals and scientific conferences.

TRIAL REGISTRATION NUMBER

NCT05998239; pre-results.

The Permutation Distancing Test for dependent single-case observational AB-phase design data: A Monte Carlo simulation study

The Permutation Distancing Test (PDT) is a nonparametric test for evaluating treatment effects in dependent single-case observational design (SCOD) AB-phase data without linear trends. Monte Carlo methods were used to estimate the PDT power and type I error rate, and to compare them to those of the Single-Case Randomization Test (SCRT) assuming a randomly determined intervention point and the traditional permutation test assuming full exchangeability. Data were simulated without linear trends for five treatment effect levels (- 2, - 1, 0, 1, 2), five autocorrelation levels (0, .15, .30, .45, .60), and four observation number levels (30, 60, 90, 120). The power was calculated multiple times for all combinations of factor levels each generating 1000 replications. With 30 observations, the PDT showed sufficient power (≥ 80%) to detect medium treatment effects up to autocorrelation ≤ .45. Using 60 observations, the PDT showed sufficient power to detect medium treatment effects regardless of autocorrelation. With ≥ 90 observations, the PDT could also detect small treatment effects up to autocorrelation ≤ .30. With 30 observations, the type I error rate was 5-7%. With 60 observations and more, the type I error rate was ≤ 5% with autocorrelation < .60. The PDT outperformed the SCRT regarding power, particularly with a small number of observations. The PDT outperformed the traditional permutation test regarding type I error rate control, especially when autocorrelation increased. In conclusion, the PDT is a useful and promising nonparametric test to evaluate treatment effects in dependent SCOD AB-phase data without linear trends.

Brief individual experiential schema therapy in adult outpatients with cluster C personality disorders: Does it work?

This study investigated the effectiveness of brief individual experiential schema therapy (ST) in 12 adult outpatients with cluster C personality disorders (PD) using randomised multiple baseline design. Waitlist period was followed by five explorative sessions, 18 experiential ST sessions, two treatment follow-up (FU) booster sessions and a 6-month FU assessment. Overall well-being (ORS), behavioural treatment goals and negative core beliefs were assessed 60-91 times, global symptomatic distress (BSI) six times. PD severity (SCID-5-PD) was pre-post-analysed. Randomisation and non-parametric tests showed large significant effects (d = 1.08-2.38, r = .53-.66) on all outcomes at treatment-FU and 6-month FU assessment. This is the first study providing preliminary evidence of effectiveness of brief individual experiential ST for patients with cluster C PD, tentatively challenging the common tenet that long treatment duration is required. Due to limitations, replication is recommended.

Daily socket comfort in transtibial amputee with a vacuum-assisted suspension system: study protocol of a randomized, multicenter, double-blind multiple N-of-1 trial

BACKGROUND

The main aim of this paper is to present the feasibility of rigorously designed multiple N-of-1 design in prosthetics research. While research of adequate power and high quality is often lacking in rehabilitation, N-of-1 trials can offer a feasible alternative to randomized controlled group trials, both increasing design power at group level and allowing a rigorous, statistically confirmed evaluation of effectiveness at a single patient level. The paper presents a multiple N-of-1 trial protocol, which aim is to evaluate the effectiveness of Unity, a prosthetic add-on suspension system for amputees, on patient-reported comfort during daily activities (main outcome measure), prosthesis wearing time, perception of limb-prosthesis fitting and stump volume and functional walking parameters.

METHODS

Multicenter, randomized, prospective, double-blind multiple N-of-1 trial using an introduction/withdrawal design alternating Unity connected/disconnected phases of randomized length on twenty patients with unilateral transtibial amputation. The primary outcome measure is the Prosthetic Socket Comfort Score (SCS), a validated measure of comfort, administered daily by an phone app designed for the study. Secondary outcomes measures will be collected during the 50 days period of the N-of-1 trial: (1) by the same app, daily for patient-reported limb-prosthesis fitting, stump volume variation, and daily wearing time of the prosthesis; (2) by a pedometer for the number of steps per day; (3) by blind assessors in the rehabilitation center during adjustment visits for functional walking parameter (L-Test, 6-minute walk test), and by the patient for the QUEST, and ABC-S. Effectiveness of the Unity system regarding SCS and daily secondary outcome measures will be tested by randomization test. The secondary outcome measures assessed during visits in the rehabilitation center will be analyzed by Non Overlap of All pairs. An estimate of the effect on the amputee population will be generated by aggregating each individual clinical trial (N-of-1 trial) by Hierarchical Bayesian methods.

DISCUSSION

This study protocol was designed to answer the question "which device is best for THIS patient" and to conclude at a group level on the effectiveness of a new devic, using a Multiple N-of-1 trial, which is promising but underused in prosthetics research so far.

TRIAL REGISTRATION

N° ID-RCB 2020-A01309-30 Clintrial.gov : NCT04804150 - Retrospectively registered March 20th 2021.

Single-case experimental designs for child neurological rehabilitation and developmental disability research

Single-case experimental designs (SCEDs) are a group of methodologies of growing interest, aiming to test the effectiveness of an intervention at the single-participant level, using a rigorous and prospective methodology. SCEDs may promote flexibility on how we design research protocols and inform clinical decision-making, especially for personalized outcome measures, inclusion of families with challenging needs, measurement of children's progress in relation to parental implementation of interventions, and focus on personal goals. Design options for SCEDs are discussed in relation to an expected on/off effect of the intervention (e.g. school/environmental adaptation, assistive technology devices) or, alternatively, on an expected carry-on/maintenance of effects (interventions aiming to develop or restore a function). Randomization in multiple-baseline designs and 'power' calculations are explained. The most frequent reasons for not detecting an intervention effect in SCEDs are also presented, especially in relation to baseline length, trend, and instability. The use of SCEDs on the front and back ends of randomized controlled trials is discussed.

Proof of concept and feasibility of a blended physiotherapy intervention for persons with haemophilic arthropathy

BACKGROUND

Regular physiotherapy with a physiotherapist experienced in the field is not feasible for many patients with haemophilia. We, therefore, developed a blended physiotherapy intervention for persons with haemophilic arthropathy (HA) (e-Exercise HA), integrating face-to-face physiotherapy with a smartphone application.

AIM

The aim of the study was to determine proof of concept of e- Exercise HA and to evaluate feasibility.

METHODS

Proof of concept was evaluated by a single-case multiple baseline design. Physical activity (PA) was measured with an accelerometer during a baseline, intervention and post-intervention phase and analysed using visual inspection and a single case randomisation test. Changes in limitations in activities (Haemophilia Activities List [HAL]) and a General Perceived Effect (GPE) were evaluated between baseline (T0), post-intervention (T1) and 3 months post-intervention (T2) using Wilcoxson signed rank test. Feasibility was evaluated by the number of adverse events, attended sessions and open-ended questions.

RESULTS

Nine patients with HA (90% severe, median age 57.5 (quartiles 50.5-63.3) and median HJHS 32 (quartiles 22-36)) were included. PA increased in two patients. HAL increased mean 15 (SD 9) points (p = .001) at T1, and decrease to mean +8 points (SD 7) (p = .012) at T2 compared to T0. At T1 and T2 8/9 participants scored a GPE > 3. Median 5 (range 4-7) face-to-face sessions were attended and a median 8 out of 12 information modules were viewed. No intervention-related bleeds were reported.

CONCLUSION

A blended physiotherapy intervention is feasible for persons with HA and the first indication of the effectiveness of the intervention in decreasing limitations in activities was observed.

The randomized marker method for single-case randomization tests: Handling data missing at random and data missing not at random

Single-case experiments are frequently plagued by missing data problems. In a recent study, the randomized marker method was found to be valid and powerful for single-case randomization tests when the missing data were missing completely at random. However, in real-life experiments, it is difficult for researchers to ascertain the missing data mechanism. For analyzing such experiments, it is essential that the missing data handling method is valid and powerful for various missing data mechanisms. Hence, we examined the performance of the randomized marker method for data that are missing at random and data that are missing not at random. In addition, we compared the randomized marker method with multiple imputation, because the latter is often considered the gold standard among imputation techniques. To compare and evaluate these two methods under various simulation conditions, we calculated the type I error rate and statistical power in single-case randomization tests using these two methods of handling missing data and compared them to the type I error rate and statistical power using complete datasets. The results indicate that while multiple imputation presents an advantage in the presence of strongly correlated covariate data, the randomized marker method remains valid and results in sufficient statistical power for most of the missing data conditions simulated in this study.

Telepractice Treatment of Residual Rhotic Errors Using App-Based Biofeedback: A Pilot Study

PURPOSE

Although mobile apps are used extensively by speech-language pathologists, evidence for app-based treatments remains limited in quantity and quality. This study investigated the efficacy of app-based visual-acoustic biofeedback relative to nonbiofeedback treatment using a single-case randomization design. Because of COVID-19, all intervention was delivered via telepractice.

METHOD

Participants were four children aged 9-10 years with residual errors affecting American English /ɹ/. Using a randomization design, individual sessions were randomly assigned to feature practice with or without biofeedback, all delivered using the speech app Speech Therapist's App for /r/ Treatment. Progress was assessed using blinded listener ratings of word probes administered at baseline, posttreatment, and immediately before and after each treatment session.

RESULTS

All participants showed a clinically significant response to the overall treatment package, with effect sizes ranging from moderate to very large. One participant showed a significant advantage for biofeedback over nonbiofeedback treatment, although the order of treatment delivery poses a potential confound for interpretation in this case.

CONCLUSIONS

While larger scale studies are needed, these results suggest that app-based treatment for residual errors can be effective when delivered via telepractice. These results are compatible with previous findings in the motor learning literature regarding the importance of treatment dose and the timing of feedback conditions.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.18461576.

Acceptance and commitment therapy for individuals with depressive and anxiety symptoms following acquired brain injury: A non-concurrent multiple baseline design across four cases

Patients with acquired brain injury (ABI) often experience symptoms of anxiety and depression. Until now, evidence-based treatment is scarce. This study aimed to investigate the effectiveness of Acceptance and Commitment Therapy (ACT) for patients with ABI. To evaluate the effect of ACT for people with ABI, a non-concurrent multiple baseline design across four cases was used. Participants were randomly assigned to a baseline period, followed by treatment and then follow-up phases. Anxiety and depressive symptoms were repeatedly measured. During six measurement moments over a year, participants filled in questionnaires measuring anxiety, depression, stress, participation, quality of life, and ACT-related processes. Randomization tests and NAP scores were used to calculate the level of change across phases. Clinically significant change was defined with the Reliable Change Index. Three out of four participants showed medium to large decreases in anxiety and depressive symptoms (NAP = 0.85 till 0.99). Furthermore, participants showed improvements regarding stress, cognitive fusion, and quality of life. There were no improvements regarding psychological flexibility, value-driven behaviour, or social participation. This study shows that ACT is possibly an effective treatment option for people experiencing ABI-related anxiety and depression symptoms. Replication with single case or large scale group studies is needed to confirm these findings.

Changed dynamic symptom networks after a self-compassion training in patients with somatic symptom disorder: A multiple single-case pilot project

OBJECTIVE

Pre-to-post mean group differences of intermittently assessed generic outcome variables may not capture all relevant treatment-related changes in individual patients with somatic symptom disorder (SSD). Aim of this multiple single-case observational pilot project was to find out whether the Experience Sampling Method (ESM) and dynamic symptom networks may offer new opportunities in evaluating treatment outcomes for individual patients with SSD.

METHODS

Patients with SSD (N = 6 in study 1, N = 7 in study 2) received a self-compassion training in a tertiary care mental health expert center. Using a single-case pre-post treatment observational design, intensive longitudinal data were collected with ESM. A brief questionnaire was presented via the patient's smartphone three times per day for 16 weeks before, during and after the training in study 1, and for 5 weeks before and 5 weeks after the training in study 2. Eleven questions comprised somatic symptoms, functional disability, stress, self-compassion, and acceptance of affect; three personalized questions comprised self-chosen affects and an additional symptom.

RESULTS

Sufficient observations for means and network comparison were obtained for 11 and 10 patients, respectively. After the training, self-compassion was significantly increased in 10 patients, functional disability, stress and affect improved in 6 patients, and (although not a treatment goal) somatic symptoms decreased in 6 patients. Dynamic symptom networks significantly changed in 5 patients.

CONCLUSION

Patient-specific changes in means and dynamic symptom networks were observed after self-compassion training. In future clinical trials, single-case ESM may offer new opportunities to evaluate treatment outcomes in patients with SSD.

A practitioner's guide to conducting and analysing embedded randomized single-case experimental designs

Single-case experimental designs (SCEDs) are a class of experimental designs suited for answering research questions at an individual level. The main designs available in SCED research are phase designs, multiple baseline designs, alternation designs, and changing criterion designs. Embedded designs, also referred to as combination or hybrid designs, consist of one of these basic designs forms embedded in another design (e.g., a changing criterion design embedded in a multiple baseline design). Systematic reviews of SCEDs have repeatedly indicated that embedded designs are frequently used in applied SCED research. In spite of their popularity, specific recommendations on the conduct and analysis of embedded SCED designs are lacking to date. The purpose of the present article is therefore to provide guidance to applied researchers wishing to conduct embedded SCED designs in terms of design options, design requirements, randomization, and data analysis.

Comparing Biofeedback Types for Children With Residual /ɹ/ Errors in American English: A Single-Case Randomization Design

Purpose Research comparing different biofeedback types could lead to individualized treatments for those with residual speech errors. This study examines within-treatment response to ultrasound and visual-acoustic biofeedback, as well as generalization to untrained words, for errors affecting the American English rhotic /ɹ/. We investigated whether some children demonstrated greater improvement in /ɹ/ during ultrasound or visual-acoustic biofeedback. Each participant received both biofeedback types. Individual predictors of treatment response (i.e., age, auditory-perceptual skill, oral somatosensory skill, and growth mindset) were also explored. Method Seven children ages 9-16 years with residual rhotic errors participated in 10 treatment visits. Each visit consisted of two conditions: 45 min of ultrasound biofeedback and 45 min of visual-acoustic biofeedback. The order of biofeedback conditions was randomized within a single-case experimental design. Acquisition of /ɹ/ was evaluated through acoustic measurements (normalized F3-F2 difference) of selected nonbiofeedback productions during practice. Generalization of /ɹ/ was evaluated through acoustic measurements and perceptual ratings of pretreatment/posttreatment probes. Results Five participants demonstrated acquisition of practiced words during the combined treatment package. Three participants demonstrated a clinically significant degree of generalization to untreated words on posttreatment probes. Randomization tests indicated one participant demonstrated a significant advantage for visual-acoustic over ultrasound biofeedback. Participants' auditory-perceptual acuity on an /ɹ/-/w/ identification task was identified as a possible correlate of generalization following treatment. Conclusions Most participants did not demonstrate a statistically significant difference in acoustic productions between the ultrasound and visual-acoustic conditions, but one participant showed greater improvement in /ɹ/ during visual-acoustic biofeedback. Supplemental Material https://doi.org/10.23641/asha.14881101.

The Additive Effect of CBT Elements on the Video Game 'Mindlight' in Decreasing Anxiety Symptoms of Children with Autism Spectrum Disorder

The aim of the present study was to examine the additive effect of elements of cognitive behavioral therapy (CBT) on the video game Mindlight in decreasing anxiety of children with an autism spectrum disorder (ASD). A non-concurrent multiple baseline design with 8 children with ASD in the age of 8–12 was used. CBT did not have the hypothesized additive effect on Mindlight in decreasing anxiety of children with ASD. Instead, multiple participants already experienced a decrease in anxiety during the Mindlight sessions. Yet, several participants did experience a stabilization in anxiety at a low level during the CBT sessions. For now, it can be concluded that CBT does not have an additive effect on Mindlight.

Assessing Intervention Effects in the Presence of Missing Scores

Due to repeated observations of an outcome behavior in N-of-1 or single-case design (SCD) intervention studies, the occurrence of missing scores is inevitable in such studies. Approximately 21% of SCD articles published in five reputable journals between 2015 and 2019 exhibited evidence of missing scores. Missing rates varied by designs, with the highest rate (24%) found in multiple baseline/probe designs. Missing scores cause difficulties in data analysis. And inappropriate treatments of missing scores lead to consequences that threaten internal validity and weaken generalizability of intervention effects reported in SCD research. In this paper, we comprehensively review nine methods for treating missing SCD data: the available data method, six single imputations, and two model-based methods. The strengths, weaknesses, assumptions, and examples of these methods are summarized. The available data method and three single imputation methods are further demonstrated in assessing an intervention effect at the class and students’ levels. Assessment results are interpreted in terms of effect sizes, statistical significances, and visual analysis of data. Differences in results among the four methods are noted and discussed. The extensive review of problems caused by missing scores and possible treatments should empower researchers and practitioners to account for missing scores effectively and to support evidence-based interventions vigorously. The paper concludes with a discussion of contingencies for implementing the nine methods and practical strategies for managing missing scores in single-case intervention studies.

The Effectiveness of Hybrid Telepractice and In-Person Fluency Treatment for a School-Aged Child: A Case Study

Purpose

This case study assessed the effects of hybrid telepractice and in-person delivery of the Smooth Speech Treatment (Craig, 2010) for a school-aged child who stutters.

Method

The participant was an 8-year-old child who stuttered. Telepractice or in-person delivery were randomly assigned to treatment sessions in a single-subject AB design. The primary outcome measures were the percentage of syllables stuttered in conversation and reading probes taken at the beginning and end of each treatment session. Communication attitudes were measured before treatment began and at a posttreatment maintenance session. The pattern of treatment gains by delivery platform was also evaluated.

Results

Visual inspection of trends for outcome measures supported an association between the overall treatment package and a reduction in syllables stuttered. Pre- to post- treatment measures indicated an improvement in the child's attitudes toward stuttering. Within-session gains for telepractice sessions were more variable than in-person sessions.

Conclusions

These results indicate that fluency treatment delivered using a blend of telepractice and in-person delivery can be effective for a school-aged child who stutters. The findings align with prior studies showing the effectiveness of telepractice-delivered treatment for preschool children and adolescents. Replications of these results are needed with additional school-aged participants. Additional research is also warranted on factors that contribute to the variability of telepractice treatment effects for school-aged children.

Cognitive Behavioral Therapy for Treatment-Related Fatigue in Chronic Myeloid Leukemia Patients on Tyrosine Kinase Inhibitors: A Mixed-Method Study

Treatment-related fatigue significantly limits quality of life among chronic myeloid leukemia (CML) patients receiving tyrosine kinase inhibitors (TKIs), yet no interventions to reduce this symptom have been studied. We examined preliminary feasibility and efficacy of cognitive behavioral therapy for TKI treatment-related fatigue in patients with CML. We used a mixed methods convergent design and collected quantitative data through randomized single-case experiments. We included CML patients receiving TKIs and reporting severe fatigue. Within each participant, we compared CBT to a no-treatment baseline period. Fatigue severity was measured weekly with the Checklist Individual Strength. Fatigue scores were subjected to visual analyses and randomization tests for single-case experimental designs. We conducted qualitative interviews after study participation and focused on feasibility and efficacy of CBT. Visual inspection of line graphs indicated downward trends in the expected direction for fatigue in two of the four participants. The test statistics showed a decrease in fatigue severity for all participants but randomization tests did not reach statistical significance (overall p = 0.18). Participants reported qualitative improvements in level of functioning and coping with fatigue. CBT was considered feasible and acceptable for severely fatigued CML patients. Our study provided preliminary evidence for the feasibility and acceptability of CBT for severely fatigued CML patients receiving targeted therapy. We recommend further efficacy testing of this promising intervention in a pilot randomized controlled trial.

An unperceived acoustic stimulus decreases reaction time to visual information in a patient with cortical deafness

Responding to multiple stimuli of different modalities has been shown to reduce reaction time (RT), yet many different processes can potentially contribute to multisensory response enhancement. To investigate the neural circuits involved in voluntary response initiation, an acoustic stimulus of varying intensities (80, 105, or 120 dB) was presented during a visual RT task to a patient with profound bilateral cortical deafness and an intact auditory brainstem response. Despite being unable to consciously perceive sound, RT was reliably shortened (~100 ms) on trials where the unperceived acoustic stimulus was presented, confirming the presence of multisensory response enhancement. Although the exact locus of this enhancement is unclear, these results cannot be attributed to involvement of the auditory cortex. Thus, these data provide new and compelling evidence that activation from subcortical auditory processing circuits can contribute to other cortical or subcortical areas responsible for the initiation of a response, without the need for conscious perception.

Power of a randomization test in a single case multiple baseline AB design

A randomization test can be used to statistically test hypotheses in multiple baseline designs to complement the commonly used visual inspection analysis. A crossed factor simulation study was performed to investigate the power of a randomization test in an multiple baseline design. The results show that the degree of autocorrelation of the observations, the number of participants, the effect size, the overlap of possible start moments of the intervention between participants, the ratio of the number of measurements in the baseline- and intervention phase, a gradually emerging effect, and the number of measurements had strong main effects on the power. The two-way interactions between number of participants and effect size, and between the number of measurements and the number of start moments of the intervention also had a large effect. An online tool was developed to calculate the power of a multiple baseline design given several design characteristics.

Randomized Single-Case Experimental Designs in Healthcare Research: What, Why, and How?

Health problems are often idiosyncratic in nature and therefore require individualized diagnosis and treatment. In this paper, we show how single-case experimental designs (SCEDs) can meet the requirement to find and evaluate individually tailored treatments. We give a basic introduction to the methodology of SCEDs and provide an overview of the available design options. For each design, we show how an element of randomization can be incorporated to increase the internal and statistical conclusion validity and how the obtained data can be analyzed using visual tools, effect size measures, and randomization inference. We illustrate each design and data analysis technique using applied data sets from the healthcare literature.

Tailored Text Message Prompts to Increase Therapy Homework Adherence: A Single-Case Randomised Controlled Study

Background:Psychotherapy homework completion is associated with positive treatment outcomes, but many patients show low adherence to prescribed assignments. Whether text-message prompts are effective in increasing adherence to assignments is unknown.Aims:To evaluate whether tailored daily text-message prompts can increase homework adherence in a stress/anxiety treatment.Method:This study used a randomised controlled single-case alternating treatment design with parallel replication in seven participants. Participants received a five-week relaxation program for stress and anxiety with daily exercises. The intervention consisted of daily text messages tailored for each participant. Phases with or without text messages were randomly alternated over the study course. Randomisation tests were used to statistically analyse differences in mean number of completed relaxation exercises between phases.Results:There was a significant (combinedp= .018) effect of daily text messages on homework adherence across participants with weak to medium effect size improvements. No negative effects of daily text messages were identified.Conclusions:Tailored text messages can marginally improve adherence to assignments for patients in CBT. Further studies may investigate how text messages can be made relevant for more patients and whether text messages can be used to increase homework quality rather than quantity.

Assessing Consistency in Single-Case A-B-A-B Phase Designs

Previous research has introduced several effect size measures (ESMs) to quantify data aspects of single-case experimental designs (SCEDs): level, trend, variability, overlap, and immediacy. In the current article, we extend the existing literature by introducing two methods for quantifying consistency in single-case A-B-A-B phase designs. The first method assesses the consistency of data patterns across phases implementing the same condition, called CONsistency of DAta Patterns (CONDAP). The second measure assesses the consistency of the five other data aspects when changing from baseline to experimental phase, called CONsistency of the EFFects (CONEFF). We illustrate the calculation of both measures for four A-B-A-B phase designs from published literature and demonstrate how CONDAP and CONEFF can supplement visual analysis of SCED data. Finally, we discuss directions for future research.

Randomization tests as alternative analysis methods for behavior-analytic data

Randomization statistics offer alternatives to many of the statistical methods commonly used in behavior analysis and the psychological sciences, more generally. These methods are more flexible than conventional parametric and nonparametric statistical techniques in that they make no assumptions about the underlying distribution of outcome variables, are relatively robust when applied to small-n data sets, and are generally applicable to between-groups, within-subjects, mixed, and single-case research designs. In the present article, we first will provide a historical overview of randomization methods. Next, we will discuss the properties of randomization statistics that may make them particularly well suited for analysis of behavior-analytic data. We will introduce readers to the major assumptions that undergird randomization methods, as well as some practical and computational considerations for their application. Finally, we will demonstrate how randomization statistics may be calculated for mixed and single-case research designs. Throughout, we will direct readers toward resources that they may find useful in developing randomization tests for their own data.

Effectiveness and feasibility of Socratic feedback to increase awareness of deficits in patients with acquired brain injury: Four single-case experimental design (SCED) studies

OBJECTIVE

To investigate the effectiveness and feasibility of a Socratic feedback programme to improve awareness of deficits in patients with acquired brain injury (ABI).

SETTING

Rehabilitation centre.

PARTICIPANTS

Four patients with ABI with awareness problems.

DESIGN

A series of single-case experimental design studies with random intervention starting points (A-B + maintenance design).

MAIN MEASURES

Rate of trainer-feedback and self-control behaviour on everyday tasks, patient competency rating scale (PCRS), self-regulating skills interview (SRSI), hospital anxiety and depression scale.

RESULTS

All patients needed less trainer feedback, the change was significant in 3 out of 4. One patient increased in overt self-corrective behaviour. SRSI performance increased in all patients (medium to strong effect size), and PCRS performance increased in two patients (medium and strong effect size). Mood and anxiety levels were elevated in one patient at the beginning of the training and decreased to normal levels at the end of the training. The feasibility of the programme was scored 9 out of 10.

CONCLUSIONS

The Socratic feedback method is a promising intervention for improving awareness of deficits in patients with ABI. Controlled studies with larger populations are needed to draw more solid conclusions about the effect of this method.

Randomization Tests for Single Case Designs with Rapidly Alternating Conditions: An Analysis of p-Values from Published Experiments

Two common barriers to applying statistical tests to single-case experiments are that single-case data often violate the assumptions of parametric tests and that random assignment is inconsistent with the logic of single-case design. However, in the case of randomization tests applied to single-case experiments with rapidly alternating conditions, neither the statistical assumptions nor the logic of the designs are violated. To examine the utility of randomization tests for single-case data, we collected a sample of published articles including alternating treatments or multielement designs with random or semi-random condition sequences. We extracted data from graphs and used randomization tests to estimate the probability of obtaining results at least as extreme as the results in the experiment by chance alone (i.e., p-value). We compared the distribution of p-values from experimental comparisons that did and did not indicate a functional relation based on visual analysis and evaluated agreement between visual and statistical analysis at several levels of α. Results showed different means, shapes, and spreads for the p-value distributions and substantial agreement between visual and statistical analysis when α = .05, with lower agreement when α was adjusted to preserve family-wise error at .05. Questions remain, however, on the appropriate application and interpretation of randomization tests for single-case designs.

Use of the experience sampling method in the context of clinical trials

Objective

The experience sampling method (ESM) is a structured diary technique to appraise subjective experiences in daily life. It is applied in psychiatric patients, as well as in patients with somatic illness. Despite the potential of ESM assessment, the improved logistics and its increased administration in research, its use in clinical trials remains limited. This paper introduces ESM for clinical trials in psychiatry and beyond.

Methods

ESM is an ecologically valid method that yields a comprehensive view of an individual's daily life. It allows the assessment of various constructs (eg, quality of life, psychopathology) and psychological mechanisms (eg, stress-sensitivity, coping). These constructs are difficult to assess using cross-sectional questionnaires. ESM can be applied in treatment monitoring, as an ecological momentary intervention, in clinical trials, or in single case clinical trials. Technological advances (eg, smartphone applications) make its implementation easier.

Results

Advantages of ESM are highlighted and disadvantages are discussed. Furthermore, the ecological nature of ESM data and its consequences are explored, including the potential pitfalls of ambiguously formulated research questions and the specificities of ESM in statistical analyses. The last section focuses on ESM in relation to clinical trials and discusses its future use in optimising clinical decision-making.

Conclusions

ESM can be a valuable asset in clinical trial research and should be used more often to study the benefits of treatment in psychiatry and somatic health.

One by One: Accumulating Evidence by using Meta-Analytical Procedures for Single-Case Experiments

This paper presents a unilevel and multilevel approach for the analysis and meta-analysis of single-case experiments (SCEs). We propose a definition of SCEs and derive the specific features of SCEs’ data that have to be taken into account when analysing and meta-analysing SCEs. We discuss multilevel models of increasing complexity and propose alternative and complementary techniques based on probability combining and randomisation test wrapping. The proposed techniques are demonstrated with real-life data and corresponding R code.

Research Design Options for Intervention Studies

PURPOSE

To review research designs for rehabilitation.

SUMMARY OF KEY POINTS

Single-case, observational, and qualitative designs are highlighted in terms of recent advances and ability to answer important scientific questions about rehabilitation.

STATEMENT OF CONCLUSIONS

Single-case, observational, and qualitative designs can be conducted in a systematic and rigorous manner that provides important information that cannot be acquired using more common designs, such as randomized controlled trials. These less commonly used designs may be more feasible and effective in answering many research questions in the field of rehabilitation.

RECOMMENDATIONS FOR CLINICAL PRACTICE

Researchers should consider these designs when selecting the optimal design to answer their research questions. We should improve education about the advantages and disadvantages of existing research designs to enable more critical analysis of the scientific literature we read and review to avoid undervaluing studies not within more commonly used categories.

Efficacy of Visual-Acoustic Biofeedback Intervention for Residual Rhotic Errors: A Single-Subject Randomization Study

PURPOSE

This study documented the efficacy of visual-acoustic biofeedback intervention for residual rhotic errors, relative to a comparison condition involving traditional articulatory treatment. All participants received both treatments in a single-subject experimental design featuring alternating treatments with blocked randomization of sessions to treatment conditions.

METHOD

Seven child and adolescent participants received 20 half-hour sessions of individual treatment over 10 weeks. Within each week, sessions were randomly assigned to feature traditional or biofeedback intervention. Perceptual accuracy of rhotic production was assessed in a blinded, randomized fashion. Each participant's response to the combined treatment package was evaluated by using effect sizes and visual inspection. Differences in the magnitude of response to traditional versus biofeedback intervention were measured with individual randomization tests.

RESULTS

Four of 7 participants demonstrated a clinically meaningful response to the combined treatment package. Three of 7 participants showed a statistically significant difference between treatment conditions. In all 3 cases, the magnitude of within-session gains associated with biofeedback exceeded the gains associated with traditional treatment.

CONCLUSIONS

These results suggest that the inclusion of visual-acoustic biofeedback can enhance the efficacy of intervention for some individuals with residual rhotic errors. Further research is needed to understand which participants represent better or poorer candidates for biofeedback treatment.

Mood disturbances during combined oral contraceptive use and the effect of androgen supplementation. Results of a double-blind, placebo-controlled, single-case alternation design pilot study

OBJECTIVES

To evaluate the effect of androgen supplementation in healthy combined oral contraceptive (COC) users who experience mood disturbances during COC-use only.

METHODS

Six women with mood disturbances during COC-use only, received COC with co-treatment of 50 mg dehydroepiandrosterone (DHEA) during three cycles and placebo during another three cycles in an individualized random order. Daily mood rating was measured by a single item: 'In what kind of mood have you been in the past 24 h?' The results were analysed using a randomisation test for single-case experimental designs.

RESULTS

The p values for the alternation design randomisation tests on the raw data of the six healthy individuals varied between 0.21 and 1, indicating that the average daily mood ratings of the active treatment DHEA are not statistically significantly larger than the average daily mood ratings of placebo. The combined p value of the subjects using a DRSP-containing pill was 0.97, and of the subjects using an LNG-containing pill was 0.65, indicating no statistically significant treatment effect for any of the pill types.

CONCLUSIONS

In this single-case alternation design study, concomitant treatment with DHEA for intermittent periods of 4 weeks did not result in improvement of mood disturbances related to COC-use, but had also no side-effects.

Acquisition, retention, and generalization of rhotics with and without ultrasound visual feedback

PURPOSE

The purpose was to provide a preliminary within-participant comparison of speech therapy with and without exposure to ultrasound visual feedback for postvocalic rhotics (/r/- colored vowels). Effects of the two treatments on acquisition, retention, and generalization were explored. It was hypothesized that treatment with ultrasound would facilitate acquisition but hinder retention and generalization.

METHODS

A single subject randomized block design was replicated across four American English-speaking participants ages 7-9 years. Each participant was trained on postvocalic /r/. Each week for seven weeks, one session with ultrasound visual feedback and one session with no ultrasound were randomly ordered. A Training Probe and Generalization Probe were used to measure acquisition within each session as well as retention and generalization between two consecutive sessions. Graphical displays of the data, effect size calculation, and statistical results from a randomization test were used to analyze the results.

RESULTS

Two participants showed essentially no evidence of acquisition, retention or generalization of rhotics (<5%). Of the two who showed evidence of acquisition, one participant showed a significant advantage and large effect size for ultrasound sessions over no ultrasound sessions in acquisition of rhotics. However, no participants showed differences between treatment conditions in generalization or retention of rhotics.

CONCLUSION

For some children, acquisition may be facilitated by ultrasound visual feedback. Ultrasound visual feedback neither inhibited nor facilitated retention or generalization of rhotics. As a whole, the 14 treatment sessions (7 with ultrasound and 7 without) were effective for 2 of the 4 participants when comparing pre/post generalization scores. Future studies should evaluate the effectiveness of ultrasound visual feedback given a larger dose and differing age groups.

Probability and Visual Aids for Assessing Intervention Effectiveness in Single-Case Designs

Single-case data analysis still relies heavily on visual inspection, and, at the same time, it is not clear to what extent the results of different quantitative procedures converge in identifying an intervention effect and its magnitude when applied to the same data; this is the type of evidence provided here for two procedures. One of the procedures, included due to the importance of providing objective criteria to visual analysts, is a visual aid fitting and projecting split-middle trend while taking into account data variability. The other procedure converts several different metrics into probabilities making their results comparable. In the present study, we expore to what extend these two procedures coincide in the magnitude of intervention effect taking place in a set of studies stemming from a recent meta-analysis. The procedures concur to a greater extent with the values of the indices computed and with each other and, to a lesser extent, with our own visual analysis. For distinguishing smaller from larger effects, the probability-based approach seems somewhat better suited. Moreover, the results of the field test suggest that the latter is a reasonably good mechanism for translating different metrics into similar labels. User friendly R code is provided for promoting the use of the visual aid, together with a quantification based on nonoverlap and the label provided by the probability approach.

Permutation Tests in the Educational and Behavioral Sciences

Over the past two decades, permutation tests (PTs) have received much attention in the educational and behavioral sciences. The aim of this article is to review the theoretical developments of PTs, the active areas in the educational and behavioral research using PTs, and the types of analysis under which PTs have been applied. We obtained 224 published articles, which included 141 theoretical articles and 83 application articles. After scrutinizing each article, we are happy to see that (1) some researchers began to advocate introducing PTs into basic statistics training; (2) computing load for PTs may be reduced dramatically by some intelligent algorithms; (3) PTs began to be applied in new areas such as studies on the relationship between brain and behavior and the relationship between gene and behavior; (4) besides simple types of analysis such as independent two-group comparison, PTs can also be carried out under more complex situations such as multivariate analysis. However, we should also notice that PTs are still mostly used for simple analyses (e.g., randomness analysis).

Single-case experimental designs: reflections on conduct and analysis

In this editorial discussion we reflect on the issues addressed by, and arising from, the papers in this special issue on Single-Case Experimental Design (SCED) study methodology. We identify areas of consensus and disagreement regarding the conduct and analysis of SCED studies. Despite the long history of application of SCEDs in studies of interventions in clinical and educational settings, the field is still developing. There is an emerging consensus on methodological quality criteria for many aspects of SCEDs, but disagreement on what are the most appropriate methods of SCED data analysis. Our aim is to stimulate this ongoing debate and highlight issues requiring further attention from applied researchers and methodologists. In addition we offer tentative criteria to support decision-making in relation to the selection of analytical techniques in SCED studies. Finally, we stress that large-scale interdisciplinary collaborations, such as the current Special Issue, are necessary if SCEDs are going to play a significant role in the development of the evidence base for clinical practice.

Dissociable mechanisms of speed-accuracy tradeoff during visual perceptual learning are revealed by a hierarchical drift-diffusion model

Two phenomena are commonly observed in decision-making. First, there is a speed-accuracy tradeoff (SAT) such that decisions are slower and more accurate when instructions emphasize accuracy over speed, and vice versa. Second, decision performance improves with practice, as a task is learnt. The SAT and learning effects have been explained under a well-established evidence-accumulation framework for decision-making, which suggests that evidence supporting each choice is accumulated over time, and a decision is committed to when the accumulated evidence reaches a decision boundary. This framework suggests that changing the decision boundary creates the tradeoff between decision speed and accuracy, while increasing the rate of accumulation leads to more accurate and faster decisions after learning. However, recent studies challenged the view that SAT and learning are associated with changes in distinct, single decision parameters. Further, the influence of speed-accuracy instructions over the course of learning remains largely unknown. Here, we used a hierarchical drift-diffusion model to examine the SAT during learning of a coherent motion discrimination task across multiple training sessions, and a transfer test session. The influence of speed-accuracy instructions was robust over training and generalized across untrained stimulus features. Emphasizing decision accuracy rather than speed was associated with increased boundary separation, drift rate and non-decision time at the beginning of training. However, after training, an emphasis on decision accuracy was only associated with increased boundary separation. In addition, faster and more accurate decisions after learning were due to a gradual decrease in boundary separation and an increase in drift rate. The results suggest that speed-accuracy instructions and learning differentially shape decision-making processes at different time scales.

A case series exploring possible predictors and mechanisms of change in hearing voices groups

OBJECTIVES

Hearing voices groups (HVGs) are increasingly common in National Health Services and are often preferred to individual therapy by both service users and providers. Whilst a range of approaches exist, mixed results have been found and only cognitive behaviour therapy (CBT) was supported by well-controlled trials. This study aimed to explore possible predictors and mechanisms of change in a seven-session CBT group for voice hearers.

DESIGN

An exploratory case series design was used. Method. Fifteen outpatients with a diagnosis of schizophrenia or schizoaffective disorder completed a HVG and carried out weekly measures of distress, negative beliefs about voices, self-esteem, effective coping strategies, and activity levels. Visual inspection and quantitative rules were used to group participants with similar results and cross-correlations and t tests were used to verify key findings.

RESULTS

Several pathways emerged across therapy. Despite measurement frequency, changes on different outcomes tended to occur simultaneously, making conclusions about mechanisms difficult. However, changes in beliefs about voice malevolence and omnipotence correlated most frequently with changes in distress. Visual analysis indicated 53% of participants improved on a measured outcome but satisfaction scores were higher, with 93% feeling the group helped them deal with their problems more effectively. Clients especially valued the chance to meet similar others.

CONCLUSIONS

The results suggest HVGs are valued by clients, regardless of their background or symptoms. HVGs should emphasize testing negative beliefs about voices and allow space for supportive discussions between clients. Recommendations for future research are discussed, including consideration of benefits not detected by outcome measures.