1
|
Manolov R, Tanious R. Type I error rates and power of two randomization test procedures for the changing criterion design. Behav Res Methods 2024; 56:3637-3657. [PMID: 38082114 PMCID: PMC11133045 DOI: 10.3758/s13428-023-02303-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/21/2023] [Indexed: 05/30/2024]
Abstract
Single-case experimental design (SCED) data can be analyzed following different approaches. One of the first historically proposed options is randomizations tests, benefiting from the inclusion of randomization in the design: a desirable methodological feature. Randomization tests have become more feasible with the availability of computational resources, and such tests have been proposed for all major types of SCEDs: multiple-baseline, reversal/withdrawal, alternating treatments, and changing criterion designs. The focus of the current text is on the last of these, given that they have not been the subject of any previous simulation study. Specifically, we estimate type I error rates and statistical power for two different randomization procedures applicable to changing criterion designs: the phase change moment randomization and the blocked alternating criterion randomization. We include different series lengths, number of phases, levels of autocorrelation, and random variability. The results suggest that type I error rates are generally controlled and that sufficient power can be achieved with as few as 28-30 measurements for independent data, although more measurements are needed in case of positive autocorrelation. The presence of a reversal to a previous criterion level is beneficial. R code is provided for carrying out randomization tests following the two randomization procedures.
Collapse
Affiliation(s)
- Rumen Manolov
- Department of Social Psychology and Quantitative Psychology, Faculty of Psychology, University of Barcelona, Passeig de la Vall d'Hebron 171, 08035, Barcelona, Spain.
| | - René Tanious
- Faculty of Psychology and Neuroscience, Maastricht University, Universiteitssingel 40, 6229 ER, Maastricht, the Netherlands
| |
Collapse
|
2
|
Manolov R, Onghena P. Testing delayed, gradual, and temporary treatment effects in randomized single-case experiments: A general response function framework. Behav Res Methods 2024; 56:3915-3936. [PMID: 37749426 PMCID: PMC11133040 DOI: 10.3758/s13428-023-02230-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/29/2023] [Indexed: 09/27/2023]
Abstract
Randomization tests represent a class of significance tests to assess the statistical significance of treatment effects in randomized single-case experiments. Most applications of single-case randomization tests concern simple treatment effects: immediate, abrupt, and permanent changes in the level of the outcome variable. However, researchers are confronted with delayed, gradual, and temporary treatment effects; in general, with "response functions" that are markedly different from single-step functions. We here introduce a general framework that allows specifying a test statistic for a randomization test based on predicted response functions that is sensitive to a wide variety of data patterns beyond immediate and sustained changes in level: different latencies (degrees of delay) of effect, abrupt versus gradual effects, and different durations of the effect (permanent or temporary). There may be reasonable expectations regarding the kind of effect (abrupt or gradual), entailing a different focal data feature (e.g., level or slope). However, the exact amount of latency and the exact duration of a temporary effect may not be known a priori, justifying an exploratory approach studying the effect of specifying different latencies or delayed effects and different durations for temporary effects. We provide illustrations of the proposal with real data, and we present a user-friendly freely available web application implementing it.
Collapse
Affiliation(s)
- Rumen Manolov
- Department of Social Psychology and Quantitative Psychology, Faculty of Psychology, University of Barcelona, Passeig de la Vall d'Hebron 171, 08035, Barcelona, Spain.
| | - Patrick Onghena
- Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research Group, KU Leuven, Tiensestraat 102, 3000, Leuven, Belgium
| |
Collapse
|
3
|
Aydin O. Rise of single-case experimental designs: A historical overview of the necessity of single-case methodology. Neuropsychol Rehabil 2024; 34:301-334. [PMID: 36811612 DOI: 10.1080/09602011.2023.2181191] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2022] [Accepted: 02/13/2023] [Indexed: 02/24/2023]
Abstract
ABSTRACTWindelband ([1894]1980) advocated that two approaches are used for accumulating scientific knowledge. The first is the idiographic approach that derives knowledge from a single unit, and the second is the nomothetic approach that accumulates knowledge of a group. Given these two approaches, the former matches case studies while the latter is more appropriate with experimental group studies. Scientists have criticized both methodologies for their various limitations. Later, the single-case methodology emerged as an alternative that potentially allays these limitations. In this context, this narrative review aims to describe the historical roots of single-case experimental designs (SCEDs) that have emerged to eliminate the tension of nomothetic and idiographic approaches over time. First, the review focuses on the emergence of SCEDs. Second, the strengths and challenges of SCEDs are reviewed, including those to address the limitations of group experimental and case studies. Third, the use and analyses of SCEDs are outlined, considering their current status. Fourth, this narrative review continues to delineate the dissemination of SCEDs in the modern scientific world. As a result, SCEDs can be evaluated as a method that has the potential to overcome the issues encountered in case description and group experimental research. Thus, that helps accumulate nomothetic and idiographic knowledge in determining evidence-based practices.
Collapse
Affiliation(s)
- Orhan Aydin
- Faculty of Education, Erzincan Binali Yildirim University, Erzincan, Türkiye
| |
Collapse
|
4
|
Manolov R. Does the choice of a linear trend-assessment technique matter in the context of single-case data? Behav Res Methods 2023; 55:4200-4221. [PMID: 36622560 DOI: 10.3758/s13428-022-02013-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/22/2022] [Indexed: 01/10/2023]
Abstract
Trend is one of the data aspects that is an object of assessment in the context of single-case experimental designs. This assessment can be performed both visually and quantitatively. Given that trend, just like other relevant data features such as level, immediacy, or overlap does not have a single operative definition, a comparison among the existing alternatives is necessary. Previous studies have included illustrations of differences between trend-line fitting techniques using real data. In the current study, I carry out a simulation to study the degree to which different trend-line fitting techniques lead to different degrees of bias, mean square error, and statistical power for a variety of quantifications that entail trend lines. The simulation involves generating both continuous and count data, for several phase lengths, degrees of autocorrelation, and effect sizes (change in level and change in slope). The results suggest that, in general, ordinary least squares estimation performs well in terms of relative bias and mean square error. Especially, a quantification of slope change is associated with better statistical results than quantifying an average difference between conditions on the basis of a projected baseline trend. In contrast, the performance of the split-middle (bisplit) technique is less than optimal.
Collapse
Affiliation(s)
- Rumen Manolov
- Department of Social Psychology and Quantitative Psychology, Faculty of Psychology, University of Barcelona, Passeig de la Vall d'Hebron 171, 08035, Barcelona, Spain.
| |
Collapse
|
5
|
Abstract
To strengthen the scientific credibility arguments for single-case intervention studies, randomization design-and-analysis methods have been developed for the multiple-baseline, ABAB, and alternating treatment designs, including options for preplanned designs, wherein the series and phase lengths are established prior to gathering data, as well as options for response-guided designs, wherein ongoing visual analyses guide decisions about when to intervene. Our purpose here is to develop randomization methods for another class of single-case design, the changing criterion design. We first illustrate randomization design-and-analysis methods for preplanned changing criterion designs and then develop and illustrate methods for response-guided changing criterion designs. We discuss the limitations associated with the randomization methods and the validity of the corresponding intervention-effect inferences.
Collapse
|
6
|
Maric M, Schumacher L, Van den Noortgate W, Bettelli L, Engelbertink W, Stikkelbroek Y. A Multilevel Meta-analysis of Single-Case Research on Interventions for Internalizing Disorders in Children and Adolescents. Clin Child Fam Psychol Rev 2023; 26:416-429. [PMID: 37010669 PMCID: PMC10123043 DOI: 10.1007/s10567-023-00432-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/17/2023] [Indexed: 04/04/2023]
Abstract
The effectiveness of interventions for internalizing disorders in children and adolescents was studied using a review and meta-analysis of published single-case research. Databases and other resources were searched for quantitative single-case studies in youth with anxiety, depressive, and posttraumatic stress disorders. Raw data from individual cases were aggregated and analyzed by means of multilevel meta-analytic models. Outcome variables were symptom severity assessed across baseline and treatment phases of the studies, and diagnostic status at post- and follow-up treatment. Single-case studies were rated for quality. We identified 71 studies including 321 cases (Mage = 10.66 years; 55% female). The mean quality of the studies was rated as below average, although there were considerable differences between the studies. Overall, positive within-person changes during the treatment phase in comparison to the baseline phase were found. In addition, positive changes in the diagnostic status were observed at post- and follow-up treatment. Yet high variability in treatment effects was found between cases and studies. This meta-analysis harvests the knowledge from published single-case research in youth-internalizing disorders and illustrates how within-person information from single-case studies can be summarized to explore the generalizability of the results from this type of research. The results emphasize the importance of keeping account of individual variability in providing and investigating youth interventions.
Collapse
Affiliation(s)
- Marija Maric
- Developmental Psychology, University of Amsterdam, Amsterdam, The Netherlands.
- Research Institute of Child Development and Education, University of Amsterdam, Amsterdam, The Netherlands.
| | - Lea Schumacher
- Medical Psychology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Wim Van den Noortgate
- Faculty of Psychology and Educational Sciences & Itec, an Imec Research Group, KU Leuven, Leuven, Belgium
| | - Linda Bettelli
- Developmental Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Wies Engelbertink
- Developmental Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Yvonne Stikkelbroek
- Child and Adolescent Studies, Utrecht University, Utrecht, The Netherlands
- Depression Expert Center for Youth, Mental Health Care Oost-Brabant, Boekel, The Netherlands
| |
Collapse
|
7
|
Kratochwill TR, Horner RH, Levin JR, Machalicek W, Ferron J, Johnson A. Single-case intervention research design standards: Additional proposed upgrades and future directions. J Sch Psychol 2023; 97:192-216. [PMID: 36914365 DOI: 10.1016/j.jsp.2022.12.002] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2021] [Revised: 05/28/2022] [Accepted: 12/23/2022] [Indexed: 02/25/2023]
Abstract
Single-case intervention research design standards have evolved considerably over the past decade. These standards serve the dual role of assisting in single-case design (SCD) intervention research methodology and as guidelines for literature syntheses within a particular research domain. In a recent article (Kratochwill et al., 2021), we argued for a need to clarify key features of these standards. In this article we offer additional recommendations for SCD research and synthesis standards that have been either underdeveloped or missing in the conduct of research and in literature syntheses. Our recommendations are organized into three categories: expanding design standards, expanding evidence standards, and expanding the applications and consistency of SCDs. The recommendations we advance are for consideration for future standards, research design training, and they are especially important to guide the reporting of SCD intervention investigations as they enter the literature-synthesis phase of evidence-based practice initiatives.
Collapse
Affiliation(s)
| | | | | | | | - John Ferron
- University of South Florida,United States of America
| | - Austin Johnson
- University of California, Riverside, United States of America
| |
Collapse
|
8
|
de Vries FS, van Dongen RTM, Bertens D. Pain education and pain management skills in virtual reality in the treatment of chronic low back pain: A multiple baseline single-case experimental design. Behav Res Ther 2023; 162:104257. [PMID: 36731183 DOI: 10.1016/j.brat.2023.104257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Revised: 01/14/2023] [Accepted: 01/17/2023] [Indexed: 01/19/2023]
Abstract
Chronic lower back pain is a major health problem and current treatments do not always lead to adequate pain control. Virtual reality (VR) is an upcoming technology that has shown to be effective in reducing acute pain. However, the value of VR in reducing chronic pain is still unknown. Therefore, the current study focuses on the effects of a recently developed VR application 'Reducept' using a multiple baseline single-case experimental design in 8 patients (N = 8). Reducept is a VR-training program aiming to improve pain management skills and providing pain education in patients with (chronic lower) back pain. Results based on visual and statistical analyses indicated that Reducept has the potential to reduce chronic lower back pain, although its clinical relevance was small. This study is one of the first that focuses on the possible effects of Reducept using sophisticated visual and statistical analyses. Our study shows a detailed overview of individual changes in pain intensity over time. Further research is necessary to investigate the working mechanism of Reducept and its impact on chronic pain conditions.
Collapse
Affiliation(s)
- Froukje S de Vries
- Canisius-Wilhelmina Medical Center, Weg Door Jonkerbos 100, 6532 SZ, Nijmegen, the Netherlands.
| | - Robert T M van Dongen
- Canisius-Wilhelmina Medical Center, Weg Door Jonkerbos 100, 6532 SZ, Nijmegen, the Netherlands; Radboud University Medical Center, Geert Grooteplein-Zuid 10, 6525GA, Nijmegen, the Netherlands
| | - Dirk Bertens
- Klimmendaal Rehabilitation Specialists, Heijenoordseweg 5, 6813 GG, Arnhem, the Netherlands; Radboud University, Donders Institute for Brain, Cognition and Behaviour, Thomas van Aquinostraat 4, 6525 GD, Nijmegen, the Netherlands
| |
Collapse
|
9
|
Manolov R, Onghena P. Defining and assessing immediacy in single-case experimental designs. J Exp Anal Behav 2022; 118:462-492. [PMID: 36106573 PMCID: PMC9825864 DOI: 10.1002/jeab.799] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 08/24/2022] [Accepted: 08/28/2022] [Indexed: 01/11/2023]
Abstract
Immediacy is one of six data aspects (alongside level, trend, variability, overlap, and consistency) that has to be accounted for when visually analyzing single-case data. Given that it is one of the aspects that has received considerably less attention than other data aspects, the current text offers a review of the proposed conceptual definitions of immediacy (i.e., what it refers to) and also of the suggested operational definitions (i.e., how exactly is it assessed and/or quantified). Provided that a variety of conceptual and operational definitions is identified, we propose following a sensitivity analysis using a randomization test for assessing immediate effects in single-case experimental designs, by identifying when changes were most clear. In such a sensitivity analysis, the immediate effects are tested for multiple possible intervention points and for different possible operational definitions. Robust immediate effects can be detected if the results for the different operational definitions converge.
Collapse
Affiliation(s)
- Rumen Manolov
- Department of Social Psychology and Quantitative Psychology, Faculty of PsychologyUniversity of Barcelona
| | - Patrick Onghena
- Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research GroupKU Leuven – University of LeuvenLeuvenBelgium
| |
Collapse
|
10
|
Tanious R, Onghena P. Applied hybrid single-case experiments published between 2016 and 2020: A systematic review. METHODOLOGICAL INNOVATIONS 2022. [DOI: 10.1177/20597991221077910] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Single-case experimental designs (SCEDs) are frequently used research designs in psychology, (special) education, and related fields. Hybrid designs are formed by combining two or more of the basic SCED forms (i.e. phase designs, alternation designs, multiple baseline designs, and changing criterion designs). Hybrid designs have the potential to tackle complex research questions and increase internal validity, but relatively little is known about their use in actual research practice. Therefore, we systematically reviewed SCED hybrid designs published between 2016 and 2020. The systematic review of 67 studies indicates that a hybrid of phase designs and multiple baseline designs is most popular. Hybrid designs are most frequently analyzed by means of visual analysis paired with descriptive statistics. Randomization in the study design is common only for one particular kind of hybrid design. Examples of hybrid studies reveal that these designs are particularly popular in educational research. We compare some of the results of the systematic review to those obtained by Hammond and Gast, Shadish and Sullivan, and Tanious and Onghena. Finally, we discuss the results of the present systematic review in light of the need for specific guidelines for hybrid designs, including analytical methods, design specific randomization and reporting, and the need for terminological clarification.
Collapse
Affiliation(s)
- René Tanious
- Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research Group, KU Leuven, Leuven, Belgium
| | - Patrick Onghena
- Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research Group, KU Leuven, Leuven, Belgium
| |
Collapse
|
11
|
Tanious R. Additional randomization test procedures for the changing criterion design. BEHAVIORAL INTERVENTIONS 2022. [DOI: 10.1002/bin.1875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Affiliation(s)
- René Tanious
- Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research Group, KU Leuven Leuven Belgium
| |
Collapse
|
12
|
MacKinnon DP, Smyth HL, Somers J, Ho E, Norget J, Miočević M. A Randomization Permutation Test for Single Subject Mediation. Eval Health Prof 2022; 45:54-65. [PMID: 35209736 PMCID: PMC8995141 DOI: 10.1177/01632787211070811] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
In response to the importance of individual-level effects, the purpose of this paper is to describe the new randomization permutation (RP) test for a mediation mechanism for a single subject. We extend seminal work on permutation tests for individual-level data by proposing a test for mediation for one person. The method requires random assignment to the levels of the treatment variable at each measurement occasion, and repeated measures of the mediator and outcome from one subject. If several assumptions are met, the process by which a treatment changes an outcome can be statistically evaluated for a single subject, using the permutation mediation test method and the permutation confidence interval method for residuals. A simulation study evaluated the statistical properties of the new method suggesting that at least eight repeated measures are needed to control Type I error rates and larger sample sizes are needed for power approaching .8 even for large effects. The RP mediation test is a promising method for elucidating intraindividual processes of change that may inform personalized medicine and tailoring of process-based treatments for one subject.
Collapse
Affiliation(s)
- David P MacKinnon
- Department of Psychology, Arizona State University7864, Tempe, AZ, USA
| | - Heather L Smyth
- Department of Psychology, Arizona State University7864, Tempe, AZ, USA
| | - Jennifer Somers
- Department of Psychology, Arizona State University7864, Tempe, AZ, USA
| | - Emily Ho
- Department of Medical Social Sciences, Northwestern University, 7864Chicago, IL
| | - Julia Norget
- Psychological Methods and Evaluation, Bielefeld University9167, Bielefeld, Germany
| | - Milica Miočević
- Department of Psychology, McGill University5620, Montreal, QC, Canada
| |
Collapse
|
13
|
Tanious R, Manolov R. A practitioner's guide to conducting and analysing embedded randomized single-case experimental designs. Neuropsychol Rehabil 2022; 33:613-645. [PMID: 35179088 DOI: 10.1080/09602011.2022.2035774] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
Single-case experimental designs (SCEDs) are a class of experimental designs suited for answering research questions at an individual level. The main designs available in SCED research are phase designs, multiple baseline designs, alternation designs, and changing criterion designs. Embedded designs, also referred to as combination or hybrid designs, consist of one of these basic designs forms embedded in another design (e.g., a changing criterion design embedded in a multiple baseline design). Systematic reviews of SCEDs have repeatedly indicated that embedded designs are frequently used in applied SCED research. In spite of their popularity, specific recommendations on the conduct and analysis of embedded SCED designs are lacking to date. The purpose of the present article is therefore to provide guidance to applied researchers wishing to conduct embedded SCED designs in terms of design options, design requirements, randomization, and data analysis.
Collapse
Affiliation(s)
- René Tanious
- Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research Group, KU Leuven, Leuven, Belgium
| | - Rumen Manolov
- Faculty of Psychology, Department of Social Psychology and Quantitative Psychology, University of Barcelona, Barcelona, Spain
| |
Collapse
|
14
|
Introduction to the Special Section: Translating Advanced Quantitative Techniques for Single-Case Experimental Design Data. Perspect Behav Sci 2022; 45:1-4. [PMID: 35342871 PMCID: PMC8894512 DOI: 10.1007/s40614-022-00327-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/11/2022] [Indexed: 10/19/2022] Open
Abstract
The articles in this special section offer strategies to single-case experimental design (SCED) researchers to interpret their outcomes, communicate their results, and compare the results using common, quantitative results. Advancing quantitative methods applied to SCED data will facilitate communication with scientists and other professionals that do not typically interpret graphed data of the dependent variable. Horner and Ferron aptly note that innovative statistical procedures are improving the precision and credibility of SCED research as disseminate our findings to an increasingly diverse audience. This special section promotes the translation of these quantitative methods to encourage their adoption in research using single case experimental designs.
Collapse
|
15
|
Levin JR, Ferron JM. Different randomized multiple-baseline models for different situations: A practical guide for single-case intervention researchers. J Sch Psychol 2021; 86:169-177. [PMID: 34051912 DOI: 10.1016/j.jsp.2021.03.003] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Revised: 01/02/2021] [Accepted: 03/21/2021] [Indexed: 10/21/2022]
Abstract
Single-case researchers often implement multiple-baseline designs as their preferred methodology for intervention evaluations. Recent writings and empirical investigations have argued in favor of incorporating various forms of randomization into such designs for the purpose of elevating the intervention study's internal validity and scientific credibility. In this article, we consider a variety of randomized multiple-baseline designs and associated randomization statistical tests, along with their potential strengths and limitations. In what amounts to a practical guide, we refer school psychology researchers to these versatile randomization procedures for planning and executing their intervention studies.
Collapse
|
16
|
Manolov R, Moeyaert M, Fingerhut JE. A Priori Justification for Effect Measures in Single-Case Experimental Designs. Perspect Behav Sci 2021; 45:153-186. [DOI: 10.1007/s40614-021-00282-2] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/27/2021] [Indexed: 01/31/2023] Open
|
17
|
A systematic review of applied single-case research published between 2016 and 2018: Study designs, randomization, data aspects, and data analysis. Behav Res Methods 2020; 53:1371-1384. [PMID: 33104956 DOI: 10.3758/s13428-020-01502-4] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/09/2020] [Indexed: 11/08/2022]
Abstract
Single-case experimental designs (SCEDs) have become a popular research methodology in educational science, psychology, and beyond. The growing popularity has been accompanied by the development of specific guidelines for the conduct and analysis of SCEDs. In this paper, we examine recent practices in the conduct and analysis of SCEDs by systematically reviewing applied SCEDs published over a period of three years (2016-2018). Specifically, we were interested in which designs are most frequently used and how common randomization in the study design is, which data aspects applied single-case researchers analyze, and which analytical methods are used. The systematic review of 423 studies suggests that the multiple baseline design continues to be the most widely used design and that the difference in central tendency level is by far most popular in SCED effect evaluation. Visual analysis paired with descriptive statistics is the most frequently used method of data analysis. However, inferential statistical methods and the inclusion of randomization in the study design are not uncommon. We discuss these results in light of the findings of earlier systematic reviews and suggest future directions for the development of SCED methodology.
Collapse
|
18
|
Tanious R, Onghena P. Randomized Single-Case Experimental Designs in Healthcare Research: What, Why, and How? Healthcare (Basel) 2019; 7:E143. [PMID: 31766188 PMCID: PMC6955662 DOI: 10.3390/healthcare7040143] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2019] [Revised: 09/26/2019] [Accepted: 11/07/2019] [Indexed: 11/17/2022] Open
Abstract
Health problems are often idiosyncratic in nature and therefore require individualized diagnosis and treatment. In this paper, we show how single-case experimental designs (SCEDs) can meet the requirement to find and evaluate individually tailored treatments. We give a basic introduction to the methodology of SCEDs and provide an overview of the available design options. For each design, we show how an element of randomization can be incorporated to increase the internal and statistical conclusion validity and how the obtained data can be analyzed using visual tools, effect size measures, and randomization inference. We illustrate each design and data analysis technique using applied data sets from the healthcare literature.
Collapse
Affiliation(s)
- René Tanious
- Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research Group, KU Leuven—University of Leuven, 3000 Leuven, Belgium;
| | | |
Collapse
|
19
|
Tanious R, Manolov R, Onghena P. The Assessment of Consistency in Single-Case Experiments: Beyond A-B-A-B Designs. Behav Modif 2019; 45:560-580. [PMID: 31619052 DOI: 10.1177/0145445519882889] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Quality standards for single-case experimental designs (SCEDs) recommend inspecting six data aspects: level, trend, variability, overlap, immediacy, and consistency of data patterns. The data aspect consistency has long been neglected by visual and statistical analysts of SCEDs despite its importance for inferring a causal relationship. However, recently a first quantification has been proposed in the context of A-B-A-B designs, called CONsistency of DAta Patterns (CONDAP). In the current paper, we extend the existing CONDAP measure for assessing consistency in designs with more than two successive A-B elements (e.g., A-B-A-B-A-B), multiple baseline designs, and changing criterion designs. We illustrate each quantification with published research.
Collapse
Affiliation(s)
- René Tanious
- Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research Group, KU Leuven - University of Leuven, Leuven, Belgium
| | - Rumen Manolov
- Department of Social Psychology and Quantitative Psychology, Faculty of Psychology, University of Barcelona, Barcelona, Spain
| | - Patrick Onghena
- Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research Group, KU Leuven - University of Leuven, Leuven, Belgium
| |
Collapse
|
20
|
A randomization test wrapper for synthesizing single-case experiments using multilevel models: A Monte Carlo simulation study. Behav Res Methods 2019; 52:654-666. [PMID: 31270794 DOI: 10.3758/s13428-019-01266-6] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Multilevel models (MLMs) have been proposed in single-case research, to synthesize data from a group of cases in a multiple-baseline design (MBD). A limitation of this approach is that MLMs require several statistical assumptions that are often violated in single-case research. In this article we propose a solution to this limitation by presenting a randomization test (RT) wrapper for MLMs that offers a nonparametric way to evaluate treatment effects, without making distributional assumptions or an assumption of random sampling. We present the rationale underlying the proposed technique and validate its performance (with respect to Type I error rate and power) as compared to parametric statistical inference in MLMs, in the context of evaluating the average treatment effect across cases in an MBD. We performed a simulation study that manipulated the numbers of cases and of observations per case in a dataset, the data variability between cases, the distributional characteristics of the data, the level of autocorrelation, and the size of the treatment effect in the data. The results showed that the power of the RT wrapper is superior to the power of parametric tests based on F distributions for MBDs with fewer than five cases, and that the Type I error rate of the RT wrapper is controlled for bimodal data, whereas this is not the case for traditional MLMs.
Collapse
|