1
|
Tincani M, Gilroy SP, Dowdy A. Extensions of open science for applied behavior analysis: Preregistration for single-case experimental designs. J Appl Behav Anal 2024. [PMID: 39140415 DOI: 10.1002/jaba.2909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2024] [Accepted: 07/23/2024] [Indexed: 08/15/2024]
Abstract
Open science practices are designed to enhance the utility, integrity, and credibility of scientific research. This article highlights how preregistration in open science practice can be leveraged to enhance the rigor and transparency of single-case experimental designs within an applied behavior analysis framework. We provide an overview of the benefits of preregistration including increased transparency, reduced risk of researcher bias, and improved replicability, and we review the specific contexts under which these practices most benefit the proposed framework. We discuss potential concerns with and unique considerations for preregistering experiments that use single-case designs, with practical guidance for researchers who are seeking to preregister their studies. We present a checklist as a tool for researchers in applied behavior analysis to use for preregistration and provide recommendations for our field to strengthen the contingencies for open science practices that include preregistration.
Collapse
Affiliation(s)
- Matt Tincani
- Department of Teaching and Learning, Temple University, Philadelphia, PA, USA
| | - Shawn P Gilroy
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA
| | - Art Dowdy
- Department of Teaching and Learning, Temple University, Philadelphia, PA, USA
| |
Collapse
|
2
|
Laureano B, Fernandez N, Hagopian LP. Efficacy of competing stimulus assessments: A summary of 35 consecutively encountered cases. J Appl Behav Anal 2023; 56:428-441. [PMID: 36855029 PMCID: PMC10329484 DOI: 10.1002/jaba.979] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Accepted: 02/12/2023] [Indexed: 03/02/2023]
Abstract
Competing stimulus assessments (CSAs) are designed to identify stimuli that reduce challenging behavior through competition with its maintaining reinforcers. Recently, Haddock and Hagopian (2020) found that over 92% of CSAs described in published studies identified at least one high-competition stimulus (i.e., a stimulus correlated with at least an 80% reduction in challenging behavior). The current study describes the outcomes of CSAs in a retrospective consecutive controlled case series study of 35 cases (individuals) admitted to an inpatient setting. Findings on the limited relation between the level of stimulus engagement and reductions in challenging behavior were replicated; however, the efficacy of CSAs was lower (only 47% of CSAs were successful in identifying one or more high-competition stimuli). Discrepant findings across studies on the efficacy of CSAs are discussed in terms of differences in the sample participants and how outcomes are reported, which vary depending on the study's research questions.
Collapse
Affiliation(s)
- Brianna Laureano
- Department of Behavioral Psychology, Kennedy Krieger Institute
- Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine
| | - Nathalie Fernandez
- Department of Behavioral Psychology, Kennedy Krieger Institute
- Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine
| | - Louis P. Hagopian
- Department of Behavioral Psychology, Kennedy Krieger Institute
- Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine
| |
Collapse
|
3
|
Nilles C, Amorelli G, Pringsheim TM, Martino D. "Unvoluntary" Movement Disorders: Distinguishing between Tics, Akathisia, Restless Legs, and Stereotypies. Semin Neurol 2023; 43:123-146. [PMID: 36854394 DOI: 10.1055/s-0043-1764164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/02/2023]
Abstract
Tics, stereotypies, akathisia, and restless legs fall at different places on the spectrum of discrete, unwanted and potentially disabling motor routines. Unlike tremor, chorea, myoclonus, or dystonia, this subgroup of abnormal movements is characterized by the subject's variable ability to inhibit or release undesired motor patterns on demand. Though it may be sometimes clinically challenging, it is crucial to distinguish these "unvoluntary" motor behaviors because secondary causes and management approaches differ substantially. To this end, physicians must consider the degree of repetitiveness of the movements, the existence of volitional control, and the association with sensory symptoms, or cognitive-ideational antecedent. This review aims to summarize the current existing knowledge on phenomenology, diagnosis, and treatment of tics, stereotypies, akathisia, and restless leg syndrome.
Collapse
Affiliation(s)
- Christelle Nilles
- Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Gabriel Amorelli
- Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Tamara M Pringsheim
- Department of Clinical Neurosciences, Psychiatry, Pediatrics and Community Health Sciences, University of Calgary, Calgary, Alberta, Canada.,Mathison Centre for Mental Health Research and Education, Calgary, Alberta, Canada.,Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
| | - Davide Martino
- Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada.,Mathison Centre for Mental Health Research and Education, Calgary, Alberta, Canada.,Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
| |
Collapse
|
4
|
Ledford JR, Trump C, Chazin KT, Windsor SA, Eyler PB, Wunderlich K. Systematic review of interruption and redirection procedures for autistic individuals. BEHAVIORAL INTERVENTIONS 2022. [DOI: 10.1002/bin.1905] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Affiliation(s)
| | - Cary Trump
- University of Northern Colorado Greeley Colorado USA
| | | | | | | | | |
Collapse
|
5
|
Taylor T, Lanovaz MJ. Agreement between visual inspection and objective analysis methods: A replication and extension. J Appl Behav Anal 2022; 55:986-996. [PMID: 35478098 PMCID: PMC9323513 DOI: 10.1002/jaba.921] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Revised: 03/14/2022] [Accepted: 03/18/2022] [Indexed: 11/30/2022]
Abstract
Behavior analysts typically rely on visual inspection of single‐case experimental designs to make treatment decisions. However, visual inspection is subjective, which has led to the development of supplemental objective methods such as the conservative dual‐criteria method. To replicate and extend a study conducted by Wolfe et al. (2018) on the topic, we examined agreement between the visual inspection of five raters, the conservative dual‐criteria method, and a machine‐learning algorithm (i.e., the support vector classifier) on 198 AB graphs extracted from clinical data. The results indicated that average agreement between the 3 methods was generally consistent. Mean interrater agreement was 84%, whereas raters agreed with the conservative dual‐criteria method and the support vector classifier on 84% and 85% of graphs, respectively. Our results indicate that both objective methods produce results consistent with visual inspection, which may support their future use.
Collapse
Affiliation(s)
- Tessa Taylor
- University of Canterbury/Te Whare Wānanga o Waitaha.,Paediatric Feeding International
| | | |
Collapse
|
6
|
Dowdy A, Jessel J, Saini V, Peltier C. Structured visual analysis of single-case experimental design data: Developments and technological advancements. J Appl Behav Anal 2021; 55:451-462. [PMID: 34962646 DOI: 10.1002/jaba.899] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Revised: 11/22/2021] [Accepted: 11/22/2021] [Indexed: 11/05/2022]
Abstract
Visual analysis is the primary method used to interpret single-case experimental design (SCED) data in applied behavior analysis. Research shows that agreement between visual analysts can be suboptimal at times. To address the inconsistent interpretations of SCED data, recent structured visual-analysis technological advancements have been developed. To assess the extent to which structured visual analysis is used to guide or supplement applied behavior analysts' interpretation of SCED graphs, a systematic review between the years of 2015 to 2020 in the Journal of Applied Behavior Analysis was conducted. Findings showed that despite recent efforts to develop structured visual-analysis tools and criteria, these methods are rarely used to analyze SCED data. An overview of structured visual-analysis tools is shared, their utility is delineated, common characteristics are brought to light, and future directions for both research and their clinical use are highlighted.
Collapse
Affiliation(s)
- Art Dowdy
- Department of Teaching and Learning, Temple University
| | | | - Valdeep Saini
- Department of Applied Disability Studies, Brock University
| | - Corey Peltier
- Department of Educational Psychology, University of Oklahoma
| |
Collapse
|
7
|
Muharib R, Dowdy A, Rajaraman A, Jessel J. Contingency-based delay to reinforcement following functional communication training for autistic individuals: A multilevel meta-analysis. AUTISM : THE INTERNATIONAL JOURNAL OF RESEARCH AND PRACTICE 2021; 26:761-781. [PMID: 34961394 DOI: 10.1177/13623613211065540] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
LAY ABSTRACT Functional communication training, an intervention for challenging behavior rooted in principles of applied behavior analysis, has copious empirical support dating back to the mid-1980s for autistic individuals. Recently, there has been a concerted effort to thin reinforcement delivery during functional communication training using contingency-based delays that, in turn, are designed to enhance practicality and feasibility while not compromising efficacy. In this synthesis, we meta-analyzed the literature base with the goal of investigating both combined and across type effectiveness of contingency-based delays. We also aimed to investigate moderating variables that might impact intervention outcomes. Findings showed that contingency-based delays were effective for individuals with an autism spectrum disorder diagnosis and most effective when the delay incorporated some form of positive reinforcement. In addition, differential reinforcement of alternative-based delays was overall more effective when compared to differential reinforcement of other behavior-based delays. Noteworthy moderating variables found to impact contingency-based delay efficacy included the intervention dosage and the topography of behavior. We discuss these findings and highlight directions where additional empirical research is warranted to improve our understanding about contingency-based delays for individuals diagnosed with autism spectrum disorder.
Collapse
|
8
|
Meta-Analytic Methods to Detect Publication Bias in Behavior Science Research. Perspect Behav Sci 2021; 45:37-52. [DOI: 10.1007/s40614-021-00303-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/02/2021] [Indexed: 01/01/2023] Open
|
9
|
Lanovaz MJ, Hranchuk K. Machine learning to analyze single-case graphs: A comparison to visual inspection. J Appl Behav Anal 2021; 54:1541-1552. [PMID: 34263923 PMCID: PMC8596748 DOI: 10.1002/jaba.863] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2021] [Revised: 06/04/2021] [Accepted: 06/04/2021] [Indexed: 01/12/2023]
Abstract
Behavior analysts commonly use visual inspection to analyze single‐case graphs, but studies on its reliability have produced mixed results. To examine this issue, we compared the Type I error rate and power of visual inspection with a novel approach—machine learning. Five expert visual raters analyzed 1,024 simulated AB graphs, which differed on number of points per phase, autocorrelation, trend, variability, and effect size. The ratings were compared to those obtained by the conservative dual‐criteria method and two models derived from machine learning. On average, visual raters agreed with each other on only 75% of graphs. In contrast, both models derived from machine learning showed the best balance between Type I error rate and power while producing more consistent results across different graph characteristics. The results suggest that machine learning may support researchers and practitioners in making fewer errors when analyzing single‐case graphs, but replications remain necessary.
Collapse
Affiliation(s)
- Marc J Lanovaz
- École de psychoéducation, Université de Montréal.,Centre de recherche de l'Institut universitaire en santé mentale de Montréal
| | - Kieva Hranchuk
- Behavioural Science and Behavioural Psychology, St. Lawrence College
| |
Collapse
|
10
|
Dowdy A, Peltier C, Tincani M, Schneider WJ, Hantula DA, Travers JC. Meta-analyses and effect sizes in applied behavior analysis: A review and discussion. J Appl Behav Anal 2021; 54:1317-1340. [PMID: 34219222 DOI: 10.1002/jaba.862] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2020] [Revised: 06/01/2021] [Accepted: 06/02/2021] [Indexed: 12/28/2022]
Abstract
For more than four decades, researchers have used meta-analyses to synthesize data from multiple experimental studies often to draw conclusions that are not supported by individual studies. More recently, single-case experimental design (SCED) researchers have adopted meta-analysis techniques to answer research questions with data gleaned from SCED experiments. Meta-analyses enable researchers to answer questions regarding intervention efficacy, generality, and condition boundaries. Here we discuss meta-analysis techniques, the rationale for their adaptation with SCED studies, and current indices used to quantify the effect of SCED data in applied behavior analysis.
Collapse
Affiliation(s)
- Art Dowdy
- Department of Teaching and Learning, Temple University
| | - Corey Peltier
- Department of Educational Psychology, University of Oklahoma
| | - Matt Tincani
- Department of Teaching and Learning, Temple University
| | - W Joel Schneider
- Department of Psychological Studies in Education, Temple University
| | | | | |
Collapse
|
11
|
A Decade Review of Two Potential Analysis Altering Variables in Graph Construction. J Autism Dev Disord 2021; 52:714-724. [PMID: 33763782 DOI: 10.1007/s10803-021-04959-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/02/2021] [Indexed: 10/21/2022]
Abstract
In single-case research designs (SCDs) to determine a functional relation a time-series graph is constructed. Preliminary evidence suggest the approach used to scale the vertical axis and the data points per x- to y-axis ratio (DPPXYR) impact visual analysts' decisions. We conducted a systematic review to evaluate time-series graphs published in the last decade in four premier journals in the field of autism. We included 348 articles including 2675 graphs. We identified large variation across and within types of SCDs when evaluating the lengths of the vertical and horizontal axis using the y:x ratio and the DPPXYR, with few adhering to current recommendations. A majority of graphs used an appropriate method to scale the vertical axis that would not increase Type I error rates.
Collapse
|
12
|
King S, Burch T. Blending as a treatment for feeding disorders: A review of the literature. BEHAVIORAL INTERVENTIONS 2020. [DOI: 10.1002/bin.1748] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Affiliation(s)
- Seth King
- Department of Instruction and Learning University of Iowa Iowa City Iowa USA
| | - Taneal Burch
- Department of Curriculum of Instruction Tennessee Technological University Cookeville Tennessee USA
| |
Collapse
|
13
|
Peltier C, Lingo ME, Autry F, Deardorff ME, Palacios M. Schema-Based Instruction Implemented under Routine Conditions. JOURNAL OF APPLIED SCHOOL PSYCHOLOGY 2020. [DOI: 10.1080/15377903.2020.1821273] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
- Corey Peltier
- Department of Educational Psychology, University of Oklahoma, Norman, Oklahoma, USA
| | - Mindy E. Lingo
- Department of Educational Psychology, University of Oklahoma, Norman, Oklahoma, USA
| | - Faye Autry
- Department of Educational Psychology, University of Oklahoma, Norman, Oklahoma, USA
| | - Malarie E. Deardorff
- Department of Educational Psychology, University of Oklahoma, Norman, Oklahoma, USA
| | - Maegann Palacios
- Department of Educational Psychology, University of Oklahoma, Norman, Oklahoma, USA
| |
Collapse
|