The effects of procedural integrity errors during auditory–visual conditional discrimination training: A preliminary investigation

Comparison of Enhanced and Standard Data Sheets on Treatment Fidelity and Data Collection for Tact Training

Specifically designed data sheets have been recommended to assist with the fidelity of implementation of treatment procedures. The present study extended previous research (e.g., Bottini et al. Behavior Analysis: Research & Practice 21(2), 140-152, 2021; LeBlanc et al. Behavior Analysis in Practice 13(1), 53-62, 2020) by comparing an enhanced data sheet (i.e., the inclusion of randomized targets, prompts for treatment components of securing attending and reinforcement) to a standard data sheet (i.e., targets not preset, no prompts for treatment components) on the fidelity of tact training of features. Ten behavior therapists participated in each condition (n = 20). Participants first watched a brief instructional video explaining the teaching procedure and their assigned data sheet, followed by conducting a treatment session with a confederate serving as a child with autism spectrum disorder. The enhanced data sheet resulted in higher fidelity on multiple variables including randomizing of targets and data collection.

Effects of omission and commission errors during tact instruction

The purpose of the current study was to compare the effects of omission and commission errors of reinforcement during tact instruction via telehealth with three children, 6 to 7 years of age, who were diagnosed with an autism spectrum disorder. We used an adapted alternating treatment design to evaluate skill acquisition of target stimuli across high-integrity, commission errors, and omission errors conditions. The high-integrity condition produced mastery criteria in fewer sessions compared with the integrity-error conditions in four of six comparisons, and the omission condition reached mastery criteria in fewer sessions than the commission condition in five of six comparisons.

A detailed examination of reporting procedural fidelity in the Journal of Applied Behavior Analysis

Few reviews on procedural fidelity-the degree to which procedures are implemented as designed-provide details to gauge the quality of fidelity reporting in behavior-analytic research. This review focused on experiments in the Journal of Applied Behavior Analysis (2006-2021) with "integrity" or "fidelity" in the abstract or body. When fidelity data were collected, the coders characterized measurement details (e.g., description of calculation, report of single or multiple values, frequency of fidelity checks, checklist use). The researchers found increasing trends in describing the calculation(s), reporting multiple values, and stating the frequency of measurement. Few studies described using a checklist. Most studies reported fidelity as a percentage, with high obtained values (M = 97%). When not collecting fidelity data was stated as a limitation, authors were unlikely to provide a rationale for the omission. We discuss recommendations for reporting procedural fidelity to increase the quality of and transparency in behavior-analytic research.

Peter CC, Feng J, Long BP. Trends in Reporting Procedural Integrity: A Comparison

Procedural integrity refers to the extent to which an independent variable is implemented as described. Measuring procedural integrity is one important factor when considering internal and external validity of experiments. Experimental articles in behavior-analytic journals have rarely reported procedural-integrity data. The purpose of this study was to update previous reviews of whether articles published in the Journal of Applied Behavior Analysis reported procedural integrity, spanning a period from 1980 to 2020, and compare reporting in JABA to recent reviews of studies published in Behavior Analysis in Practice (2008-2019) and the Journal of Organizational Behavior Management (2000-2020). Procedural integrity continues to be underreported across all three journals, but an increasing trend in reporting procedural integrity is evident in the Journal of Applied Behavior Analysis and Behavior Analysis in Practice. In addition to our recommendations and implications for research and practice, we provide examples and resources to assist researchers and practitioners with recording and reporting integrity data.

Comparing Multiple Methods to Measure Procedural Fidelity of Discrete-trial Instruction

Procedural fidelity is the extent to which an intervention is implemented as designed and is an important component of research and practice. There are multiple ways to measure procedural fidelity, and few studies have explored how procedural fidelity varies based on the method of measurement. The current study compared adherence to discrete-trial instruction protocols by behavior technicians with a child with autism when observers used different procedural-fidelity measures. We collected individual-component and individual-trial fidelity with an occurrence-nonoccurrence data sheet and compared these scores to global fidelity and all-or-nothing, 3-point Likert scale, and 5-point Likert scale measurement methods. The all-or-nothing method required all instances of a component or trial be implemented without error to be scored correct. The Likert scales used a rating system to score components and trials. At the component level, we found that the global, 3-point Likert, and 5-point Likert methods were likely to overestimate fidelity and mask component errors, and the all-or-nothing method was unlikely to mask errors. At the trial level, we found that the global and 5-point Likert methods approximated individual-trial fidelity, the 3-point Likert method overestimated fidelity, and the all-or-nothing method underestimated fidelity. The occurrence-nonoccurrence method required the most time to complete, and all-or-nothing by trial required the least. We discuss the implications of measuring procedural fidelity with different methods of measurement, including false positives and false negatives, and provide suggestions for practice and research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43494-023-00094-w.

A parametric analysis of procedural fidelity errors following mastery of a task: A translational study

Procedural fidelity is defined as the extent to which the independent variable is implemented as prescribed. Research using computerized tasks has shown that fidelity errors involving consequences for behavior can hinder skill acquisition. However, studies examining the effects of these errors once skills have been mastered are lacking. Thus, this translational study investigated the effects of varying levels of fidelity following mastery of a computerized arbitrary matching-to-sample task. A group design (consisting of five groups) was used in which college students initially completed 250 trials during which no programmed errors (i.e., perfect fidelity) were arranged, followed by an additional 250 trials with consequences delivered across various levels of fidelity (i.e., 20, 40, 60, 80, and 100% of trials administered without errors). The results showed that participants assigned to higher fidelity conditions performed better (on average). These results extended the findings of previous studies by demonstrating how errors involving consequences affect behavior across various stages of learning.

Strengthening the procedural fidelity research-to-practice loop in animal behavior

Procedural fidelity is the extent to which components of an intervention are implemented as designed. Procedural fidelity is measured as a dependent variable and manipulated as an independent variable. In research and practice, procedural-fidelity data should be collected, monitored, and reported. Procedural fidelity as an independent variable has been investigated in humans using parametric analyses, and the current article summarizes some of the research conducted on the effects of procedural-fidelity errors in behavior-reduction and skill-acquisition interventions. Connections were drawn to applied animal researchers and the work of animal behavior practitioners to produce implications for practice with human and animal clients and suggestions for future research. Further, there are multiple ways to measure procedural fidelity, and different conclusions can be drawn based on the measure and computation method. The current article describes procedural-fidelity measures that are most applicable to animal behavior researchers and professionals.

When do errors in reinforcer delivery affect learning? A parametric analysis of treatment integrity

Treatment integrity is the extent to which components of an intervention are implemented as intended (Gresham, 1989). Recent behavior-analytic literature has begun to evaluate the effects of reduced-treatment integrity on the efficacy and efficiency of skill-acquisition interventions. This study extended the current literature on the effects of errors of omission and commission of reinforcer delivery by replicating and extending Hirst and DiGennaro Reed (2015). Using a randomized-control group design, we compared undergraduate student participants' acquisition of conditional discriminations in a parametric analysis of different error values. A computer program erred in reinforcer delivery on 0%, 5%, 10%, 15%, 20%, 25% and 50% of trials. The purpose of the current study was to identify which levels of reduced integrity slowed or prevented acquisition. Our data replicated the findings of Hirst and DiGennaro Reed, and extended parametric analyses by identifying that errors in reinforcer delivery occurring on 15% or fewer trials (i.e., 85% integrity) were unlikely to prevent participants' responding from meeting the mastery criterion. These results could inform future research on how treatment-integrity errors change behavior-analytic procedures and the effects on skill acquisition for consumers of applied behavior analysis.