A parametric analysis of errors of commission during discrete-trial training

Training individuals to implement discrete-trial teaching procedures using behavioral skills training: A scoping review with implications for practice and research

Behavioral skills training (BST) is an evidence-based approach for training individuals to implement discrete-trial teaching procedures. Despite the effectiveness of this approach, implementing BST can be time and resource intensive, which may interfere with a clinical organization's adoption of this training format. We conducted a scoping review of studies using BST components for training discrete-trial teaching procedures in peer-reviewed articles between 1977 and 2021. We identified 51 studies in 46 publications involving 354 participants. We coded descriptive data on (a) participant characteristics, (b) study characteristics, (c) training conditions (including instructions, modeling, rehearsal, and feedback), and (d) training outcomes. The results indicated that studies have primarily attempted to improve the efficacy and efficiency of BST by modifying or omitting common training components. We provide best-practice considerations for using BST to teach discrete-trial teaching procedures and offer a research agenda to guide future investigation in this area.

Effects of omission and commission errors during tact instruction

The purpose of the current study was to compare the effects of omission and commission errors of reinforcement during tact instruction via telehealth with three children, 6 to 7 years of age, who were diagnosed with an autism spectrum disorder. We used an adapted alternating treatment design to evaluate skill acquisition of target stimuli across high-integrity, commission errors, and omission errors conditions. The high-integrity condition produced mastery criteria in fewer sessions compared with the integrity-error conditions in four of six comparisons, and the omission condition reached mastery criteria in fewer sessions than the commission condition in five of six comparisons.

Peter CC, Feng J, Long BP. Trends in Reporting Procedural Integrity: A Comparison

Procedural integrity refers to the extent to which an independent variable is implemented as described. Measuring procedural integrity is one important factor when considering internal and external validity of experiments. Experimental articles in behavior-analytic journals have rarely reported procedural-integrity data. The purpose of this study was to update previous reviews of whether articles published in the Journal of Applied Behavior Analysis reported procedural integrity, spanning a period from 1980 to 2020, and compare reporting in JABA to recent reviews of studies published in Behavior Analysis in Practice (2008-2019) and the Journal of Organizational Behavior Management (2000-2020). Procedural integrity continues to be underreported across all three journals, but an increasing trend in reporting procedural integrity is evident in the Journal of Applied Behavior Analysis and Behavior Analysis in Practice. In addition to our recommendations and implications for research and practice, we provide examples and resources to assist researchers and practitioners with recording and reporting integrity data.

Comparing Multiple Methods to Measure Procedural Fidelity of Discrete-trial Instruction

Procedural fidelity is the extent to which an intervention is implemented as designed and is an important component of research and practice. There are multiple ways to measure procedural fidelity, and few studies have explored how procedural fidelity varies based on the method of measurement. The current study compared adherence to discrete-trial instruction protocols by behavior technicians with a child with autism when observers used different procedural-fidelity measures. We collected individual-component and individual-trial fidelity with an occurrence-nonoccurrence data sheet and compared these scores to global fidelity and all-or-nothing, 3-point Likert scale, and 5-point Likert scale measurement methods. The all-or-nothing method required all instances of a component or trial be implemented without error to be scored correct. The Likert scales used a rating system to score components and trials. At the component level, we found that the global, 3-point Likert, and 5-point Likert methods were likely to overestimate fidelity and mask component errors, and the all-or-nothing method was unlikely to mask errors. At the trial level, we found that the global and 5-point Likert methods approximated individual-trial fidelity, the 3-point Likert method overestimated fidelity, and the all-or-nothing method underestimated fidelity. The occurrence-nonoccurrence method required the most time to complete, and all-or-nothing by trial required the least. We discuss the implications of measuring procedural fidelity with different methods of measurement, including false positives and false negatives, and provide suggestions for practice and research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43494-023-00094-w.

Caregiver-Implemented Digital Activity Schedules With Virtual Coaching

During the COVID-19 pandemic, many insurance companies approved the funding of telehealth-based behavior analytic services for both training and direct-care purposes. Activity schedules are a simple and effective intervention that can be used in the home environment to improve independence for children with ASD. Recent efforts have shifted the format of activity schedules from paper-based schedules to digital platforms that make the schedules more portable and provide easier access for both the caregiver and the Board Certified Behavior Analyst (BCBA). Google Slides^® is a readily available web-based platform that has been used to design and deliver behavior analytic instruction during the COVID-19 pandemic. A non-concurrent multiple baseline design across three child-caregiver dyads was used to evaluate the effectiveness of a caregiver-implemented digital activity schedule intervention on the independent play behaviors of children with ASD. The activity schedule was created and shared on the Google Slides^® platform and caregivers received bug-in-ear (BIE) coaching from practitioners to implement the intervention. Regardless of some minor treatment implementation difficulties attributed to the remote service delivery and the digital format, the digital platform and coaching intervention were effective in increasing independent schedule following with minimal training on the part of the caregiver. We hope that the results of this study provide BCBAs with additional guidance on effective interventions and procedures for remote service delivery.

A Comparison Between Direct Telehealth and In-Person Methods of Teaching Expressive Labels to Children Diagnosed With Autism Spectrum Disorder

Recent behavior analytic research has demonstrated that the provision of applied behavior analytic services via direct telehealth can be an effective teaching modality for some learners with autism spectrum disorder (ASD). Historically, teaching procedures based on applied behavior analysis (ABA), including discrete trial teaching (DTT), have been provided and evaluated via in-person delivery. This study sought to compare the implementation of DTT via direct telehealth to DTT implemented in-person within and across participants. Specifically, this study evaluated the two delivery modalities in terms of skill acquisition, maintenance, efficiency, and learner responding during teaching sessions. Results of an adapted alternating treatments design nested into a multiple baseline design demonstrated that all three participants diagnosed with ASD met the mastery criteria for the expressive labels taught. Areas of future research, participant prerequisite skills, and clinical implications will be discussed in the context of these results.

Effectiveness of Remote Delayed Performance Feedback on Accurate Implementation of Caregiver Coaching

The supervision of field experiences is an indispensable component of Board-Certified Behavior Analyst (BCBA®) training. During the supervised field experience, supervisors regularly provide performance feedback to trainees for the purpose of improving fidelity of implementation of various assessments and interventions. Emerging evidence supports the efficacy of using telehealth to train teachers and parents to implement interventions, but no study has evaluated the effectiveness of the remote delayed performance feedback among individuals completing BCBA® training. We used videoconference equipment and software to deliver remote delayed performance feedback to seven participants enrolled in a graduate program and completing supervised field experience. Remote delayed performance feedback was provided regarding participants' implementation of caregiver coaching. The results indicate that delayed performance feedback provided remotely increased the correct implementation of caregiver coaching. These preliminary results indicate the efficacy of remote supervision and delayed performance feedback.

Strengthening the procedural fidelity research-to-practice loop in animal behavior

Procedural fidelity is the extent to which components of an intervention are implemented as designed. Procedural fidelity is measured as a dependent variable and manipulated as an independent variable. In research and practice, procedural-fidelity data should be collected, monitored, and reported. Procedural fidelity as an independent variable has been investigated in humans using parametric analyses, and the current article summarizes some of the research conducted on the effects of procedural-fidelity errors in behavior-reduction and skill-acquisition interventions. Connections were drawn to applied animal researchers and the work of animal behavior practitioners to produce implications for practice with human and animal clients and suggestions for future research. Further, there are multiple ways to measure procedural fidelity, and different conclusions can be drawn based on the measure and computation method. The current article describes procedural-fidelity measures that are most applicable to animal behavior researchers and professionals.

Human temporal learning with mixed signals

The influence of cue informativeness on human temporal discrimination was evaluated using a peak-interval (PI) procedure. A target moved across the computer monitor, reaching the center at 2 or 4 s. Key presses shot the center of the screen. Participants earned points when shots hit the target and lost points for misses. The target was masked during occasional, extended PI trials, allowing for measurement of temporal discrimination. During PI trials, the screen background color could exert stimulus control by providing information about target speed. Cue informativeness was represented as the correlation (φ) between light or dark green backgrounds and the 2- or 4-s target and was manipulated across 4 conditions: a multiple schedule (φ = 1), mixed signals (φ = 0.8, 0.4), and a mixed schedule (φ = 0). In Experiment 1, participants were randomly assigned to one of the 4 conditions. In Experiment 2, each participant experienced all 4 conditions. Participants learned to respond at both intervals in all conditions. Cue informativeness did not affect peak time or spread. For the most part, temporal distributions of responses for the two background colors suggested a cover-both-bases strategy in the presence of mixed signals. Participants incorporated probabilistic information from cues to allocate responding in time.

Using the Performance Diagnostic Checklist to evaluate and promote paraeducators' treatment fidelity

Paraeducators, who increasingly serve as implementers for students in special education, can struggle with intervention fidelity. Special educators, often responsible for providing paraeducators with support, receive limited supervision training. From organizational behavior management, the Performance Diagnostic Checklist-Human Services (PDC-HS) is a checklist to detect reasons for poor workplace performance and identify aligned support. Utilizing a multiple baseline design, this study evaluated the impact of PDC-HS indicated implementation support on the intervention fidelity of five paraeducators implementing six behavior intervention plans for students receiving special education services. The PDC-HS indicated that all paraeducators would benefit from behavior skills training and visuals of the plans, in addition to other specific supports for individual paraeducators (e.g., prompts, feedback). These implementation supports were effective in improving intervention fidelity for four paraeducators, whereas one paraeducator (across two students) did not respond to these supports. Changes in student outcomes (i.e., academic engagement, disruptive behavior) were inconsistent, suggesting that the behavior intervention plans may have needed to be revised. Usability data suggested that the PDC-HS and indicated supports were generally acceptable, but revisions may be warranted. The implications of this study for school-based implementation support research are discussed.

When do errors in reinforcer delivery affect learning? A parametric analysis of treatment integrity

Treatment integrity is the extent to which components of an intervention are implemented as intended (Gresham, 1989). Recent behavior-analytic literature has begun to evaluate the effects of reduced-treatment integrity on the efficacy and efficiency of skill-acquisition interventions. This study extended the current literature on the effects of errors of omission and commission of reinforcer delivery by replicating and extending Hirst and DiGennaro Reed (2015). Using a randomized-control group design, we compared undergraduate student participants' acquisition of conditional discriminations in a parametric analysis of different error values. A computer program erred in reinforcer delivery on 0%, 5%, 10%, 15%, 20%, 25% and 50% of trials. The purpose of the current study was to identify which levels of reduced integrity slowed or prevented acquisition. Our data replicated the findings of Hirst and DiGennaro Reed, and extended parametric analyses by identifying that errors in reinforcer delivery occurring on 15% or fewer trials (i.e., 85% integrity) were unlikely to prevent participants' responding from meeting the mastery criterion. These results could inform future research on how treatment-integrity errors change behavior-analytic procedures and the effects on skill acquisition for consumers of applied behavior analysis.

Identifying the relation between feedback preferences and performance

Performance feedback is a common organizational behavior management intervention (Gravina et al., 2018) that typically yields consistent effects when combined with other procedures (Alvero et al., 2001). A noteworthy characteristic that might affect feedback effectiveness is timing (Lechermeier & Fassnacht, 2018). Given the frequent use and practical utility of feedback, we assessed feedback preference across three feedback timing options: after step, after trial, and after session. We used a direct-choice paradigm to assess feedback timing preferences with undergraduate students completing two multistep computerized tasks. The data obtained suggest that most participants shifted their preference from relatively proximal (e.g., after step) to distal feedback (e.g., after session) as they acquired the tasks. A few participants' preferences varied despite increases in performance. We discuss implications based on the current findings and future directions for research.

Increasing Verbal Behavior in a Young Adolescent Girl with Catatonia and Autism Spectrum Disorder

Catatonia is a syndrome characterized as a cluster of difficulties in verbal and motor behavior that interferes with everyday function. This study analyzed the effects of a prompt-fading behavioral treatment package on the verbal behavior of an adolescent girl with autism and catatonia. Data were collected on three verbal responses previously observed in the participant's repertoire, including "Hi," "That sounds great," and "Excuse me." Instructors provided full manual guidance, in conjunction with a verbal model, when needed, to assist the participant in emitting the verbal responses. A functional relation was demonstrated between the prompt-fading package and the three verbal responses, and independent verbal responding emerged as manual prompts were faded. Responding was displayed across unfamiliar adults and maintained over time.

The Effects of Standard and Enhanced Data Sheets and Brief Video Training on Implementation of Conditional Discrimination Training

Grow and LeBlanc (2013) described practice recommendations for conducting conditional discrimination training with children with autism. One recommendation involved using a specially designed datasheet to provide the preset target stimulus for each trial along with counterbalancing the location of stimuli if a three-item array of comparison stimuli. This study evaluated whether the recommended data sheet would lead to higher procedural integrity of counterbalancing trials compared to a standard data sheet (i.e., targets and arrays are not pre-set). Forty behavior therapists from two provider agencies participated. Participants were randomly assigned to either the standard data sheet condition or the enhanced data sheet condition. Participants watched a short video on Grow and LeBlanc's practice recommendations for a matching task and an orientation to the datasheet for the assigned condition, and then implemented the matching task with a confederate serving in the role of the child with autism. The enhanced data sheet resulted in higher accuracy of implementation on counterbalancing than the standard data sheet, with the largest difference for rotation of the target stimulus across trials and for counterbalancing the placement of the correct comparison stimulus in the array.

Evaluating the Performance Diagnostic Checklist-Human Services to assess incorrect error-correction procedures by preschool paraprofessionals

The Performance Diagnostic Checklist-Human Services (PDC-HS) has been used to assess variables contributing to undesirable staff performance. In this study, three preschool teachers completed the PDC-HS to identify the factors contributing to four paraprofessionals' inaccurate implementation of error-correction procedures during discrete trial training sessions. The PDC-HS indicated insufficient training as a contributing factor. We then implemented a nonindicated intervention (posting reminders), followed by an indicated intervention (behavioral skills training). The nonindicated intervention failed to produce desired performance improvements; however, the PDC-HS indicated intervention resulted in improvements for all paraprofessionals.

Impact of a Teacher-as-Coach Model: Improving Paraprofessionals Fidelity of Implementation of Discrete Trial Training for Students with Moderate-to-Severe Developmental Disabilities

Ensuring educational progress for students with moderate-to-severe developmental disabilities requires exposure to well executed evidence-based practices. This necessitates that the special education workforce, including paraprofessionals, be well-trained. Yet evidence regarding effective training mechanisms for paraprofessionals is limited. A multiple baseline design across five teachers was used to evaluate the impact of online instructional modules and a Practice-Based Coaching (PBC) model with teacher-as-coach on their paraprofessionals' fidelity of discrete trial training (DTT). Implementation of the instructional modules yielded little to no change in paraprofessionals' DTT fidelity, however, a clear functional relation between PBC and improvement in paraprofessionals' fidelity of implementation of DTT was demonstrated. Implications for future research and practice are discussed.

Selection and Implementation of Skill Acquisition Programs by Special Education Teachers and Staff for Students With Autism Spectrum Disorder

The present investigation examined special education teachers’ selection and use of teaching strategies for receptive identification training with children with autism spectrum disorder (ASD) in their classrooms. Teachers first responded to a survey in which they provided examples of receptive identification tasks taught in their classrooms, rated the efficacy of teaching strategies, described how they determined whether skills were mastered, listed any assessments they conducted to identify relevant prerequisite skills prior to receptive identification training, described how they selected teaching strategies for use in their classrooms, and listed their years of experience as a teacher and working with children with ASD. Subsequent observations of implementation of teaching strategies during trial-based instruction occurred in a proportion of teachers’ classrooms. The results of the observations showed that participants did not consistently implement components of trial-based instruction as described in the literature, and there were differences in implementation depending on the types of skills targeted during instruction.

Impact of treatment integrity on intervention effectiveness

Treatment integrity has cogent implications for intervention effectiveness. Understanding these implications is an important, but often neglected, undertaking in behavior analysis. This paper reviews current research on treatment integrity in applied behavior analysis. Specifically, we review research evaluating the relation between integrity failures and the efficacy of behavioral interventions. Avenues for future research are provided.