Using e-learning modules to teach ongoing-visual inspection of functional analyses

Evaluating an Excel-based tool for interpreting functional analyses: A functional analysis decision support system

When applied to functional analysis results, structured visual inspection criteria have resulted in improvements in the levels of agreement between raters as well as earlier identification of the function of challenging behavior. However, multistep criteria can be difficult to apply in real time, which could be a barrier to widespread adoption in practice. This study evaluated a Microsoft-Excel-based functional analysis decision support system (FADSS), which could aid behavior analysts with interpreting functional analysis results. Final overall agreement between the FADSS and post hoc visual inspection was high at 95%. Final overall agreement between the post hoc results generated by FADSS and ongoing results generated by FADSS was acceptable at 81%, representing a 50% increase in efficiency. These results indicate that FADSS could aid behavior analysts when interpreting functional analysis results in real time.

Generalized Linear Mixed Effects Modeling (GLMM) of Functional Analysis Graphical Construction Elements on Visual Analysis

Multielement designs are the quintessential design tactic to evaluate outcomes of a functional analysis in applied behavior analysis. Protecting the credibility of the data collection, graphing, and visual analysis processes from a functional analysis increases the likelihood that optimal intervention decisions are made for individuals. Time-series graphs and visual analysis are the most prevalent method used to interpret functional analysis data. The current project included two principal aims. First, we tested whether the graphical construction manipulation of the x-to-y axes ratio (i.e., data points per x- axis to y-axis ratio [DPPXYR]) influenced visual analyst's detection of a function on 32 multielement design graphs displaying functional analyses. Second, we investigated the alignment between board certified behavior analysts (BCBAs; N = 59) visual analysis with the modified visual inspection criteria (Roane et al., Journal of Applied Behavior Analysis, 46, 130-146, 2013). We found that the crossed GLMM that included random slopes, random intercepts, and did not include an interaction effect (AIC = 1406.1, BIC = 1478.2) performed optimally. Second, alignment between BCBAs decisions and the MVI appeared to be low across data sets. We also leveraged current best practices in Open Science for raw data and analysis transparency.

A detailed examination of reporting procedural fidelity in the Journal of Applied Behavior Analysis

Few reviews on procedural fidelity-the degree to which procedures are implemented as designed-provide details to gauge the quality of fidelity reporting in behavior-analytic research. This review focused on experiments in the Journal of Applied Behavior Analysis (2006-2021) with "integrity" or "fidelity" in the abstract or body. When fidelity data were collected, the coders characterized measurement details (e.g., description of calculation, report of single or multiple values, frequency of fidelity checks, checklist use). The researchers found increasing trends in describing the calculation(s), reporting multiple values, and stating the frequency of measurement. Few studies described using a checklist. Most studies reported fidelity as a percentage, with high obtained values (M = 97%). When not collecting fidelity data was stated as a limitation, authors were unlikely to provide a rationale for the omission. We discuss recommendations for reporting procedural fidelity to increase the quality of and transparency in behavior-analytic research.

Remarkably reproducible psychological (memory) phenomena in the classroom: some evidence for generality from small-N research

BACKGROUND

Mainstream psychology is experiencing a crisis of confidence. Many of the methodological solutions offered in response have focused largely on statistical alternatives to null hypothesis statistical testing, ignoring nonstatistical remedies that are readily available within psychology; namely, use of small-N designs. In fact, many classic memory studies that have passed the test of replicability used them. That methodological legacy warranted a retrospective look at nonexperimental data to explore the generality of the reported effects.

METHOD

Various classroom demonstrations were conducted over multiple semesters in introductory psychology courses with typical, mostly freshman students from a predominantly white private Catholic university in the US Midwest based on classic memory experiments on immediate memory span, chunking, and depth of processing.

RESULTS

Students tended to remember 7 ± 2 digits, remembered more digits of π following an attached meaningful story, and remembered more words after elaborative rehearsal than after maintenance rehearsal. These results amount to replications under uncontrolled classroom environments of the classic experiments originally conducted largely outside of null hypothesis statistical testing frameworks.

CONCLUSIONS

In light of the ongoing replication crisis in psychology, the results are remarkable and noteworthy, validating these historically important psychological findings. They are testament to the reliability of reproducible effects as the hallmark of empirical findings in science and suggest an alternative approach to commonly proffered solutions to the replication crisis.

Creating Responsive Asynchronous Instructional Sequences Using PowerPointTM for Microsoft 365®

The prevalence of distance education utilizing asynchronous instruction has increased in recent years. Asynchronous instruction differs from the more common synchronous instruction in that learners primarily contact the lessons and educational materials on their own rather than with a live instructor. Though not without its limitations, asynchronous instruction offers a variety of advantages that can make instruction more efficient, produce better outcomes, and increase accessibility to a greater variety of learners if created using known principles of effective instructional design. Though many platforms exist for creating asynchronous instruction, these are often accompanied by barriers to their widespread use. A potential cost-effective and flexible alternative to these is Microsoft® PowerPoint^TM. The present report serves as a guide for creating interactive and responsive asynchronous instructional sequences with PowerPoint for Microsoft 365® using principles and procedures derived from programmed instruction (Skinner, 1968). Ideas for additional response types are also provided, as are the limitations of designing instructional sequences with this software. Previous papers on the use of PowerPoint as an instructional tool have been primarily geared towards instruction for young learners or learners with autism. As such, the present article expands on the use of PowerPoint specifically to higher education.

On the validity of interpreting functional analyses of inappropriate mealtime behavior using structured criteria

Visual inspection is the traditional method behavior analysts use to interpret functional-analysis results. Limitations of visual inspection include lack of standardized rules, subjectivity, and inconsistent interrater reliability (Fisch, 1998). To address these limitations, researchers have developed, evaluated, and refined structured criteria to aid interpretation of functional analyses of destructive behavior (Hagopian et al., 1997; Roane et al., 2013; Saini et al., 2018). The current study applied the structured criteria Saini et al. (2018) described to functional analyses of inappropriate mealtime behavior. We assessed its predictive validity and evaluated its efficiency relative to 3 post hoc visual inspection procedures. Validity metrics were lower than those in Saini et al. however, ongoing visual inspection increased the efficiency of functional analyses by more than 30%. We discuss these findings relative to the procedural differences between functional analyses of destructive behavior and inappropriate mealtime behavior.

Structured visual analysis of single-case experimental design data: Developments and technological advancements

Visual analysis is the primary method used to interpret single-case experimental design (SCED) data in applied behavior analysis. Research shows that agreement between visual analysts can be suboptimal at times. To address the inconsistent interpretations of SCED data, recent structured visual-analysis technological advancements have been developed. To assess the extent to which structured visual analysis is used to guide or supplement applied behavior analysts' interpretation of SCED graphs, a systematic review between the years of 2015 to 2020 in the Journal of Applied Behavior Analysis was conducted. Findings showed that despite recent efforts to develop structured visual-analysis tools and criteria, these methods are rarely used to analyze SCED data. An overview of structured visual-analysis tools is shared, their utility is delineated, common characteristics are brought to light, and future directions for both research and their clinical use are highlighted.

How Many Tiers Do We Need? Type I Errors and Power in Multiple Baseline Designs

Design quality guidelines typically recommend that multiple baseline designs include at least three demonstrations of effects. Despite its widespread adoption, this recommendation does not appear grounded in empirical evidence. The main purpose of our study was to address this issue by assessing Type I error rate and power in multiple baseline designs. First, we generated 10,000 multiple baseline graphs, applied the dual-criteria method to each tier, and computed Type I error rate and power for different number of tiers showing a clear change. Second, two raters categorized the tiers for 300 multiple baseline graphs to replicate our analyses using visual inspection. When multiple baseline designs had at least three tiers and two or more of these tiers showed a clear change, the Type I error rate remained adequate (< .05) while power also reached acceptable levels (> .80). In contrast, requiring all tiers to show a clear change resulted in overly stringent conclusions (i.e., unacceptably low power). Therefore, our results suggest that researchers and practitioners should carefully consider limitations in power when requiring all tiers of a multiple baseline design to show a clear change in their analyses.

Further Evaluation of Teaching Behavior Technicians to Input Data and Graph Using GraphPad Prism

We replicated and extended Mitteer, Greer, Fisher, and Cohrs (2018) by examining the effects of a video model on inputting data into GraphPad Prism, which is a necessary skill for graph construction. We used a concurrent multiple-probe-across-behavior design with two behavior technicians to assess data-input and graphing skills separately prior to and during access to relevant video models. We evaluated the generality of the training procedures by assessing both skills during data-input-plus-graphing sessions without access to the video models. The video models resulted in mastery of data-input and graphing skills when assessed individually. We observed training effects generalize to data-input-plus-graphing sessions once behavior technicians experienced all relevant video models. These results suggest that individuals should view both data-input and graphing video models prior to depicting single-case design data in Prism but that these skills can maintain at high levels of accuracy without continued access to the training materials.