Applied implications of reinforcement history effects

An assessment of token value and effectiveness: A replication

The purpose of the current study was to conduct a systematic replication of Fiske et al. (2015) to extend the behavior-analytic literature on token reinforcement. Specifically, the researchers addressed several of the limitations of Fiske et al. by including specific conditioning procedures, creating a controlled history of reinforcement with the token system, including participants with and without disabilities to extend the generality of the findings, and equating the magnitude of reinforcement across the primary and paired-token conditions. The current study evaluated the reinforcing value of tokens by using progressive-ratio schedules to compare the efficacy of primary reinforcement, paired tokens, and unpaired tokens. The results suggest that paired tokens may function as reinforcers, but they were not as consistently effective as primary reinforcers.

Introduction to Teaching the History of Behavior Analysis: Past, Purpose, and Prologue

This article introduces a special section of Perspectives on Behavior Science on teaching the history of behavior analysis. Although behavior is distinctive, behavior analysis is diverse, and the history of behavior analysis is deep, teaching the field's history often is not. The special section offers means for remedying this. The introduction has three sections. First, it relates the genesis of the special section: the 2018 meeting of the Association for Behavior Analysis International and, before that, the 2015 meeting of Cheiron: The International Society for the History of the Behavioral Sciences. Second, it addresses the purposes-reasons and rationales-for teaching history, especially the history of behavior analysis. And third, it offers a prologue for teaching the field's history based on a review of what is taught or not in recent textbooks and handbooks on the field's basic and applied research and their conceptual foundations. In its conclusion, the article previews the section's other articles: (1) three exemplars on how history can be embedded in courses on the field's foregoing three subdisciplines; (2) an exemplar of teaching history in a stand-alone course on the field's history overall; (3) a discussion that addresses how to improve instruction in these courses through narrative methods; and (4) a conclusion about the present and future of teaching the field's history (e.g., giving the history of behavior analysis away).

The influence of reinforcement schedule on experience-dependent changes in motivation

The progressive ratio procedure is used across fields to assess motivation for different reinforcers, define the effects of experimental interventions on motivation, and determine experience-dependent changes in motivation. However, less is known about how operant training schedules affect performance on this widely utilized task. Here we designed an experiment to examine the effect of variable ratio versus fixed ratio training schedules of reinforcement on progressive ratio performance while holding other performance variables constant between groups. We found a robust increase in maximum ratio completed between the pretest and posttraining test highlighting a robust training effect on progressive ratio performance. However, it did not matter if the training was under a fixed or variable ratio schedule. Additionally, we show that neither individual rates during training nor extinction responding correlated with maximum ratio achieved during the sessions. Finally, we show that rates during the training sessions do correlate with extinction performance, suggesting that these variables measure a different aspect of performance that does not predict motivation.

Machine learning with a snapshot of data: Spiking neural network 'predicts' reinforcement histories of pigeons' choice behavior

An accumulated body of choice research has demonstrated that choice behavior can be understood within the context of its history of reinforcement by measuring response patterns. Traditionally, work on predicting choice behaviors has been based on the relationship between the history of reinforcement—the reinforcer arrangement used in training conditions—and choice behavior. We suggest an alternative method that treats the reinforcement history as unknown and focuses only on operant choices to accurately predict (more precisely, retrodict) reinforcement histories. We trained machine learning models known as artificial spiking neural networks (SNNs) on previously published pigeon datasets to detect patterns in choices with specific reinforcement histories—seven arranged concurrent variable‐interval schedules in effect for nine reinforcers. Notably, SNN extracted information from a small ‘window’ of observational data to predict reinforcer arrangements. The models' generalization ability was then tested with new choices of the same pigeons to predict the type of schedule used in training. We examined whether the amount of the data provided affected the prediction accuracy and our results demonstrated that choices made by the pigeons immediately after the delivery of reinforcers provided sufficient information for the model to determine the reinforcement history. These results support the idea that SNNs can process small sets of behavioral data for pattern detection, when the reinforcement history is unknown. This novel approach can influence our decisions to determine appropriate interventions; it can be a valuable addition to our toolbox, for both therapy design and research.

Modeling resurgence with an evolutionary theory of behavior dynamics

McDowell's (2004) Evolutionary Theory of Behavior Dynamics (ETBD) is a computational theory that has reproduced a wide variety of behavioral phenomena observed in material reality. Here, we extended the generality of the ETBD by successfully replicating laboratory studies of resurgence with live animals using artificial organisms (AOs) animated by the theory. We ran AOs on concurrent random-interval random-interval (conc RI RI) schedules of reinforcement wherein one alternative (i.e., a target behavior) was reinforced while the other alternative (i.e., an alternative behavior) was not reinforced. Then, we placed the target behavior on extinction and reinforced the alternative response, producing a shift in allocation of responding from the target behavior to the alternative response. Finally, schedule thinning of the alternative response (i.e., downshifts) resulted in resurgence of target behavior. Our findings indicated that resurgence increased as a function of the relative downshift in reinforcement rate and magnitude, replicating findings from previous studies with live animals. These results further illustrate the utility of the ETBD for generating dynamic behavioral data and serve as a proof-of-concept for a novel computational approach for studying and understanding resurgence in future studies.

Pilot Cohorts for Development of Concurrent Mobile Treatment for Alcohol and Tobacco Use Disorders

Alcohol and tobacco are the 2 most frequently used drugs in the United States and represent the highest co-occurrence of polysubstance use. The objective of this study was to refine an intervention combining mobile contingency management with cognitive-behavioral telephone counseling for concurrent treatment of alcohol and tobacco use disorders. Two cohorts (n = 13 total, n = 5 women) of participants were enrolled, with 10/13 completing treatment and 7/13 completing the 6-month follow-up. At enrollment, participants were drinking a mean of 28.9 drinks per week (SD = 14.1), with a mean of 14.7 heavy drinking days in the past month (SD = 9.9), and a mean of 18.1 cigarettes per day (SD = 11.7). Treatment included a mobile application that participants used to record carbon monoxide and breath alcohol content readings to bioverify abstinence. Participants received up to 4 sessions of phone cognitive-behavioral therapy and monetary reinforcement contingent on abstinence. In cohort 1, 4/6 participants reported abstinent or low-risk drinking post-monitoring. Six weeks post quit-date, 2/6 participants were CO-bioverified abstinent from tobacco use, with 2/6 in dual remission. These results were maintained at 6-months. In cohort 2, 6/7 reported abstinent or low-risk drinking post-monitoring, 5 weeks post quit-date. At the post-monitoring visit, 5/7 were CO-bioverified abstinent from smoking, with 5/7 in dual remission. At 6-months, 3/7 reporting abstinent or low-risk drinking, 1/7 had bioverified abstinence from smoking, with 1/7 in dual remission. Observations suggest that it is possible to develop a concurrent mobile treatment for alcohol and tobacco use disorders.

Stimulus generalization of behavioral history: Interspecies generality and persistence of generalization gradients

Four pigeons were exposed to a tandem variable-interval (VI) fixed-ratio (FR) schedule in the presence of a 50-pixel (about 15 mm) square or an 80-pixel (about 24 mm) square and to a tandem VI differential-reinforcement-of-low-rate (DRL) schedule when a second 80-pixel or 50-pixel square was present. The values of the VI and FR schedules were adjusted to equate reinforcement rates in the two tandem schedules. Following this, a square-size continuum generalization test was administered under a fixed-interval (FI) schedule or extinction. In the first testing session, response frequency was a graded function of the similarity of the test stimuli to the training stimuli for all pigeons. These systematic generalization gradients persisted longer under the FI schedule than under extinction.

Prevalence of renewal of problem behavior: Replication and extension to an inpatient setting

Individuals with intellectual and developmental disabilities who exhibit problem behavior often receive behavioral assessment and treatment in specialized inpatient and outpatient clinics. However, problem behavior sometimes reemerges as a function of changes in contexts and stimulus conditions, such as returning to the home environment. This reemergence is called renewal. Recently, Muething et al. (2020) found that renewal occurred in over half (67%) of cases from an outpatient clinic. Their sample was obtained exclusively from an outpatient setting and despite the applied relevance of renewal, its clinical prevalence in other populations is unknown. Accordingly, we replicated Muething et al.'s procedures and analyzed renewal in 37 inpatient treatment applications across 34 cases via consecutive-controlled case series. Renewal was present in 59% of cases; however, we found that renewal occurred in only 24% of context changes compared to 42% reported by Muething et al. Various factors related to the prevalence of renewal were evaluated.

Behavioral momentum and resistance to extinction across repeated extinction tests

The present experiments assessed whether resistance to extinction of pigeons' key pecking decreased across repeated extinction tests. An additional impetus for this research was to determine how the quantitative framework provided by behavioral momentum theory might be used to describe any such changes across tests. Pigeons pecked keys in two-component multiple schedules (one component associated with a higher reinforcer rate and the other with a lower rate) in which baseline and extinction conditions alternated. In Experiment 1, baseline and extinction conditions alternated every session, and, in Experiment 2, these conditions lasted for 10 and 7 sessions, respectively. Resistance to extinction decreased across successive extinction conditions in both experiments. Fits of the behavioral-momentum based model of extinction to the data returned uncertain results in Experiment 1 but implicated both generalization decrement and response-reinforcer contingency termination as the possible mechanisms responsible for behavior change in Experiment 2. Thus, these data suggest that experimental manipulations that affect discrimination of changes in reinforcement contingencies may influence resistance to extinction by modulating the disruptive impacts of removing reinforcers from the experimental context and of suspending response-reinforcer contingencies.

The role of signals in two variations of differential-reinforcement-of-low-rate procedures

Differential-reinforcement-of-low-rate (DRL) schedules are used to decrease the overall rate of, but not eliminate, a target response. Two variations of DRL, spaced-responding and full-session, exist. Preliminary comparative analyses suggest that the two schedules function differently when unsignaled. We compared response rates under these two DRL variations with and without signals. In Experiment 1, five preschool students played a game in which points were earned under DRL schedules. In some sessions, a stimulus signaled when responses would be reinforced (S+) or not reinforced (S-). In others, only an S- was present. Signals (S+/S-) facilitated and maintained responding in both types of DRL schedules. In Experiment 2, we modified the signals with five different preschoolers. Instead of an S- only, we did not present any signals. Elimination and high variability of the target response were observed with the S- only and absence of S+/S-, respectively. Signaled DRL schedules are recommended for application.

Affective-associative two-process theory: a neurocomputational account of partial reinforcement extinction effects

The partial reinforcement extinction effect (PREE) is an experimentally established phenomenon: behavioural response to a given stimulus is more persistent when previously inconsistently rewarded than when consistently rewarded. This phenomenon is, however, controversial in animal/human learning theory. Contradictory findings exist regarding when the PREE occurs. One body of research has found a within-subjects PREE, while another has found a within-subjects reversed PREE (RPREE). These opposing findings constitute what is considered the most important problem of PREE for theoreticians to explain. Here, we provide a neurocomputational account of the PREE, which helps to reconcile these seemingly contradictory findings of within-subjects experimental conditions. The performance of our model demonstrates how omission expectancy, learned according to low probability reward, comes to control response choice following discontinuation of reward presentation (extinction). We find that a PREE will occur when multiple responses become controlled by omission expectation in extinction, but not when only one omission-mediated response is available. Our model exploits the affective states of reward acquisition and reward omission expectancy in order to differentially classify stimuli and differentially mediate response choice. We demonstrate that stimulus-response (retrospective) and stimulus-expectation-response (prospective) routes are required to provide a necessary and sufficient explanation of the PREE versus RPREE data and that Omission representation is key for explaining the nonlinear nature of extinction data.

Basing assessment and treatment of problem behavior on behavioral momentum theory: Analyses of behavioral persistence

The connection, or bridge, between applied and basic behavior analysis has been long-established (Hake, 1982; Mace & Critchfield, 2010). In this article, we describe how clinical decisions can be based more directly on behavioral processes and how basing clinical procedures on behavioral processes can lead to improved clinical outcomes. As a case in point, we describe how applied behavior analyses of maintenance, and specifically the long-term maintenance of treatment effects related to problem behavior, can be adjusted and potentially enhanced by basing treatment on Behavioral Momentum Theory. We provide a brief review of the literature including descriptions of two translational studies that proposed changes in how differential reinforcement of alternative behavior treatments are conducted based on Behavioral Momentum Theory. We then describe current clinical examples of how these translations are continuing to impact the definitions, designs, analyses, and treatment procedures used in our clinical practice.

AN EVALUATION OF RESURGENCE DURING FUNCTIONAL COMMUNICATION TRAINING

Three children who displayed destructive behavior maintained by negative reinforcement received functional communication training (FCT). During FCT, the children were required to complete a demand and then to mand (touch a card attached to a microswitch, sign, or vocalize) to receive brief play breaks. Prior to and 1 to 3 times following the initiation of FCT, extinction probes were conducted to evaluate the resurgence of destructive behavior when the microswitch without the mand card was present or the microswitch and the mand card were absent to determine if different patterns of resurgence occurred when the microswitch was present or absent and, for 2 of the children, if changes in resurgence occurred at different points in treatment. Results showed that FCT led to relatively rapid reductions in destructive behavior. During all extinction sessions, resurgence of destructive behavior occurred with only minimal differences across the switch/no card and no-switch conditions.

Nationwide access to an internet-based contingency management intervention to promote smoking cessation: a randomized controlled trial

BACKGROUND AND AIMS

Contingency management (CM) is one of the most effective behavioral interventions to promote drug abstinence, but availability of this treatment is limited. We evaluated the efficacy and acceptability of internet-based CM relative to an internet-based monitoring and goal-setting control group in a nationwide sample of cigarette smokers.

DESIGN

Randomized controlled trial with 3- and 6-month follow-ups.

SETTING

United States.

PARTICIPANTS

Smokers (n = 94) from 26 states were enrolled (mean age 36, 56% female).

INTERVENTION AND COMPARATOR

Participants were randomized to earn financial incentives (up to $480 over 7 weeks) based on video-verified abstinence using breath carbon monoxide (CO) output (n = 48; abstinent contingent group, AC), or based on submitting CO samples (n = 46, submission contingent, SC). Both groups also received the same CO-based goals. A $50 deposit was required in both groups that could be recouped from initial earnings.

MEASURES

The primary outcome was point prevalence at week 4. Secondary outcomes were point prevalence at the 3- and 6-month follow-ups, percentages of negative CO samples, adherence to the CO sampling protocol, and treatment acceptability ratings on a 0-100-mm visual analog scale.

FINDINGS

Abstinence rates differed at 4 weeks between the AC (39.6%) and SC (13.0%) groups [odds ratio (OR) = 4.4, 95% confidence interval (CI) = 1.6-12.3], but not at the 3- (29.2% AC and 19.6% SC, OR = 1.7, 95% CI = 0.6-4.4) or 6- (22.9% AC and 13.0% SC, OR = 2.0, 95% CI = 0.7-5.9) month follow-ups. During the two main treatment phases, there were significant differences in negative COs (53.9% AC and 24.8% SC, OR = 3.5, 95% CI = 3.1-4.0; 43.4% AC and 24.6% SC, OR = 2.3, 95% CI = 1.6-3.4). Adherence to the CO submission protocol was equivalent (78% AC and 85% SC, difference = 7.0%, 95% CI = -10.3 to 23.8 %, F < 1, P = 0.39). The lowest acceptability ratings were for the items assessing the deposit, whereas the highest ratings concerned the ease of the intervention, the graph of CO results, and earning money.

CONCLUSIONS

A contingency management/financial incentive program delivered via the internet improved short-term abstinence rates compared with an internet program without the incentives.

Modulation of neural activity by reward in medial intraparietal cortex is sensitive to temporal sequence of reward

To restore movements to paralyzed patients, neural prosthetic systems must accurately decode patients' intentions from neural signals. Despite significant advancements, current systems are unable to restore complex movements. Decoding reward-related signals from the medial intraparietal area (MIP) could enhance prosthetic performance. However, the dynamics of reward sensitivity in MIP is not known. Furthermore, reward-related modulation in premotor areas has been attributed to behavioral confounds. Here we investigated the stability of reward encoding in MIP by assessing the effect of reward history on reward sensitivity. We recorded from neurons in MIP while monkeys performed a delayed-reach task under two reward schedules. In the variable schedule, an equal number of small- and large-rewards trials were randomly interleaved. In the constant schedule, one reward size was delivered for a block of trials. The memory period firing rate of most neurons in response to identical rewards varied according to schedule. Using systems identification tools, we attributed the schedule sensitivity to the dependence of neural activity on the history of reward. We did not find schedule-dependent behavioral changes, suggesting that reward modulates neural activity in MIP. Neural discrimination between rewards was less in the variable than in the constant schedule, degrading our ability to decode reach target and reward simultaneously. The effect of schedule was mitigated by adding Haar wavelet coefficients to the decoding model. This raises the possibility of multiple encoding schemes at different timescales and reinforces the potential utility of reward information for prosthetic performance.

The role of increased exposure to transfer-of-stimulus-control procedures on the acquisition of intraverbal behavior

Studies that have compared the effectiveness of differing prompt types to teach intraverbal responses have yielded mixed results, suggesting that individuals' reinforcement histories with prompt types may influence which prompt will be most effective. The purpose of this study was to test whether programmed increases in exposure to specific prompt types would produce concomitant increases in the acquisition rate of intraverbal responding. We compared acquisition rates among 4 typically developing preschool-aged children when taught via either echoic or tact prompts following exposure training with 1 prompt type. For all participants, the prompt method most recently used to teach intraverbal responses required fewer trials to teach new intraverbal responses compared to a prompt method that had not been used recently. The results are discussed in terms of the effects of reinforcement history on the acquisition of verbal behavior.

Some determinants of remote behavioral history effects in humans

Undergraduates were exposed to a series of reinforcement schedules: first, to a fixed-ratio (FR) schedule in the presence of one stimulus and to a differential-reinforcement-of-low-rate (DRL) schedule in the presence of another (multiple FR DRL training), then to a fixed-interval (FI) schedule in the presence of a third stimulus (FI baseline), next to the FI schedule under the stimuli previously correlated with the FR and DRL schedules (multiple FI FI testing), and, finally, to a single session of the multiple FR DRL schedule again (multiple FR DRL testing). Response rates during the multiple FI FI schedule were higher under the former FR stimulus than under the former DRL stimulus. This effect of remote histories was prolonged when either the number of FI-baseline sessions was small or zero, or the time interval between the multiple FR DRL training and the multiple FI FI testing was short. Response rates under these two stimuli converged with continued exposure to the multiple FI FI schedule in most cases, but quickly differentiated when the schedule returned to the multiple FR DRL.

Reversal Without Remapping

The “cognitive revolution” in psychology is often framed as a departure from associationist principles rooted in animal learning research, yet it is clear that these principles have immediate relevance for contemporary questions in cognitive and social psychology. Intuitions about the consequences of learning procedures can easily be misleading, making these principles particularly important. To illustrate this point, we identified recent examples of studies applying a particular learning paradigm—response-reversal training—to the study of three different psychological problems (e.g., why objects in the right side of space are preferred to those in the left in right-handed people). The strategy of each study was to alter a typically encountered contingency once in the laboratory, in order to reverse a hypothesized learned response. Yet, contrary to intuitions, we demonstrate that behavior changes can be observed without the reversal of a prior association. Further, many different associative changes can underlie response reversals. We focus on these examples of response-reversal training, but our broader aim is to help connect the animal learning literature to problems in cognitive and social psychology in an effort to strengthen the inferences that might be drawn about learned associations in these contexts.

An initial investigation of the generalization of a school-based social competence intervention for youth with high-functioning autism

This study evaluated the impact of generalization of the Social Competence Intervention-Adolescent (SCI-A) curriculum in a school setting for individuals with high-functioning autism or Asperger's Syndrome (N = 6). This study examined to what degree the generalization of the SCI-A curriculum could be measured when delivered in a school setting. Across the six participants preliminary results suggest improvement on teacher reports of social skills and executive functioning. Some improvements were also evident in direct measures of facial-expression recognition. Data collected in the nonintervention settings indicated that some generalization of social interaction skills may have occurred for all six participants. Future research directions are discussed.

Development and modification of a response class via positive and negative reinforcement: a translational approach

When responses function to produce the same reinforcer, a response class exists. Researchers have examined response classes in applied settings; however, the challenges associated with conducting applied research on response class development have recently necessitated the development of an analogue response class model. To date, little research has examined response classes that are strengthened by negative reinforcement. The current investigation was designed to develop a laboratory model of a response class through positive reinforcement (i.e., points exchangeable for money) and through negative reinforcement (i.e., the avoidance of scheduled point losses) with 11 college students as participants and clicks as the operant. Results of both the positive and negative reinforcement evaluations showed that participants usually selected the least effortful response that produced points or the avoidance of point losses, respectively. The applied implications of the findings are discussed, along with the relevance of the present model to the study of punishment and resurgence.

Behavior analysis and ecological psychology: past, present, and future. a review of Harry Heft's Ecological Psychology in context

Relations between behavior analysis and ecological psychology have been strained for years, notwithstanding the occasional comment on their affinities. Harry Heft's (2001)Ecological Psychology in Context provides an occasion for reviewing anew those relations and affinities. It describes the genesis of ecological psychology in James's radical empiricism; addresses Holt's neorealism and Gestalt psychology; and synthesizes Gibson's ecological psychology and Barker's ecobehavioral science as a means for understanding everyday human behavior. Although behavior analysis is excluded from this account, Heft's book warrants a review nonetheless: It describes ecological psychology in ways that are congruent and complementary with behavior analysis (e.g., nonmediational theorizing; the provinces of natural history and natural science). After introducing modern ecological psychology, I comment on (a) Heft's admirable, albeit selective, historiography; (b) his ecological psychology-past and present-as it relates to Skinner's science and system (e.g., affordances, molar behavior); (c) his misunderstandings of Skinner's behaviorism (e.g., reductionistic, mechanistic, molecular); and (d) the theoretical status of Heft's cognitive terms and talk (i.e., in ontology, epistemology, syntax). I conclude by considering the alliance and integration of ecological psychology and behavior analysis, and their implications for unifying and transforming psychology as a life science, albeit more for the future than at present.