The distribution of response bout lengths and its sensitivity to differential reinforcement

Longer operant lever-press duration requirements induce fewer but longer response bouts in rats

Operant behavior is organized in bouts that are particularly visible under variable-interval (VI) schedules of reinforcement. Previous research showed that increasing the work required to produce a response decreases the rate at which bouts are emitted and increases the minimum interresponse time (IRT). In the current study, the minimum effective IRT was directly manipulated by changing the minimum duration of effective lever presses reinforced on a VI 40-s schedule. Contrary to assumptions of previous models, response durations were variable. Response durations were typically 0.5 s greater than the minimum duration threshold; durations that exceeded this threshold were approximately log-normally distributed. As the required duration threshold increased, rats emitted fewer but longer bouts. This effect may reflect an effort-induced reduction in motivation and a duration-induced facilitation of a response-outcome association.

Behavior, process, and scale: Comments on Shimp (2020), "Molecular (moment-to-moment) and molar (aggregate) analyses of behavior"

If we study the behavior of organisms, we must understand the ontological status of both "organism" and "behavior." A living organism maintains itself alive by constantly interacting with the environment, taking in energy and discarding waste. Ontologically, an organism is a process. Its interactions with the environment, which constitute its behavior, are processes also, because the parts of any process are themselves processes. Processes serve functions, and the function of a process must be part of its identity. A process, by definition, extends in time. Time is the fundamental and universal measure of behavior. All processes have the property of scale. Activities of an organism have parts that are themselves activities on a smaller time scale. Scale varies continuously, and behavior may be studied on as large or as small a time scale as seems necessary. When researchers refer to the "structure" of behavior, they refer to smaller-scale activities. Attaching a switch to a lever or key is convenient, but one should never confuse operation of a switch with a unit of behavior. Shimp's (2020) "molecular" measures are small-scale measures. The molecular view based on discrete events has outlived its usefulness and should be replaced by a multiscale molar paradigm.

Internal-Clock Models and Misguided Views of Mechanistic Explanations: A Reply to Eckard & Lattal (2020)

Eckard and Lattal's Perspectives on Behavior Science, 43(1), 5-19 (2020) critique of internal clock (IC) mechanisms is based on narrow concepts of clocks, of their internality, of their mechanistic nature, and of scientific explanations in general. This reply broadens these concepts to characterize all timekeeping objects-physical and otherwise-as clocks, all intrinsic properties of such objects as internal to them, and all simulatable explanations of such properties as mechanisms. Eckard and Lattal's critique reflects a restrictive billiard-ball view of causation, in which environmental manipulations and behavioral effects are connected by a single chain of contiguous events. In contrast, this reply offers a more inclusive stochastic view of causation, in which environmental manipulations are probabilistically connected to behavioral effects. From either view of causation, computational ICs are hypothetical and unobservable, but their heuristic value and parsimony can only be appreciated from a stochastic view of causation. Billiard-ball and stochastic views have contrasting implications for potential explanations of interval timing. As illustrated by accounts of the variability in start times in fixed-interval schedules of reinforcement, of the two views of causality examined, only the stochastic account supports falsifiable predictions beyond simple replications. It is thus not surprising that the experimental analysis of behavior has progressively adopted a stochastic view of causation, and that it has reaped its benefits. This reply invites experimental behavior analysts to continue on that trajectory.

Simulating bout-and-pause patterns with reinforcement learning

Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice-an agent makes a choice between operant and other behaviors. The second mechanism is cost-a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.

A computational formulation of the behavior systems account of the temporal organization of motivated behavior

The behavior systems framework suggests that motivated behavior-e.g., seeking food and mates, avoiding predators-consists of sequences of actions organized within nested behavioral states. This framework has bridged behavioral ecology and experimental psychology, providing key insights into critical behavioral processes. In particular, the behavior systems framework entails a particular organization of behavior over time. The present paper examines whether such organization emerges from a generic Markov process, where the current behavioral state determines the probability distribution of subsequent behavioral states. This proposition is developed as a systematic examination of increasingly complex Markov models, seeking a computational formulation that balances adherence to the behavior systems approach, parsimony, and conformity to data. As a result of this exercise, a nonstationary partially hidden Markov model is selected as a computational formulation of the predatory subsystem. It is noted that the temporal distribution of discrete responses may further unveil the structure and parameters of the model but, without proper mathematical modeling, these discrete responses may be misleading. Opportunities for further elaboration of the proposed computational formulation are identified, including developments in its architecture, extensions to defensive and reproductive subsystems, and methodological refinements.

Between-session memory degradation accounts for within-session changes in fixed-interval performance

A common assumption in the study of fixed-interval (FI) timing is that FI performance is largely stable within sessions, once it is stable between sessions. Within-session changes in FI performance were examined in published data (Daniels and Sanabria, 2017), wherein some rats were trained on a FI 30-s schedule of food reinforcement (FI30) and others on a FI 90-s schedule (FI90). Following stability, FI90 rats were pre-fed for five sessions. Response rates declined as a function of trial, due more to latency lengthening than to run-rate reduction. Latencies were best described by a dynamic gamma-exponential mixture distribution, in which latency lengthening was driven by the growth of the criterion pulse count for a response and not by a reduction in the speed of an endogenous clock. The speed of the clock was selectively sensitive to the length of the FI; the prevalence and length of exponentially-distributed latencies were selectively sensitive to pre-feeding. These findings reveal (a) that parameters governing FI latencies are selectively sensitive to a range of manipulations, (b) a potential degradation of the criterion pulse count between consecutive sessions, and (c) a subsequent recovery of the criterion pulse count within sessions.

Interval timing under a behavioral microscope: Dissociating motivational and timing processes in fixed-interval performance

The distribution of latencies and interresponse times (IRTs) of rats was compared between two fixed-interval (FI) schedules of food reinforcement (FI 30 s and FI 90 s), and between two levels of food deprivation. Computational modeling revealed that latencies and IRTs were well described by mixture probability distributions embodying two-state Markov chains. Analysis of these models revealed that only a subset of latencies is sensitive to the periodicity of reinforcement, and prefeeding only reduces the size of this subset. The distribution of IRTs suggests that behavior in FI schedules is organized in bouts that lengthen and ramp up in frequency with proximity to reinforcement. Prefeeding slowed down the lengthening of bouts and increased the time between bouts. When concatenated, latency and IRT models adequately reproduced sigmoidal FI response functions. These findings suggest that behavior in FI schedules fluctuates in and out of schedule control; an account of such fluctuation suggests that timing and motivation are dissociable components of FI performance. These mixture-distribution models also provide novel insights on the motivational, associative, and timing processes expressed in FI performance. These processes may be obscured, however, when performance in timing tasks is analyzed in terms of mean response rates.

Suppressive and enhancing effects of nicotine on food-seeking behavior

The present study examined how systemic low doses of nicotine affect the microstructure of reinforced food-seeking behavior in rats. Rats were first given an acute saline or nicotine treatment (0.1-0.6mg/kg, with an inter-injection interval of at least 48h), and then a chronic saline or nicotine treatment (0.3mg/kg/day for 10 consecutive days). Immediately after each injection, rats were required to press a lever five times to obtain food that was available at unpredictable times (on average every 80s) with constant probability. Acute nicotine dose-dependently suppressed behavior prior to the delivery of the first reinforcer, but enhanced food-reinforced behavior afterwards. These effects were primarily observed in the time it took rats to initiate food-seeking behavior. Enhancing effects were also observed in the microstructure of food-seeking behavior, with lower nicotine doses (0.1-0.3mg/kg) increasing the rate at which response bouts were initiated, and higher doses (0.3-0.6mg/kg) increasing within-bout response rates. A pre-feeding control suggests that changes in appetite alone cannot explain these effects. Over the course of chronic nicotine exposure, tolerance developed to the suppressive, but not to the enhancing effects of nicotine on food-seeking behavior. These results suggest that (a) lower doses of nicotine enhance the reward value of food and/or food-associated stimuli, (b) higher doses of nicotine enhance motoric activity, and (c) ostensive sensitization effects of nicotine on behavior partially reflect a tolerance to its transient suppressive motoric effects.

Methods of comparing associative models and an application to retrospective revaluation

Contemporary theories of associative learning are increasingly complex, which necessitates the use of computational methods to reveal predictions of these models. We argue that comparisons across multiple models in terms of goodness of fit to empirical data from experiments often reveal more about the actual mechanisms of learning and behavior than do simulations of only a single model. Such comparisons are best made when the values of free parameters are discovered through some optimization procedure based on the specific data being fit (e.g., hill climbing), so that the comparisons hinge on the psychological mechanisms assumed by each model rather than being biased by using parameters that differ in quality across models with respect to the data being fit. Statistics like the Bayesian information criterion facilitate comparisons among models that have different numbers of free parameters. These issues are examined using retrospective revaluation data.

A bout analysis of operant response disruption

Operant behavior appears to be organized in bouts of responses, whose parameters are differentially sensitive to various manipulations. This study investigated potential differential effects of three forms of operant response disruption-extinction (EXT), non-contingent reinforcement (NCR), and prefeeding (PRE)-on response bouts. In Experiment 1, Wistar Kyoto rats (WKY) were trained on a tandem variable-time (VT) 120s fixed-ratio (FR) 5 schedule of reinforcement; after stability was established, their responding was disrupted for three sessions with one of the three disrupters (EXT, NCR, or PRE). In Experiment 2, Long Evans (LE) rats were trained on a tandem VT 240s FR 5 to stability, and their responding disrupted with EXT or NCR. In EXT and NCR, response rates declined significantly and progressively over the course of the session, primarily due to a declining bout-initiation rate in EXT, and to fewer responses per bout in NCR. In contrast, a session-wide drop in response rate was observed in PRE, primarily due to a reduction in bout-initiation rate at the start of the session. These findings suggest that each form of disruption differentially impacts dissociable aspects of behavior. Theories of behavioral persistence should account for these functional relations, which appear to be obscured in response rate measures.