Concurrent performance as bouts of behavior

Bouts, Pauses, and Units of Operant Performance: A Primer

Operant behavior typically occurs in bouts and pauses. The microstructural analysis of bouts and pauses reveals important and separable information about the physical characteristics of the operant and the motivation behind it. An analysis of interresponse times (IRTs) often reveals a mixture of two exponential distributions. One corresponds to short IRTs within ongoing response bouts, reflecting motor properties of the operant, and the other corresponds to longer intervals between bouts, reflecting the motivation behind the response. Partitioning responses into bout initiations and within-bout responses via this two-mode framework reveals the mechanisms underlying behavior maintenance and change. This approach is used in the fields of neurotoxicology, behavioral pharmacology, and behavioral neuroscience to disentangle the contribution of motivational and motoric variables to the pattern of operant behavior. In this article, we present a primer aimed at providing essential concepts related to the analysis of response bouts and temporal dynamics of operant performance.

Longer operant lever-press duration requirements induce fewer but longer response bouts in rats

Operant behavior is organized in bouts that are particularly visible under variable-interval (VI) schedules of reinforcement. Previous research showed that increasing the work required to produce a response decreases the rate at which bouts are emitted and increases the minimum interresponse time (IRT). In the current study, the minimum effective IRT was directly manipulated by changing the minimum duration of effective lever presses reinforced on a VI 40-s schedule. Contrary to assumptions of previous models, response durations were variable. Response durations were typically 0.5 s greater than the minimum duration threshold; durations that exceeded this threshold were approximately log-normally distributed. As the required duration threshold increased, rats emitted fewer but longer bouts. This effect may reflect an effort-induced reduction in motivation and a duration-induced facilitation of a response-outcome association.

Simulating bout-and-pause patterns with reinforcement learning

Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice-an agent makes a choice between operant and other behaviors. The second mechanism is cost-a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.

Human free-operant performance varies with a concurrent task: Probability learning without a task, and schedule-consistent with a task

Three experiments examined human rates and patterns of responding during exposure to various schedules of reinforcement with or without a concurrent task. In the presence of the concurrent task, performances were similar to those typically noted for nonhumans. Overall response rates were higher on medium-sized ratio schedules than on smaller or larger ratio schedules (Experiment 1), on interval schedules with shorter than longer values (Experiment 2), and on ratio compared with interval schedules with the same rate of reinforcement (Experiment 3). Moreover, bout-initiation responses were more susceptible to influence by rates of reinforcement than were within-bout responses across all experiments. In contrast, in the absence of a concurrent task, human schedule performance did not always display characteristics of nonhuman performance, but tended to be related to the relationship between rates of responding and reinforcement (feedback function), irrespective of the schedule of reinforcement employed. This was also true of within-bout responding, but not bout-initiations, which were not affected by the presence of a concurrent task. These data suggest the existence of two strategies for human responding on free-operant schedules, relatively mechanistic ones that apply to bout-initiation, and relatively explicit ones, that tend to apply to within-bout responding, and dominate human performance when other demands are not made on resources.

A computational formulation of the behavior systems account of the temporal organization of motivated behavior

The behavior systems framework suggests that motivated behavior-e.g., seeking food and mates, avoiding predators-consists of sequences of actions organized within nested behavioral states. This framework has bridged behavioral ecology and experimental psychology, providing key insights into critical behavioral processes. In particular, the behavior systems framework entails a particular organization of behavior over time. The present paper examines whether such organization emerges from a generic Markov process, where the current behavioral state determines the probability distribution of subsequent behavioral states. This proposition is developed as a systematic examination of increasingly complex Markov models, seeking a computational formulation that balances adherence to the behavior systems approach, parsimony, and conformity to data. As a result of this exercise, a nonstationary partially hidden Markov model is selected as a computational formulation of the predatory subsystem. It is noted that the temporal distribution of discrete responses may further unveil the structure and parameters of the model but, without proper mathematical modeling, these discrete responses may be misleading. Opportunities for further elaboration of the proposed computational formulation are identified, including developments in its architecture, extensions to defensive and reproductive subsystems, and methodological refinements.

An Information Theoretic Approach to Model Selection: A Tutorial with Monte Carlo Confirmation

A reliance on null hypothesis significance testing (NHST) and misinterpretations of its results are thought to contribute to the replication crisis while impeding the development of a cumulative science. One solution is a data-analytic approach called Information-Theoretic (I-T) Model Selection, which builds upon Maximum Likelihood estimates. In the I-T approach, the scientist examines a set of candidate models and determines for each one the probability that it is the closer to the truth than all others in the set. Although the theoretical development is subtle, the implementation of I-T analysis is straightforward. Models are sorted according to the probability that they are the best in light of the data collected. It encourages the examination of multiple models, something investigators desire and that NHST discourages. This article is structured to address two objectives. The first is to illustrate the application of I-T data analysis to data from a virtual experiment. A noisy delay-discounting data set is generated and seven quantitative models are examined. In the illustration, it is demonstrated that it is not necessary to know the "truth" is to identify the one that is closest to it and that the most likely models conform to the model that generated the data. Second, we examine claims made by advocates of the I-T approach using Monte Carlo simulations in which 10,000 different data sets are generated and analyzed. The simulations showed that 1) the probabilities associated with each model returned by the single virtual experiment approximated those that resulted from the simulations, 2) models that were deemed close to the truth produced the most precise parameter estimates, and 3) adding a single replicate sharpens the ability to identify the most probable model.

Modeling bout-pause response patterns in variable-ratio and variable-interval schedules using hierarchical Bayesian methodology

Streams of operant responses are arranged in bouts separated by pauses and differences in performance in reinforcement schedules with identical inter-reinforcement intervals (IRIs) are primarily due to differences in within-bout response rate, not in bout-initiation rate. The present study used hierarchical Bayesian modeling as a new method to quantify the properties of the response bout. A Bernoulli distribution was utilized to express the probability to stay in bout/pause, while a Poisson distribution was utilized to quantify the within-bout response rates. We compared bout/pause patterns between variable-ratio (VR) and variable-interval (VI) schedules across IRIs. The model estimation revealed no difference in within-bout staying probability between schedules. However, response rates of within-bout responses were higher in VR than VI across IRIs. These results are consistent with previous analyses using a log-survivor plot to describe within-bout responses and bouts-initiation responses. In addition, a simulation study was performed to examine how sensitively the model estimate the parameters according to different bout initiation rates. These result showed that the within-bout staying probability was affected by changes in between-bout while within-bout response rate parameters were not. This suggests model estimation robustness of the model estimation to dissociate within-bout and between-bout parameters during different reinforcement schedules.

Proprioceptive stimuli and habit formation: Interresponse time mediated behavior in CD-1 mice

The consolidation of behavioral sequences into relatively ballistic habits is thought to involve the formation of stimulus - response associations. Typically, the stimuli in these associations are assumed to be exteroceptive, i.e., external to the organism. However, responses, themselves, also possess stimulus properties that can mediate behavior. Indeed, these "proprioceptive cues" have long been hypothesized to underlie habit formation (Hull, 1934a, 1934b). One such stimulus involves the time durations between responses - a stimulus termed interresponse time (IRT). We hypothesize that IRTs can come to serve as stimuli that differentially control response elements during habit formation. To examine this hypothesis we report on two experiments that asked whether CD-1 mice utilize IRTs to structure behavior in a two-choice environment. In experiment 1, eight mice were exposed to a free-operant concurrent variable-interval (VI) 30-s VI 60-s reinforcement schedule. We found that switch and stay responses were differentially correlated with IRT durations. In Experiment 2 we directly and differentially reinforced stay/switch responses based on IRT durations in a two-lever procedure. For four of the subjects, the probability of receiving reinforcement after switch responses was proportional to IRT duration. For five of the subjects, these reinforcement probabilities were inversely proportional to IRT duration. Regardless, all of our subjects learned to emit IRT-mediated switching behavior that matched the reinforcement contingencies. Together, Experiments 1 and 2 provide the first evidence of which we are aware that IRTs can come to control sequential choice behavior in mice.

A bout analysis of operant response disruption

Operant behavior appears to be organized in bouts of responses, whose parameters are differentially sensitive to various manipulations. This study investigated potential differential effects of three forms of operant response disruption-extinction (EXT), non-contingent reinforcement (NCR), and prefeeding (PRE)-on response bouts. In Experiment 1, Wistar Kyoto rats (WKY) were trained on a tandem variable-time (VT) 120s fixed-ratio (FR) 5 schedule of reinforcement; after stability was established, their responding was disrupted for three sessions with one of the three disrupters (EXT, NCR, or PRE). In Experiment 2, Long Evans (LE) rats were trained on a tandem VT 240s FR 5 to stability, and their responding disrupted with EXT or NCR. In EXT and NCR, response rates declined significantly and progressively over the course of the session, primarily due to a declining bout-initiation rate in EXT, and to fewer responses per bout in NCR. In contrast, a session-wide drop in response rate was observed in PRE, primarily due to a reduction in bout-initiation rate at the start of the session. These findings suggest that each form of disruption differentially impacts dissociable aspects of behavior. Theories of behavioral persistence should account for these functional relations, which appear to be obscured in response rate measures.

Inter-response-time reinforcement and relative reinforcer frequency control choice

In Conditions 1 and 3 of our Experiment 1, rats pressed levers for food in a two-component multiple schedule. The first component was concurrent variable-ratio (VR) 20 variable-interval (VI) 90 s, and the second was concurrent yoked VI (its reinforcement rate equaled that of the prior component's VR) VI 90 s. In Condition 2, the VR was changed to tandem VR 20, differential reinforcement of low rates (DRL) 0.8 s. Local response rates were higher in the VR than in the yoked VI schedule, and this difference disappeared between tandem VR DRL and yoked VI. The relative time allocations to VR and yoked VI, as well as to tandem VR DRL and yoked VI, were approximately the same across conditions. In Experiment 2, rats chose in a single session between five different VI pairs, each lasting for 12 reinforcer presentations (variable-environment procedure). The across-schedule hourly reinforcement rates were 120 and 40, respectively, in Conditions 1-3 and 4-6. During Conditions 2 and 5, one lever's VI was changed to tandem VI, DRL 2 s. High covariation between relative time allocations and relative reinforcer frequencies, as well as invariance in local response rates to the schedules, was evident in all conditions. In addition, the relative local response rates were biased toward the unchanged VI in Conditions 2 and 5. These results demonstrate two-process control of choice: Inter-response-time reinforcement controls the local response rate, and relative reinforcer frequency controls relative time allocations.

A bout analysis reveals age-related methylmercury neurotoxicity and nimodipine neuroprotection

Age-related deficits in motor and cognitive functioning may be driven by perturbations in calcium (Ca(2+)) homeostasis in nerve terminals, mechanisms that are also thought to mediate the neurotoxicity of methylmercury (MeHg). Calcium-channel blockers (CCBs) protect against MeHg toxicity in adult mice, but little is known about their efficacy in other age groups. Two age groups of BALB/c mice were exposed to 0 or 1.2mg/kg/day MeHg and 0 or 20mg/kg/day of the CCB nimodipine for approximately 8.5 months. Adults began exposure on postnatal day (PND) 72 and the retired breeders on PND 296. High-rate operant behavior was maintained under a percentile schedule, which helped to decouple response rate from reinforcer rate. Responding was analyzed using a log-survivor bout analysis approach that partitioned behavior into high-rate bouts separated by pauses. MeHg-induced mortality did not depend on age but nimodipine neuroprotection was age-dependent, with poorer protection occurring in older mice. Within-bout response rate (a marker of sensorimotor function) was more sensitive to MeHg toxicity than bout-initiation rate (a marker of motivation). Within-bout rate declined almost 2 months prior to overt signs of toxicity for the MeHg-only retired breeders but not adults, suggesting greater delay to toxicity in younger animals. Motor-based decrements also appeared in relatively healthy adult MeHg+NIM animals. Aging appeared to alter the processes underlying Ca(2+) homeostasis thereby diminishing protection by nimodipine, even in mice that have not reached senescence. The study of MeHg exposure presents an experimental model by which to study potential mechanisms of aging.

Quantitative analysis of performance on a progressive-ratio schedule: effects of reinforcer type, food deprivation and acute treatment with Δ⁹-tetrahydrocannabinol (THC)

Rats' performance on a progressive-ratio schedule maintained by sucrose (0.6M, 50 μl) and corn oil (100%, 25 μl) reinforcers was assessed using a model derived from Killeen's (1994) theory of schedule-controlled behaviour, 'Mathematical Principles of Reinforcement'. When the rats were maintained at 80% of their free-feeding body weights, the parameter expressing incentive value, a, was greater for the corn oil than for the sucrose reinforcer; the response-time parameter, δ, did not differ between the reinforcer types, but a parameter derived from the linear waiting principle (T0), indicated that the minimum post-reinforcement pause was longer for corn oil than for sucrose. When the rats were maintained under free-feeding conditions, a was reduced, indicating a reduction of incentive value, but δ was unaltered. Under the food-deprived condition, the CB1 cannabinoid receptor agonist Δ(9)-tetrahydrocannabinol (THC: 0.3, 1 and 3 mg kg(-1)) increased the value of a for sucrose but not for corn oil, suggesting a selective enhancement of the incentive value of sucrose; none of the other parameters was affected by THC. The results provide new information about the sensitivity of the model's parameters to deprivation and reinforcer quality, and suggest that THC selectively enhances the incentive value of sucrose.