Some parameters of behavioral contrast and allocation of interim behavior in rats

A Computational Model of Dual Competition between the Basal Ganglia and the Cortex

We propose a model that includes interactions between the cortex, the basal ganglia (BG), and the thalamus based on a dual competition. We hypothesize that the striatum, the subthalamic nucleus (STN), the internal globus pallidus (GPi), the thalamus, and the cortex are involved in closed feedback loops through the hyperdirect and direct pathways. These loops support a competition process that results in the ability of BG to make a cognitive decision followed by a motor one. Considering lateral cortical interactions, another competition takes place inside the cortex allowing the latter to make a cognitive and a motor decision. We show how this dual competition endows the model with two regimes. One is driven by reinforcement learning and the other by Hebbian learning. The final decision is made according to a combination of these two mechanisms with a gradual transfer from the former to the latter. We confirmed these theoretical results on primates (Macaca mulatta) using a novel paradigm predicted by the model.

A natural history of skills

The dorsal pallium (a.k.a. cortex in mammals) makes a loop circuit with the basal ganglia and the thalamus known to control and adapt behavior but the who's who of the functional roles of these structures is still debated. Influenced by the Triune brain theory that was proposed in the early sixties, many current theories propose a hierarchical organization on the top of which stands the cortex to which the subcortical structures are subordinated. In particular, habits formation has been proposed to reflect a switch from conscious on-line control of behavior by the cortex, to a fully automated subcortical control. In this review, we propose to revalue the function of the network in light of the current experimental evidence concerning the anatomy and physiology of the basal ganglia-cortical circuits in vertebrates. We briefly review the current theories and show that they could be encompassed in a broader framework of skill learning and performance. Then, after reminding the state of the art concerning the anatomical architecture of the network and the underlying dynamic processes, we summarize the evolution of the anatomical and physiological substrate of skill learning and performance among vertebrates. We then review experimental evidence supporting for the hypothesis that the development of automatized skills relies on the BG teaching cortical circuits and is actually a late feature linked with the development of a specialized cortex or pallium that evolved in parallel in different taxa. We finally propose a minimal computational framework where this hypothesis can be explicitly implemented and tested.

The globus pallidus pars interna in goal-oriented and routine behaviors: Resolving a long-standing paradox

BACKGROUND

There is an apparent contradiction between experimental data showing that the basal ganglia are involved in goal-oriented and routine behaviors and clinical observations. Lesion or disruption by deep brain stimulation of the globus pallidus interna has been used for various therapeutic purposes ranging from the improvement of dystonia to the treatment of Tourette's syndrome. None of these approaches has reported any severe impairment in goal-oriented or automatic movement.

METHOD

To solve this conundrum, we trained 2 monkeys to perform a variant of a 2-armed bandit-task (with different reward contingencies). In the latter we alternated blocks of trials with choices between familiar rewarded targets that elicit routine behavior and blocks with novel pairs of targets that require an intentional learning process.

RESULTS

Bilateral inactivation of the globus pallidus interna, by injection of muscimol, prevents animals from learning new contingencies while performance remains intact, although slower for the familiar stimuli. We replicate in silico these data by adding lateral competition and Hebbian learning in the cortical layer of the theoretical model of the cortex-basal ganglia loop that provided the framework of our experimental approach.

CONCLUSION

The basal ganglia play a critical role in the deliberative process that underlies learning but are not necessary for the expression of routine movements. Our approach predicts that after pallidotomy or during stimulation, patients should have difficulty with complex decision-making processes or learning new goal-oriented behaviors. © 2016 Movement Disorder Society.

Behavioral contrast in competitive and noncompetitive environments

Three experiments examined the effects of opportunities for an alternative response (drinking) on positive behavioral contrast of rats' food-reinforced bar pressing. In both Experiments 1 and 2 the baseline multiple variable-interval schedules were rich (variable interval 10-s), and contrast was examined both with and without a water bottle present. In Experiment 1, the rats were not water deprived. When one component of the multiple schedule was changed to extinction, the rate of bar pressing increased in the constant component (positive behavioral contrast). The magnitude of contrast was larger when the bottle was absent than when it was present, as predicted by the matching law. Drinking did not shift from the constant variable-interval component to the extinction component, as might have been expected from competition theory. In Experiment 2, the rats were water deprived. Contrast was larger when the bottle was present than when it was absent, and drinking did shift to the extinction component, as predicted by competition theory. In Experiment 3, water-deprived rats responded on leaner multiple variable-interval schedules (60-s) in the presence of a water bottle. When one component was changed to extinction, contrast did not occur, and drinking did not shift to the extinction component. The present results suggest that there are at least two different sources of behavioral contrast: "competitive" contrast, observed when an alternative response occurs with high probability, and "noncompetitive" contrast, observed when an alternative response occurs with low probability. The results, in conjunction with earlier studies, also suggest that the form of the alternative response and the rate of food reinforcement provided by the multiple schedule combine to determine the amount of contrast.

Contrast and reallocation of extraneous reinforcers between multiple-schedule components

Four pigeons responded in components of multiple schedules in which two responses were available and reinforced with food. Pecks on the left key ("main" key) were reinforced at a constant rate in one component and at a rate that varied over conditions in the other component. When reinforcer rate was varied, behavioral contrast occurred in the constant component. On the right key ("extra" key), five variable-interval schedules and one variable-ratio schedule, presented conjointly, arranged reinforcers for responses in all conditions. These conjoint schedules were common to both multiple-schedule components-rather than unique to particular components-and reinforcers from these schedules could therefore be arranged in one component and obtained during the other component. In this way, the additional reinforcers were analogous to the "extraneous" reinforcers thought to maintain behavior other than pecking in conventional multiple schedules. Response rate on the extra key did not change systematically over conditions in the constant component, and in the varied component extra responding was inversely related to main-key reinforcement. All subjects obtained more extra-key reinforcers in whichever component arranged fewer main-key reinforcers. Consistent with the theory that reallocation of extraneous reinforcers may cause behavioral contrast, absolute reinforcer rate for the extra key in the constant component was low in conditions that produced positive contrast on the main key and high in those that produced negative contrast. Also consistent with this theory, behavioral contrast was reduced in two conditions that canceled extra-key reinforcers that had been arranged but not obtained at the end of components. Thus, a constraint on reallocation markedly reduced the extent of contrast.

Stimulus- and pellet-induced drinking during a successive discrimination

In three experiments, interim water drinking was examined in rats exposed to a multiple schedule whose two components were extinction and a variable-time 30-s schedule of food delivery. Two different drinking patterns were observed in Experiment 1. Pellet-induced drinking, characterized by high rates of postpellet drinking in the variable-time component, with little or no drinking in extinction, occurred when the acquisition of stable postpellet drinking preceded discrimination training. Stimulus-induced drinking, characterized by a burst of drinking at the onset of extinction, with no drinking during the variable-time schedule, occurred when discrimination training preceded all other experimental conditions. With extended training, stimulus-induced drinking eventually was accompanied by postpellet drinking. In Experiment 2, the rate of stimulus-induced drinking and the number of sessions during which it occurred without postpellet drinking were found to be inversely related to component duration. In Experiment 3, the rate of schedule-induced drinking was found to vary directly with component duration.

Varying reinforcer duration produces behavioral interactions during multiple schedules

The experiments tested the idea that changes in habituation to the reinforcer contribute to behavioral interactions during multiple schedules. This idea predicts that changing an aspect of the reinforcer should disrupt habituation and produce an interaction. Pigeons and rats responded on multiple variable interval variable interval schedules. Introducing variability into the duration of reinforcers in one component increased response rates in both components when the schedules provided high, but not low, rates of reinforcement. The increases in constant-component response rates grew larger as the session progressed. Within-session decreases in responding were smaller when the other component provided variable-, rather than fixed-, duration reinforcers. These results are consistent with the idea that changes in habituation to the reinforcer contribute to behavioral interactions. They help to explain why interactions do not occur for some subjects under conditions that produce them for others. Finally, the results question the assumption that induction and behavioral contrast are always produced by different theoretical mechanisms.

Dishabituation with component transitions may contribute to the interactions observed during multiple schedules

Rates of responding by rats were usually higher during the variable interval (VI) 30-s component of a multiple VI 30-s fixed interval (FI) 30-s schedule than during the same component of a multiple VI 30-s VI 30-s schedule (Experiment 1). Response rates were also usually higher during the FI 30-s component of a multiple VI 30-s FI 30-s schedule than during the same component of a multiple FI 30-s FI 30-s schedule (Experiment 2). The differences in response rates were not observed when the components provided VI or FI 120-s schedules. These results were predicted by the idea that differences in habituation to the reinforcer between multiple schedules contribute to behavioral interactions, such as behavioral contrast. However, differences in habituation were not apparent in the within-session patterns of responding. Finding differences in response rates in both experiments violates widely-held assumptions about behavioral interactions, including that behavioral contrast does not occur for rats and that improving the conditions of reinforcement decreases, rather than increases, response rate in the alternative component.

Contrast and undermatching with regular or irregular alternation of components

Behavioral contrast and response-ratio sensitivity to reinforcement were compared in multiple schedules in which components alternated strictly or according to a pseudorandom sequence. Average component durations in the two regimes were always 60 s, and order of presentation of component alternation regimes was counterbalanced across subjects. In Part 1, the reinforcer rate in one component was reduced from 60 per hour to zero, while that in the other component was unchanged. Positive behavioral contrast occurred in the constant component in that response rates increased, but neither the reliability nor the magnitude of contrast was affected by the manner in which components alternated. Part 2 was similar, except that a number of different reinforcer rates were used in the varied component. Neither contrast nor sensitivity of response ratios to changes in reinforcer ratios depended on the regime of component alternation. Thus, the predictability in time of future reinforcement conditions, which is a feature of regular multiple scheduling, does not appear to be a determinant of multiple-schedule performance.

Positive behavioral contrast across food and alcohol reinforcers

The present study examined behavioral contrast during concurrent and multiple schedules that provided food and alcohol reinforcers. Concurrent-schedule contrast occurred in the responding reinforced by food when alcohol reinforcers were removed. It also occurred in the responding reinforced by alcohol when food was removed. Multiple-schedule contrast appeared for food when alcohol reinforcers were removed, but not for alcohol when food was removed. These results show that behavioral contrast may, but does not always, occur across qualitatively different reinforcers. They also show that multiple-schedule contrast may be more difficult to produce than concurrent-schedule contrast. The results have implications for a model of alcohol consumption.