Still at the Choice-Point: Action Selection and Initiation in Instrumental Conditioning

Stimulus awareness is necessary for both instrumental learning and instrumental responding to previously learned stimuli

Instrumental conditioning is a crucial part of adaptive behaviour, allowing agents to selectively interact with stimuli in their environment. Recent evidence suggests that instrumental conditioning cannot proceed without stimulus awareness. However, whether accurate unconscious instrumental responding can emerge from consciously acquired knowledge of the stimulus-action-outcome contingencies is unknown. We studied this question using instrumental trace conditioning, where participants learned to make approach/avoid decisions in two within-subject modes: conscious (stimuli in plain view) and unconscious (visually masked). Both tasks were followed by an unconscious-only instrumental performance task. We show that even when the contingencies are reliably learned in the conscious mode, participants fail to act upon them in the unconscious responding task. We also replicate the previous results that no instrumental learning occurs in the unconscious mode. Consequently, the absence of stimulus awareness not only precludes instrumental conditioning, but also precludes any kind of instrumental responding to already known stimuli. This suggests that instrumental behaviour is entirely supported by conscious awareness of the world, and corroborates the proposals that consciousness may be necessary for adaptive behaviours requiring selective action.

What Does the Human Olfactory System Do, and How Does It Do It?

Historically, the human sense of smell has been regarded as the odd stepchild of the senses, especially compared to the sensory bravado of seeing, touching, and hearing. The idea that the human olfaction has little to contribute to our experience of the world is commonplace, though with the emergence of COVID-19 there has rather been a sea change in this understanding. An ever increasing body of work has convincingly highlighted the keen capabilities of the human nose and the sophistication of the human olfactory system. Here, we provide a concise overview of the neuroscience of human olfaction spanning the last 10-15 years, with focus on the peripheral and central mechanisms that underlie how odor information is processed, packaged, parceled, predicted, and perturbed to serve odor-guided behaviors. We conclude by offering some guideposts for harnessing the next decade of olfactory research in all its shapes and forms.

The Motivational Determinants of Human Action, Their Neural Bases and Functional Impact in Adolescents With Obsessive-Compulsive Disorder

Background

Establishing the motivational influences on human action is essential for understanding choice and decision making in health and disease. Here we used tests of value-based decision making, manipulating both predicted and experienced reward values to assess the motivational control of goal-directed action in healthy adolescents and those with obsessive-compulsive disorder (OCD).

Methods

After instrumental training on a two action-two outcome probabilistic task, adolescents (n = 21) underwent Pavlovian conditioning using distinct stimuli predicting either the instrumental outcomes, a third outcome, or nothing. We then assessed functional magnetic resonance imaging during choice tests in which we varied the predicted value, using specific and general Pavlovian-instrumental transfer, and the experienced value, using outcome devaluation. To establish functional significance, we tested a matched cohort of adolescents with OCD (n = 20).

Results

In healthy adolescents, both predicted and experienced values influenced the performance of goal-directed actions, mediated by distinct orbitofrontal-striatal circuits involving the lateral orbitofrontal cortex (OFC) and medial OFC, respectively. However, in adolescents with OCD, choice was insensitive to changes in either predicted or experienced values. These impairments were related to hypoactivity in the lateral OFC and hyperactivity in the medial OFC during specific Pavlovian-instrumental transfer and hypoactivity in the anterior prefrontal cortex, caudate nucleus, and their connectivity in the devaluation test.

Conclusions

We found that predicted and experienced values exerted a potent influence on the performance of goal-directed actions in adolescents via distinct orbitofrontal- and prefrontal-striatal circuits. Furthermore, the influence of these motivational processes was severely blunted in OCD, as was the functional segregation of circuits involving medial and lateral OFC, producing dysregulated action control.

Revealing abrupt transitions from goal-directed to habitual behavior

A fundamental tenet of animal behavior is that decision-making involves multiple 'controllers.' Initially, behavior is goal-directed, driven by desired outcomes, shifting later to habitual control, where cues trigger actions independent of motivational state. Clark Hull's question from 1943 still resonates today: "Is this transition abrupt, or is it gradual and progressive?"1 Despite a century-long belief in gradual transitions, this question remains unanswered2,3 as current methods cannot disambiguate goal-directed versus habitual control in real-time. Here, we introduce a novel 'volitional engagement' approach, motivating animals by palatability rather than biological need. Offering less palatable water in the home cage4,5 reduced motivation to 'work' for plain water in an auditory discrimination task when compared to water-restricted animals. Using quantitative behavior and computational modeling6, we found that palatability-driven animals learned to discriminate as quickly as water-restricted animals but exhibited state-like fluctuations when responding to the reward-predicting cue-reflecting goal-directed behavior. These fluctuations spontaneously and abruptly ceased after thousands of trials, with animals now always responding to the reward-predicting cue. In line with habitual control, post-transition behavior displayed motor automaticity, decreased error sensitivity (assessed via pupillary responses), and insensitivity to outcome devaluation. Bilateral lesions of the habit-related dorsolateral striatum7 blocked transitions to habitual behavior. Thus, 'volitional engagement' reveals spontaneous and abrupt transitions from goal-directed to habitual behavior, suggesting the involvement of a higher-level process that arbitrates between the two.

Serotonin and Dopamine Show Different Response Profiles to Acute Stress in the Nucleus Accumbens and Medial Prefrontal Cortex of Rats with Neuropathic Pain

The ability to adaptively guide behaviour requires the integration of external information with internal motivational factors. Decision-making capabilities can be impaired by acute stress and is often exacerbated by chronic pain. Chronic neuropathic pain patients often present with cognitive dysfunction, including impaired decision-making. The mechanisms underlying these changes are not well understood but may include altered monoaminergic transmission in the brain. In this study we investigated the relationships between dopamine, serotonin, and their metabolites in key brain regions that regulate motivated behaviour and decision-making. The neurochemical profiles of the medial prefrontal cortex, orbital prefrontal cortex, and nucleus accumbens were analysed using HPLC in rats that received a chronic constriction injury (CCI) of the right sciatic nerve and an acute stress (15-min restraint), prior to an outcome devaluation task. CCI alone significantly decreased dopamine but not serotonin concentrations in the medial prefrontal cortex. By contrast, restraint stress acutely increased dopamine in the medial prefrontal cortex, and the nucleus accumbens; and increased serotonin in the medial prefrontal cortex 2 h later. The sustained dopaminergic and serotonergic responses to acute stress highlight the importance of an animal's ability to mount an effective coping response. In addition, these data suggest that the impact of nerve injury and acute stress on outcome-devaluation occurs independently of dopaminergic and serotonergic transmission in the medial prefrontal cortex, orbital prefrontal cortex and nucleus accumbens of rats.

Mechanisms underlying performance in a cued go/no-go Pavlovian-to-instrumental transfer paradigm

Environmental cues that remind us of rewarding outcomes (drugs, food) play a significant role in addiction relapse. In the lab the Pavlovian-to-instrumental transfer (PIT) task has been used to formally examine how cues associated with reward or punishment can bias ongoing instrumental responding. Using a version of this paradigm that integrates PIT with a go/no-go task many studies have related stronger PIT effects (with non-drug rewards) to problematic alcohol use including risky alcohol users relative to non-risky drinkers, individuals with alcohol dependence versus healthy controls and individuals in recovery from alcohol use disorder who are more likely to relapse. However the theoretical importance of these findings and the implications for models of addiction was previously not clear. Understanding if this task indexes the general motivating effects of reward cues on instrumental responding (and whether this is sensitive to shifts in motivation for those outcomes) is critical for understanding these previous results within the context of addiction. Thus, in the current study we aimed to delineate the associative mechanisms that drive the stimulus effects observed in this PIT task. Specifically, we wished to examine whether the cueing effects observed in the cued-go/no-go task were selective in their effect on action, insofar as Pavlovian cues specifically invigorated (or suppressed) responding only if they were associated with congruent outcomes. We conclude that the PIT measured with this task is general in nature. Surprisingly however, the biasing effects of Pavlovian cues on instrumental responding did not appear to be sensitive to outcome devaluation.

Neurocomputational mechanisms of young children's observational learning of delayed gratification

The ability to delay gratification is crucial for a successful and healthy life. An effective way for young children to learn this ability is to observe the action of adult models. However, the underlying neurocomputational mechanism remains unknown. Here, we tested the hypotheses that children employed either the simple imitation strategy or the goal-inference strategy when learning from adult models in a high-uncertainty context. Results of computational modeling indicated that children used the goal-inference strategy regardless of whether the adult model was their mother or a stranger. At the neural level, results showed that successful learning of delayed gratification was associated with enhanced interpersonal neural synchronization (INS) between children and the adult models in the dorsal lateral prefrontal cortex but was not associated with children's own single-brain activity. Moreover, the discounting of future reward's value obtained from computational modeling of the goal-inference strategy was positively correlated with the strength of INS. These findings from our exploratory study suggest that, even for 3-year-olds, the goal-inference strategy is used to learn delayed gratification from adult models, and the learning strategy is associated with neural interaction between the brains of children and adult models.

Sensory-Specific Satiety Dissociates General and Specific Pavlovian-Instrumental Transfer

Pavlovian conditioning enables predictive stimuli to control action performance and action selection. The present experiments used sensory-specific satiety to examine the role of outcome value in these two forms of control. Experiment 1 employed a general Pavlovian-instrumental transfer design to show that a stimulus predicting a food outcome energizes the performance of an instrumental action earning another food outcome. This energizing effect was removed when the stimulus-predicted outcome or a novel outcome was devalued by sensory-specific satiety. Experiments 2 and 3 employed a specific Pavlovian-instrumental transfer design to demonstrate that a stimulus predicting a particular food outcome promotes the selection of an instrumental action earning the same, but not a different, food outcome. Remarkably, this effect was maintained when all or just one of the stimulus-predicted outcomes were devalued by sensory-specific satiety. These results indicate that satiety alone removes the expression of general PIT. By contrast, satiety or outcome-specific devaluation does not regulate the expression of specific PIT, which is insensitive to changes in outcome value. This dissociation is consistent with the view that general and specific PIT are two separate phenomena driven by distinct psychological mechanisms.

Animal models of action control and cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) has historically been considered a motor disorder induced by a loss of dopaminergic neurons in the substantia nigra pars compacta. More recently, it has been recognized to have significant non-motor symptoms, most prominently cognitive symptoms associated with a dysexecutive syndrome. It is common in the literature to see motor and cognitive symptoms treated separately and, indeed, there has been a general call for specialized treatment of the latter, particularly in the more severe cases of PD with mild cognitive impairment and dementia. Animal studies have similarly been developed to model the motor or non-motor symptoms. Nevertheless, considerable research has established that segregating consideration of cognition from the precursors to motor movement, particularly movement associated with goal-directed action, is difficult if not impossible. Indeed, on some contemporary views cognition is embodied in action control, something that is particularly prevalent in theory and evidence relating to the integration of goal-directed and habitual control processes. The current paper addresses these issues within the literature detailing animal models of cognitive dysfunction in PD and their neural and neurochemical bases. Generally, studies using animal models of PD provide some of the clearest evidence for the integration of these action control processes at multiple levels of analysis and imply that consideration of this integrative process may have significant benefits for developing new approaches to the treatment of PD.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Parallel and hierarchical neural mechanisms for adaptive and predictive behavioral control

Our brain can be recognized as a network of largely hierarchically organized neural circuits that operate to control specific functions, but when acting in parallel, enable the performance of complex and simultaneous behaviors. Indeed, many of our daily actions require concurrent information processing in sensorimotor, associative, and limbic circuits that are dynamically and hierarchically modulated by sensory information and previous learning. This organization of information processing in biological organisms has served as a major inspiration for artificial intelligence and has helped to create in silico systems capable of matching or even outperforming humans in several specific tasks, including visual recognition and strategy-based games. However, the development of human-like robots that are able to move as quickly as humans and respond flexibly in various situations remains a major challenge and indicates an area where further use of parallel and hierarchical architectures may hold promise. In this article we review several important neural and behavioral mechanisms organizing hierarchical and predictive processing for the acquisition and realization of flexible behavioral control. Then, inspired by the organizational features of brain circuits, we introduce a multi-timescale parallel and hierarchical learning framework for the realization of versatile and agile movement in humanoid robots.

Er81 Transcription Factor Fine-Tunes Striatal Cholinergic Interneuron Activity and Drives Habit Formation

The molecular mechanisms tuning cholinergic interneuron (CIN) activity, although crucial for striatal function and behavior, remain largely unexplored. Previous studies report that the Etv1/Er81 transcription factor is vital for regulating neuronal maturation and activity. While Er81 is known to be expressed in the striatum during development, its specific role in defining CIN properties and the resulting consequences on striatal function is unknown. We report here that Er81 is expressed in CINs and its specific ablation leads to prominent changes in their molecular, morphologic, and electrophysiological features. In particular, the lack of Er81 amplifies intrinsic delayed-rectifier and hyperpolarization-activated currents, which subsequently alters the tonic and phasic activity of CINs. We further reveal that Er81 expression is required for normal CIN pause and time-locked responses to sensorimotor inputs in awake mice. Overall, this study uncovers a new cell type-specific control of CIN function in the striatum which drives habit formation in adult male mice.SIGNIFICANCE STATEMENT Although previous studies have shown that cholinergic interneurons drive striatal activity and habit formation, the underlying molecular mechanisms controlling their function are unknown. Here we reveal that key cholinergic interneuron physiological properties are controlled by Er81, a transcription factor regulating neuronal activity and development in a cell-specific manner. Moreover, our findings uncover a link between the Er81-dependent molecular control of cholinergic interneuron function and habit formation in mice. These insights will contribute to the future enhancement of our understanding of disorders that involve behavioral inflexibility, such as autism and addiction.

How predictive learning influences choice: Evidence for a GPCR-based memory process necessary for Pavlovian-instrumental transfer

Predictive learning endows stimuli with the capacity to signal both the sensory-specific and general motivational properties of their associated rewards or outcomes. These two signals can be distinguished behaviorally by their influence on the selection and performance of instrumental actions, respectively. This review focuses on how sensory-specific predictive learning guides choice between actions that earn otherwise equally desirable outcomes. We describe evidence that outcome-specific predictive learning is encoded in the basolateral amygdala and drives the accumulation of delta-opioid receptors on the surface of cholinergic interneurons located in the nucleus accumbens shell. This accumulation constitutes a novel form of cellular memory, not for outcome-specific predictive learning per se but for the selection of, and choice between, future instrumental actions. We describe recent evidence regarding the cascade of events necessary for the formation and expression of this cellular memory and point to open questions for future research into this process. Beyond these mechanistic considerations, the discovery of this new form of memory is consistent with recent evidence suggesting that intracellular rather than synaptic changes can mediate learning-related plasticity to modify brain circuitry to prepare for future significant events.

Egr2 induction in spiny projection neurons of the ventrolateral striatum contributes to cocaine place preference in mice

Drug addiction develops due to brain-wide plasticity within neuronal ensembles, mediated by dynamic gene expression. Though the most common approach to identify such ensembles relies on immediate early gene expression, little is known of how the activity of these genes is linked to modified behavior observed following repeated drug exposure. To address this gap, we present a broad-to-specific approach, beginning with a comprehensive investigation of brain-wide cocaine-driven gene expression, through the description of dynamic spatial patterns of gene induction in subregions of the striatum, and finally address functionality of region-specific gene induction in the development of cocaine preference. Our findings reveal differential cell-type specific dynamic transcriptional recruitment patterns within two subdomains of the dorsal striatum following repeated cocaine exposure. Furthermore, we demonstrate that induction of the IEG Egr2 in the ventrolateral striatum, as well as the cells within which it is expressed, are required for the development of cocaine seeking.

The human brain is ever changing, constantly rewiring itself in response to new experiences, knowledge or information from the environment. Addictive drugs such as cocaine can hijack the genetic mechanisms responsible for this plasticity, creating dangerous, obsessive drug-seeking and consuming behaviors.

Cocaine-induced plasticity is difficult to apprehend, however, as brain regions or even cell populations can react differently to the compound. For instance, sub-regions in the striatum – the brain area that responds to rewards and helps to plan movement – show distinct responses during progressive exposure to cocaine. And while researchers know that the drug immediately changes how neurons switch certain genes on and off, it is still unclear how these genetic modifications later affect behavior.

Mukherjee, Gonzales et al. explored these questions at different scales, first focusing on how progressive cocaine exposure changed the way various gene programs were activated across the entire brain. This revealed that programs in the striatum were the most affected by the drug.

Examining this region more closely showed that cocaine switches on genes in specific ‘spiny projection’ neuron populations, depending on where these cells are located and the drug history of the mouse. Finally, Mukherjee, Gonzales et al. used genetically modified mice to piece together cocaine exposure, genetic changes and modifications in behavior. These experiments revealed that the drive to seek cocaine depended on activation of the Egr2 gene in populations of spiny projection neurons in a specific sub-region of the striatum. The gene, which codes for a protein that regulates how genes are switched on and off, was itself strongly activated by cocaine intake.

Cocaine addiction can have devastating consequences for individuals. Grasping how this drug alters the brain could pave the way for new treatments, while also providing information on the basic mechanisms underlying brain plasticity.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

The Meaning of Behavior: Discriminating Reflex and Volition in the Brain

The ability to establish behaviorally what psychological capacity an animal is deploying-to discern accurately what an animal is doing-is key to functional analyses of the brain. Our current understanding of these capacities suggests, however, that this task is complex; there is evidence that multiple capacities are engaged simultaneously and contribute independently to the control of behavior. As such, establishing the contribution of a cell, circuit, or neural system to any one function requires careful dissection of that role from its influence on other functions and, therefore, the careful selection and design of behavioral tasks fit for that purpose. Here I describe recent research that has sought to utilize behavioral tools to investigate the neural bases of instrumental conditioning, particularly the circuits and systems supporting the capacity for goal-directed action, as opposed to conditioned reflexes and habits, and how these sources of action control interact to generate adaptive behavior.

Dysfunctional approach behavior triggered by alcohol-unrelated Pavlovian cues predicts long-term relapse in alcohol dependence

We demonstrated that alcohol-dependent patients who relapsed within 1 year after detoxification showed stronger PIT effects compared with abstainers and controls. Relapsers particularly failed to correctly perform in trials where an instrumental stimulus required inhibition while a Pavlovian background cue indicated a monetary gain. Under that condition, relapsers approached the instrumental stimulus, independent of the expected punishment. The failure of inhibiting an aversive stimulus in favor of approaching an appetitive context cue reflects dysfunctional altered learning mechanisms in relapsers.

Hierarchical Action Control: Adaptive Collaboration Between Actions and Habits

It is now commonly accepted that instrumental actions can reflect goal-directed control; i.e., they can show sensitivity to changes in the relationship to and the value of their consequences. With overtraining, stress, neurodegeneration, psychiatric conditions, or after exposure to various drugs of abuse, goal-directed control declines and instrumental actions are performed independently of their consequences. Although this latter insensitivity has been argued to reflect the development of habitual control, the lack of a positive definition of habits has rendered this conclusion controversial. Here we consider various alternative definitions of habit, including recent suggestions they reflect chunked action sequences, to derive criteria with which to categorize responses as habitual. We consider various theories regarding the interaction between goal-directed and habitual controllers and propose a collaborative model based on their hierarchical integration. We argue that this model is consistent with the available data, can be instantiated both at an associative level and computationally and generates interesting predictions regarding the influence of this collaborative integration on behavior.

Loss of striatal tyrosine-hydroxylase interneurons impairs instrumental goal-directed behavior

The striatum mediates a broad range of cognitive and motor functions. Within the striatum, recently discovered tyrosine hydroxylase expressing interneurons (THINs) provide a source of intrastriatal synaptic connectivity that is critical for regulating striatal activity, yet the role of THIN's in behavior remains unknown. Given the important role of the striatum in reward-based behaviors, we investigated whether loss of striatal THINs would impact instrumental behavior in mice. We selectively ablated striatal THINs in TH-Cre mice using chemogenetic techniques, and then tested THIN-lesioned or control mice on three reward-based striatal-dependent instrumental tests: (a) progressive ratio test; (b) choice test following selective-satiety induced outcome devaluation; (c) outcome reinstatement test. Both striatal-THIN-lesioned and control mice acquired an instrumental response for flavored food pellets, and their behavior did not differ in the progressive ratio test, suggesting intact effort to obtain rewards. However, striatal THIN lesions markedly impaired choice performance following selective-satiety induced outcome devaluation. Unlike control mice, THIN-lesioned mice did not adjust their choice of actions following a change in outcome value. In the outcome reinstatement test THIN-lesioned and control mice showed response invigoration by outcome presentation, suggesting the incentive properties of outcomes were not disrupted by THIN lesions. Overall, we found that striatal THIN lesions selectively impaired goal-directed behavior, while preserving motoric and appetitive behaviors. These findings are the first to describe a function of striatal THINs in reward-based behavior, and further illustrate the important role for intrastriatal interneuronal connectivity in behavioral functions ascribed to the striatum more generally.

Individual differences in working memory capacity and cue-guided behavior in humans

Information gathered via Pavlovian and Instrumental learning can be integrated to guide behavior, in a phenomenon experimentally known as Pavlovian-to-Instrumental Transfer (PIT). In particular, in appetitive PIT, a reward-associated cue is able to enhance the instrumental response previously associated with the same (outcome-specific PIT), or a similar (general PIT), reward. The PIT effect is increasingly investigated for its numerous implications in clinical contexts as well as daily life situations. Nevertheless, the precise mechanism behind it is not yet clear. The relation between the PIT effect and high-level cognitive abilities - like working memory - is still unknown, but potentially relevant to unveil its functioning. The present study aims to examine the precise relationship between individual differences in working memory and the two forms of PIT effect, namely outcome-specific and general. For this purpose, 100 participants underwent a classical PIT paradigm. Results showed a relationship between individual working memory and outcome-specific PIT, but not general PIT. Importantly, the role of working memory was not related to the acquisition of the learning contingencies, but rather linked to an imbalance between congruent and incongruent choices. The results are discussed in terms of the adaptive and maladaptive implications for human behavior.

Motor Cortex Excitability Reflects the Subjective Value of Reward and Mediates Its Effects on Incentive-Motivated Performance

Performance-based incentives tend to increase an individual's motivation, resulting in enhancements in behavioral output. While much work has focused on understanding how the brain's reward circuitry influences incentive-motivated performance, fewer studies have investigated how such reward representations act on the motor system. Here we measured motor cortical excitability with transcranial magnetic stimulation while female and male human participants performed a motoric incentive motivation task for prospective monetary gains and losses. We found that individuals' performance increased for increasing prospective gains and losses. While motor cortical excitability appeared insensitive to prospective loss, temporal features of motor cortical excitability for prospective gains were modulated by an independent measure of an individual's subjective preferences for incentive (i.e., loss aversion). Those individuals that were more loss averse had a greater motor cortical sensitivity to prospective gain, closer to movement onset. Critically, behavioral sensitivity to incentive and motor cortical sensitivity to prospective gains were both predicted by loss aversion. Furthermore, causal modeling indicated that motor cortical sensitivity to incentive mediated the relationship between subjective preferences for incentive and behavioral sensitivity to incentive. Together, our findings suggest that motor cortical activity integrates information about the subjective value of reward to invigorate incentive-motivated performance.SIGNIFICANCE STATEMENT Increasing incentives tend to increase motivation and effort. Using a motoric incentive motivation task and transcranial magnetic stimulation, we studied the motor cortical mechanisms responsible for incentive-motivated motor performance. We provide experimental evidence that motor cortical sensitivity to incentive mediates the relationship between subjective preferences for incentive and incentive-motivated performance. These results indicate that, rather than simply being a reflection of motor output, motor cortical physiology integrates information about reward value to motivate performance.

From learning to action: the integration of dorsal striatal input and output pathways in instrumental conditioning

Considerable evidence suggests that the learning and performance of instrumental actions depend on activity in basal ganglia circuitry; however, these two functions have generally been considered independently. Whereas research investigating the associative mechanisms underlying instrumental conditioning has identified critical cortical and limbic input pathways to the dorsal striatum, the performance of instrumental actions has largely been attributed to activity in the dorsal striatal output pathways, with direct and indirect pathway projection neurons mediating action initiation, perseveration and cessation. Here, we discuss evidence that the dorsal striatal input and basal ganglia output pathways mediate the learning and performance of instrumental actions, respectively, with the dorsal striatum functioning as a transition point. From this perspective, the issue of how multiple striatal inputs are integrated at the level of the dorsal striatum and converted into relatively restricted outputs becomes one of critical significance for understanding how learning is translated into action. So too does the question of how learning signals are modulated by recent experience. We propose that this occurs through recurrent corticostriatothalamic feedback circuits that serve to integrate performance signals by updating ongoing action-related learning.

Dorsolateral Striatum Engagement Interferes with Early Discrimination Learning

In current models, learning the relationship between environmental stimuli and the outcomes of actions involves both stimulus-driven and goal-directed systems, mediated in part by the DLS and DMS, respectively. However, though these models emphasize the importance of the DLS in governing actions after extensive experience has accumulated, there is growing evidence of DLS engagement from the onset of training. Here, we used in vivo photosilencing to reveal that DLS recruitment interferes with early touchscreen discrimination learning. We also show that the direct output pathway of the DLS is preferentially recruited and causally involved in early learning and find that silencing the normal contribution of the DLS produces plasticity-related alterations in a PL-DMS circuit. These data provide further evidence suggesting that the DLS is recruited in the construction of stimulus-elicited actions that ultimately automate behavior and liberate cognitive resources for other demands, but with a cost to performance at the outset of learning.

Functionally separated networks for self-paced and externally-cued motor execution in Parkinson's disease: Evidence from deep brain recordings in humans

Spatially segregated cortico-basal ganglia networks have been proposed for the control of goal-directed and habitual behavior. In Parkinson's disease, selective loss of dopaminergic neurons regulating sensorimotor (habitual) behavior might therefore predominantly cause deficits in habitual motor control, whereas control of goal-directed movement is relatively preserved. Following this hypothesis, we examined the electrophysiology of cortico-basal ganglia networks in Parkinson patients emulating habitual and goal-directed motor control during self-paced and externally-cued finger tapping, respectively, while simultaneously recording local field potentials in the subthalamic nucleus (STN) and surface EEG. Only externally-cued movements induced a pro-kinetic event-related beta-desynchronization, whereas beta-oscillations were continuously suppressed during self-paced movements. Connectivity analysis revealed higher synchronicity (phase-locking value) between the STN and central electrodes during self-paced and higher STN to frontal phase-locking during externally-cued movements. Our data provide direct electrophysiological support for the existence of functionally segregated cortico-basal ganglia networks controlling motor behavior in Parkinson patients, and corroborate the assumption of Parkinson patients being shifted from habitual towards goal-directed behavior.

Striatal cholinergic interneurons and Parkinson's disease

Giant, aspiny cholinergic interneurons (ChIs) have long been known to be key nodes in the striatal circuitry controlling goal-directed actions and habits. In recent years, new experimental approaches, like optogenetics and monosynaptic rabies virus mapping, have expanded our understanding of how ChIs contribute to the striatal activity underlying action selection and the interplay of dopaminergic and cholinergic signaling. These approaches also have begun to reveal how ChI function is distorted in disease states affecting the basal ganglia, like Parkinson's disease (PD). This review gives a brief overview of our current understanding of the functional role played by ChIs in striatal physiology and how this changes in PD. The translational implications of these discoveries, as well as the gaps that remain to be bridged, are discussed as well.

Role Played by the Passage of Time in Reversal Learning

Reversal learning is thought to involve an extinction-like process that inhibits the expression of the initial learning. However, behavioral evidence for this inhibition remains difficult to interpret as various procedures have been employed to study reversal learning. Here, we used a discrimination task in rats to examine whether the inhibition produced by reversal learning is as sensitive to the passage of time as the inhibition produced by extinction. Experiment 1 showed that when tested immediately after reversal training, rats were able to use the reversed contingencies to solve the discrimination task in an outcome-specific manner. This ability to use outcome-specific information was lost when a delay was inserted between reversal training and test. However, interpretation of these data was made difficult by a potential floor effect. This concern was addressed in Experiment 2 in which it was confirmed that the passage of time impaired the ability of the rats to use the reversed contingencies in an outcome-specific manner to solve the task. Further, it revealed that the delay between initial learning and test was not responsible for this impairment. Additional work demonstrated that solving the discrimination task was unaffected by Pavlovian extinction but that the discriminative stimuli were able to block conditioning to a novel stimulus, suggesting that Pavlovian processes were likely to contribute to solving the discrimination. We therefore concluded that the expression of reversal and extinction learning do share the same sensitivity to the effect of time. However, this sensitivity was most obvious when we assessed outcome-specific information following reversal learning. This suggests that the processes involved in reversal learning are somehow distinct from those underlying extinction learning, as the latter has usually been found to leave outcome-specific information relatively intact. Thus, the present study reveals that a better understanding of the mechanisms supporting reversal training requires assessing the impact that this training exerts on the content of learning rather than performance per se.

The Lateral Habenula and Its Input to the Rostromedial Tegmental Nucleus Mediates Outcome-Specific Conditioned Inhibition

Animals can readily learn that stimuli predict the absence of specific appetitive outcomes; however, the neural substrates underlying such outcome-specific conditioned inhibition remain largely unexplored. Here, using female and male rats as subjects, we examined the involvement of the lateral habenula (LHb) and of its inputs onto the rostromedial tegmental nucleus (RMTg) in inhibitory learning. In these experiments, we used backward conditioning and contingency reversal to establish outcome-specific conditioned inhibitors for two distinct appetitive outcomes. Then, using the Pavlovian-instrumental transfer paradigm, we assessed the effects of manipulations of the LHb and the LHb-RMTg pathway on that inhibitory encoding. In control animals, we found that an outcome-specific conditioned inhibitor biased choice away from actions delivering that outcome and toward actions earning other outcomes. Importantly, this bias was abolished by both electrolytic lesions of the LHb and selective ablation of LHb neurons using Cre-dependent Caspase3 expression in Cre-expressing neurons projecting to the RMTg. This deficit was specific to conditioned inhibition; an excitatory predictor of a specific outcome-biased choice toward actions delivering the same outcome to a similar degree whether the LHb or the LHb-RMTg network was intact or not. LHb lesions also disrupted the ability of animals to inhibit previously encoded stimulus-outcome contingencies after their reversal, pointing to a critical role of the LHb and of its inputs onto the RMTg in outcome-specific conditioned inhibition in appetitive settings. These findings are consistent with the developing view that the LHb promotes a negative reward prediction error in Pavlovian conditioning.SIGNIFICANCE STATEMENT Stimuli that positively or negatively predict rewarding outcomes influence choice between actions that deliver those outcomes. Previous studies have found that a positive predictor of a specific outcome biases choice toward actions delivering that outcome. In contrast, a negative predictor of an outcome biases choice away from actions earning that outcome and toward other actions. Here we reveal that the lateral habenula is critical for negative predictors, but not positive predictors, to affect choice. Furthermore, these effects were found to require activation of lateral habenula inputs to the rostromedial tegmental nucleus. These results are consistent with the view that the lateral habenula establishes inhibitory relationships between stimuli and food outcomes and computes a negative prediction error in Pavlovian conditioning.

Sex differences in incentive motivation and the relationship to the development and maintenance of alcohol use disorders

Despite considerable evidence of higher rates of alcohol use disorders (AUDs) in men than in women, there is a dearth of research into the underlying causes of this disparity. As the gap in high risk drinking between men and women closes, it is critical to disentangle the biological factors that may place men and women at different risk for the development of AUDs as well as AUD-associated health problems. While sex differences in alcohol drinking have been reported in animal models and in human alcoholics, it increasingly seems that consummatory behavior may be dissociated from propensity toward inflexible and cue-elicited drug seeking and taking that characterize alcohol use disorders. While much of this work was initially performed in males a growing, yet limited, body of literature suggests that there are sex differences in both cue reactivity, and further, the relationship between cue reactivity and the maintenance of addictive behavior, indicating that males may be at greater risk for the development of a subset of addiction-related behaviors independent of alcohol consumption. Here, we will review the current literature on sex effects on the relationship between incentive motivation and addictive behavior and discuss unanswered questions that we expect will inform the development of individualized and sex-specific treatment and prevention strategies for AUDs. We believe that a greater understanding of how sex interacts with in cue reactivity to independently mediate the drug taking and risk for the development of uncontrolled drug or alcohol-seeking and -taking will inform the development of individualized treatment and prevention strategies for addiction.

Selective impairment of goal-directed decision-making following lesions to the human ventromedial prefrontal cortex

See Manohar and Akam (doi10.1093/brain/awx119) for a scientific commentary on this article.

Aging-Related Dysfunction of Striatal Cholinergic Interneurons Produces Conflict in Action Selection

For goal-directed action to remain adaptive, new strategies are required to accommodate environmental changes, a process for which parafascicular thalamic modulation of cholinergic interneurons in the striatum (PF-to-CIN) appears critical. In the elderly, however, previously acquired experience frequently interferes with new learning, yet the source of this effect has remained unexplored. Here, combining sophisticated behavioral designs, cell-specific manipulation, and extensive neuronal imaging, we investigated the involvement of the PF-to-CIN pathway in this process. We found functional alterations of this circuit in aged mice that were consistent with their incapacity to update initial goal-directed learning, resulting in faulty activation of projection neurons in the striatum. Toxicogenetic ablation of CINs in young mice reproduced these behavioral and neuronal defects, suggesting that age-related deficits in PF-to-CIN function reduce the ability of older individuals to resolve conflict between actions, likely contributing to impairments in adaptive goal-directed action and executive control in aging. VIDEO ABSTRACT.

Learning-Related Brain-Electrical Activity Dynamics Associated with the Subsequent Impact of Learnt Action-Outcome Associations

Goal-directed behavior relies on the integration of anticipated outcomes into action planning based on acquired knowledge about the current contingencies between behavioral responses (R) and desired outcomes (O) under specific stimulus conditions (S). According to ideomotor theory, bidirectional R-O associations are an integral part of this knowledge structure. Previous EEG studies have identified neural activity markers linked to the involvement of such associations, but the initial acquisition process has not yet been characterized. The present study thus examined brain-electrical activity dynamics during the rapid acquisition of novel bidirectional R-O associations during instructed S-R learning. Within a trial, we inspected response-locked and stimulus-locked activity dynamics in order to identify markers linked to the forward and backward activation of bidirectional R-O associations as they were being increasingly strengthened under forced choice conditions. We found that a post-response anterior negativity following auditory outcomes was increasingly attenuated as a function of the acquired association strength. This suggests that previously reported action-induced sensory attenuation effects under extensively trained free choice conditions can be established within few repetitions of specific R-O pairings under forced choice conditions. Furthermore, we observed the even more rapid development of a post-response but pre-outcome fronto-central positivity which was reduced for high R-O learners which might indicate the rapid deployment of preparatory attention towards predictable outcomes. Finally, we identified a learning-related stimulus-locked activity modulation within the visual P1-N1 latency range which might reflect the multi-sensory integration of the perceived antecedent visual stimulus the anticipated auditory outcome.

Thalamic Control of Dorsomedial Striatum Regulates Internal State to Guide Goal-Directed Action Selection

We (Bradfield et al., 2013) have demonstrated previously that parafascicular thalamic nucleus (PF)-controlled neurons in the posterior dorsomedial striatum (pDMS) are critical for interlacing new and existing action-outcome contingencies to control goal-directed action. Based on these findings, it was suggested that animals with a dysfunctional PF-pDMS pathway might suffer a deficit in creating or retrieving internal contexts or "states" on which such information could become conditional. To assess this hypothesis more directly, rats were given a disconnection treatment using contralateral cytotoxic lesions of the PF and pDMS (Group CONTRA) or ipsilateral control lesions (Group IPSI) and trained to press a right and left lever for sucrose and pellet outcomes, after which these contingencies were reversed. The rats were then given an outcome devaluation test (all experiments) and a test of outcome-specific reinstatement (Experiments 1 and 3). We found that devaluation performance was intact for both groups after training of initial contingencies, but impaired for Group CONTRA after reversal. However, performance was restored by additional reversal training. Furthermore, when tested a second time after reversal training, rats in both groups demonstrated responding in accordance with the original contingencies, providing direct evidence of modulation of action selection by state. Finally, we found that external context could substitute for internal state and so could rescue responding in Group CONTRA, but only in the reinstatement test. Together, these findings suggest that animals use internal state information to guide action selection and that this information is modulated by the PF-pDMS pathway.SIGNIFICANCE STATEMENT Individuals with Parkinson's disease dementia often suffer a characteristic deficit in "cognitive flexibility." It has been suggested that neurodegeneration in the pathway between the centromedian/parafascicular thalalmic nucleus (PF) and striatum might underlie such deficits (Smith et al., 2014). In rats, we have similarly observed that a functional disconnection of the PF-posterior dorsomedial striatal pathway produces a specific impairment in the ability to alter goal-directed actions (Bradfield et al., 2013). It was suggested that this impairment could be a result of a deficit in state modulation. Here, we present four experiments that provide evidence for this hypothesis and suggest several ways (e.g., extended practice, providing external cues) in which this state modulation can be rescued.

Basolateral amygdala supports the maintenance of value and effortful choice of a preferred option

The basolateral amygdala (BLA) is known to be involved in appetitive behavior, yet its role in cost-benefit choice of qualitatively different rewards (more/less preferred), beyond magnitude differences (larger/smaller), is poorly understood. We assessed the effects of BLA inactivations on effortful choice behavior. Rats were implanted with cannulae in BLA and trained to stable lever pressing for sucrose pellets on a progressive ratio schedule. Rats were then introduced to a choice: chow was concurrently available while they could work for the preferred sucrose pellets. Rats were infused with either vehicle control (aCSF) or baclofen/muscimol prior to test. BLA inactivations produced a significant decrease in lever presses for sucrose pellets compared to vehicle, and chow consumption was unaffected. Inactivation had no effect on sucrose pellet preference when both options were freely available. Critically, when lab chow was not concurrently available, BLA inactivations had no effect on the number of lever presses for sucrose pellets, indicating that primary motivation in the absence of choice remains intact with BLA offline. After a test under specific satiety for sucrose pellets, BLA inactivation rendered animals less sensitive to devaluation relative to control. The effects of BLA inactivations in our task are not mediated by decreased appetite, an inability to perform the task, a change in food preference, or decrements in primary motivation. Taken together, BLA supports the specific value and effortful choice of a preferred option.

The propositional basis of cue-controlled reward seeking

Two experiments examined the role of propositional and automatic (ideomotor) processes in cue-elicited responding for rewarding outcomes (beer and chocolate). In a training phase, participants earned either chocolate or beer points by making one of two button-press responses. Rewards were indicated by the presentation of chocolate and beer pictures. On test, each trial began with a picture of beer or chocolate, or a blank screen, and choice of the beer versus chocolate response was assessed in the presence of these three pictures. Participants tended to choose the beer and chocolate response in the presence of the beer and chocolate pictures, respectively. In Experiment 1, instructions signalling that the pictures did not indicate which response would be rewarded significantly reduced the priming effect. In Experiment 2, instructions indicating that the pictures signified which response would not be rewarded resulted in a reversed priming effect. Finally, in both experiments, the priming effect correlated with self-reported beliefs that the cues signalled which response was more likely to be reinforced. These results suggest that cue-elicited response selection is mediated by a propositional belief regarding the efficacy of the response–outcome relationship, rather than an automatic ideomotor mechanism.

Goal-Directed Behavior and Instrumental Devaluation: A Neural System-Level Computational Model

Devaluation is the key experimental paradigm used to demonstrate the presence of instrumental behaviors guided by goals in mammals. We propose a neural system-level computational model to address the question of which brain mechanisms allow the current value of rewards to control instrumental actions. The model pivots on and shows the computational soundness of the hypothesis for which the internal representation of instrumental manipulanda (e.g., levers) activate the representation of rewards (or "action-outcomes", e.g., foods) while attributing to them a value which depends on the current internal state of the animal (e.g., satiation for some but not all foods). The model also proposes an initial hypothesis of the integrated system of key brain components supporting this process and allowing the recalled outcomes to bias action selection: (a) the sub-system formed by the basolateral amygdala and insular cortex acquiring the manipulanda-outcomes associations and attributing the current value to the outcomes; (b) three basal ganglia-cortical loops selecting respectively goals, associative sensory representations, and actions; (c) the cortico-cortical and striato-nigro-striatal neural pathways supporting the selection, and selection learning, of actions based on habits and goals. The model reproduces and explains the results of several devaluation experiments carried out with control rats and rats with pre- and post-training lesions of the basolateral amygdala, the nucleus accumbens core, the prelimbic cortex, and the dorso-medial striatum. The results support the soundness of the hypotheses of the model and show its capacity to integrate, at the system-level, the operations of the key brain structures underlying devaluation. Based on its hypotheses and predictions, the model also represents an operational framework to support the design and analysis of new experiments on the motivational aspects of goal-directed behavior.

Active inference and learning

This paper offers an active inference account of choice behaviour and learning. It focuses on the distinction between goal-directed and habitual behaviour and how they contextualise each other. We show that habits emerge naturally (and autodidactically) from sequential policy optimisation when agents are equipped with state-action policies. In active inference, behaviour has explorative (epistemic) and exploitative (pragmatic) aspects that are sensitive to ambiguity and risk respectively, where epistemic (ambiguity-resolving) behaviour enables pragmatic (reward-seeking) behaviour and the subsequent emergence of habits. Although goal-directed and habitual policies are usually associated with model-based and model-free schemes, we find the more important distinction is between belief-free and belief-based schemes. The underlying (variational) belief updating provides a comprehensive (if metaphorical) process theory for several phenomena, including the transfer of dopamine responses, reversal learning, habit formation and devaluation. Finally, we show that active inference reduces to a classical (Bellman) scheme, in the absence of ambiguity.

Associative and sensorimotor cortico-basal ganglia circuit roles in effects of abused drugs

The mammalian forebrain is characterized by the presence of several parallel cortico-basal ganglia circuits that shape the learning and control of actions. Among these are the associative, limbic and sensorimotor circuits. The function of all of these circuits has now been implicated in responses to drugs of abuse, as well as drug seeking and drug taking. While the limbic circuit has been most widely examined, key roles for the other two circuits in control of goal-directed and habitual instrumental actions related to drugs of abuse have been shown. In this review we describe the three circuits and effects of acute and chronic drug exposure on circuit physiology. Our main emphasis is on drug actions in dorsal striatal components of the associative and sensorimotor circuits. We then review key findings that have implicated these circuits in drug seeking and taking behaviors, as well as drug use disorders. Finally, we consider different models describing how the three cortico-basal ganglia circuits become involved in drug-related behaviors. This topic has implications for drug use disorders and addiction, as treatments that target the balance between the different circuits may be useful for reducing excessive substance use.

Vicarious trial and error

When rats come to a decision point, they sometimes pause and look back and forth as if deliberating over the choice; at other times, they proceed as if they have already made their decision. In the 1930s, this pause-and-look behaviour was termed 'vicarious trial and error' (VTE), with the implication that the rat was 'thinking about the future'. The discovery in 2007 that the firing of hippocampal place cells gives rise to alternating representations of each of the potential path options in a serial manner during VTE suggested a possible neural mechanism that could underlie the representations of future outcomes. More-recent experiments examining VTE in rats suggest that there are direct parallels to human processes of deliberative decision making, working memory and mental time travel.

Cue-elicited food seeking is eliminated with aversive outcomes following outcome devaluation

In outcome-selective Pavlovian-to-instrumental transfer (PIT), stimuli that are predictive of specific outcomes prime instrumental responses that are associated with these outcomes. Previous human studies yielded mixed evidence in respect to whether the PIT effect is affected by a posttraining devaluation of an outcome, with the PIT effect being preserved after a devaluation of a primary reinforcer (food, drugs) but not following the devaluation of a secondary reinforcer (money). The present research examined whether outcome-selective transfer is eliminated when the devaluation of a primary (liquid) reinforcer is strong and aversive. Experiment 1 confirmed these expectations following a devaluation with bad tasting Tween 20. However, outcome-selective transfer was still observed when the earned (devalued) outcome was not consumed immediately after each test (Experiment 2). These results suggest that the capacity of a Pavlovian cue to motivate a specific response is affected by the incentive value of the shared outcome only when the devaluation yields an aversive outcome that is consumed immediately.

The neural basis of integrating pre- and post-response information for goal-directed actions

A fundamental prerequisite for goal-directed action is to encode the contingencies between responses (R) producing specific outcomes (O) in specific stimulus conditions (S). The present study aimed to characterize the functional neuroanatomy of different associational sub-components of such S-R-O contingencies during the first few trials of exposure. We devised a novel paradigm that was suited to distinguish BOLD activation patterns related to S-R, R-O, and the full S-R-O contingency. Different from previous studies our experimental design ensured that stimulus-related processes and outcome-related processes were maximally comparable, as both were learned incidentally and lacked intrinsic incentive value, and different from trial-and-error learning situations, outcomes did not serve a special role as performance feedback. We observed contingency-related dissociations between SMA, lateral OFC, and large parts of the reward system including central OFC, anterior striatum and midbrain areas. While the lateral OFC was involved in processing differential outcomes irrespective of a predictive stimulus context, the SMA was specifically engaged when differential outcomes could be predicted by the stimulus. By contrast, the activation pattern of reward system areas suggested that these regions serve a role in integrating non-incentive differential outcome information and incentive common outcome information. Together, these results support the notion that striatal and orbitofrontal regions are involved in outcome-related processes beyond trial-and-error S-R learning, that is, when outcomes are non-incentive and do not serve as reinforcing feedback that drives learning. Furthermore, our results clarify the role of the SMA in outcome-related processes thereby supporting current versions of ideomotor theory.

A Closer Look at the Effects of Repeated Cocaine Exposure on Adaptive Decision-Making under Conditions That Promote Goal-Directed Control

It has been proposed that compulsive drug seeking reflects an underlying dysregulation in adaptive behavior that favors habitual (automatic and inflexible) over goal-directed (deliberative and highly flexible) action selection. Rodent studies have established that repeated exposure to cocaine or amphetamine facilitates the development of habits, producing behavior that becomes unusually insensitive to a reduction in the value of its outcome. The current study more directly investigated the effects of cocaine pre-exposure on goal-directed learning and action selection using an approach that discourages habitual performance. After undergoing a 15-day series of cocaine (15 or 30 mg/kg, i.p.) or saline injections and a drug withdrawal period, rats were trained to perform two different lever-press actions for distinct reward options. During a subsequent outcome devaluation test, both cocaine- and saline-treated rats showed a robust bias in their choice between the two actions, preferring whichever action had been trained with the reward that retained its value. Thus, it appears that the tendency for repeated cocaine exposure to promote habit formation does not extend to a more complex behavioral scenario that encourages goal-directed control. To further explore this issue, we assessed how prior cocaine treatment would affect the rats' ability to learn about a selective reduction in the predictive relationship between one of the two actions and its outcome, which is another fundamental feature of goal-directed behavior. Interestingly, we found that cocaine-treated rats showed enhanced, rather than diminished, sensitivity to this action-outcome contingency degradation manipulation. Given their mutual dependence on striatal dopamine signaling, we suggest that cocaine's effects on habit formation and contingency learning may stem from a common adaptation in this neurochemical system.

Changes in the Influence of Alcohol-Paired Stimuli on Alcohol Seeking across Extended Training

Previous work has demonstrated that goal-directed control of alcohol-seeking and other drug-related behaviors is reduced following extended self-administration and drug exposure. Here, we examined how the magnitude of stimulus influences on responding changes across similar training and drug exposure. Rats self-administered alcohol or sucrose for 2 or 8 weeks. Previous work has shown that 8 weeks, but not 2 weeks of self-administration produces habitual alcohol seeking. Next, all animals received equivalent Pavlovian conditioning sessions where a discrete stimulus predicted the delivery of alcohol or sucrose. Finally, the impact of the stimuli on ongoing instrumental responding was examined in a Pavlovian-instrumental transfer (PIT) test. While a significant PIT effect was observed following 2 weeks of either alcohol or sucrose self-administration, the magnitude of this effect was greater following 8 weeks of training. The specificity of the PIT effect appeared unchanged by extended training. While it is well established that evaluation of the outcome of responding contributes less to behavioral control following extended training and/or drug exposure, our data indicate that reward-predictive stimuli have a stronger contribution to responding after extended training. Together, these findings provide insight into the factors that control behavior after extended drug use, which will be important for developing effective methods for controlling and ideally reducing these behaviors.