Which cue to 'want'? Opioid stimulation of central amygdala makes goal-trackers show stronger goal-tracking, just as sign-trackers show stronger sign-tracking

Accounting for negative automaintenance in pigeons: a dual learning systems approach and factored representations

Animals, including Humans, are prone to develop persistent maladaptive and suboptimal behaviours. Some of these behaviours have been suggested to arise from interactions between brain systems of Pavlovian conditioning, the acquisition of responses to initially neutral stimuli previously paired with rewards, and instrumental conditioning, the acquisition of active behaviours leading to rewards. However the mechanics of these systems and their interactions are still unclear. While extensively studied independently, few models have been developed to account for these interactions. On some experiment, pigeons have been observed to display a maladaptive behaviour that some suggest to involve conflicts between Pavlovian and instrumental conditioning. In a procedure referred as negative automaintenance, a key light is paired with the subsequent delivery of food, however any peck towards the key light results in the omission of the reward. Studies showed that in such procedure some pigeons persisted in pecking to a substantial level despite its negative consequence, while others learned to refrain from pecking and maximized their cumulative rewards. Furthermore, the pigeons that were unable to refrain from pecking could nevertheless shift their pecks towards a harmless alternative key light. We confronted a computational model that combines dual-learning systems and factored representations, recently developed to account for sign-tracking and goal-tracking behaviours in rats, to these negative automaintenance experimental data. We show that it can explain the variability of the observed behaviours and the capacity of alternative key lights to distract pigeons from their detrimental behaviours. These results confirm the proposed model as an interesting tool to reproduce experiments that could involve interactions between Pavlovian and instrumental conditioning. The model allows us to draw predictions that may be experimentally verified, which could help further investigate the neural mechanisms underlying theses interactions.

Emotion and relative reward processing: an investigation on instrumental successive negative contrast and ultrasonic vocalizations in the rat

Incentive contrast effects include changes in behavioral responses after a reward upshift (positive contrast) or downshift (negative contrast). Proposed influences on these behavioral changes are emotional state reactions after experiencing or anticipating a change in reward outcome. Rat ultrasonic vocalizations have been shown to be indicators of emotional state during behavior and anticipatory periods. The objective of the present study was to monitor rodent ultrasounds during incentive contrast using a classical runway procedure called instrumental successive negative contrast. The procedure is one that has been used often to examine incentive relativity because of its reliability in measuring negative contrast effects. Rats were trained to run in the alleyway to receive a high (12 pellets) or low magnitude (1 pellet) outcome. The high magnitude was then shifted to the low and running speeds in the alleyway for the reward and USV emission were compared. Replicating previous work, a negative contrast effect was observed with postshift running speeds significantly slower in the shifted group compared to the unshifted group. USVs did not follow the same pattern with an apparent lack of significant differences between the groups following the reward downshift. We also tested another group of animals using a visual predictive cue in the same runway test. When visual cues predicted high or low magnitude outcome, no incentive contrast was found for the running speeds following an outcome downshift, but a weak contrast effect was observed for the USV emission. These results demonstrate a separation between USVs and behavioral indicators of incentive contrast suggesting that concomitant shifts in negative affect may not be necessary for anticipatory relative reward processes.

Impulsivity, risk-taking, and distractibility in rats exhibiting robust conditioned orienting behaviors

When a neutral cue is followed by a significant event such as food delivery, some animals become engaged with the cue itself and acquire cue-directed behaviors. One type of cue-directed behavior is observed following insertion of a lever used as a conditioned stimulus (CS). Rats showing robust approach behavior to the lever also display impulsivity and altered attention, as compared to rats showing behavior directed toward the reward delivery location. The current study used a light CS to categorize rats' propensity for cue-directed behavior, and assessed whether individual differences in impulsivity and related behaviors still emerged. During the light-food pairings, some rats displayed enhanced rearing or orienting to the light (Orienters) prior to showing food cup approach behavior, while other rats only showed food cup approach behavior (Nonorienters). Our results showed that Orienters made more impulsive and risky decisions in two different choice tasks, and were quicker to leave a familiar dark environment to enter a novel bright field. Orienters also showed less accurate target detection when a visual distractor was introduced during an attentional challenge. Our current study suggests that light CS-induced rearing/orienting behavior might not necessarily share an identical mechanism with lever CS-approach behavior in predicting impulsivity-related behaviors.

Experimental predictions drawn from a computational model of sign-trackers and goal-trackers

Gaining a better understanding of the biological mechanisms underlying the individual variation observed in response to rewards and reward cues could help to identify and treat individuals more prone to disorders of impulsive control, such as addiction. Variation in response to reward cues is captured in rats undergoing autoshaping experiments where the appearance of a lever precedes food delivery. Although no response is required for food to be delivered, some rats (goal-trackers) learn to approach and avidly engage the magazine until food delivery, whereas other rats (sign-trackers) come to approach and engage avidly the lever. The impulsive and often maladaptive characteristics of the latter response are reminiscent of addictive behaviour in humans. In a previous article, we developed a computational model accounting for a set of experimental data regarding sign-trackers and goal-trackers. Here we show new simulations of the model to draw experimental predictions that could help further validate or refute the model. In particular, we apply the model to new experimental protocols such as injecting flupentixol locally into the core of the nucleus accumbens rather than systemically, and lesioning of the core of the nucleus accumbens before or after conditioning. In addition, we discuss the possibility of removing the food magazine during the inter-trial interval. The predictions from this revised model will help us better understand the role of different brain regions in the behaviours expressed by sign-trackers and goal-trackers.

The form of a conditioned stimulus can influence the degree to which it acquires incentive motivational properties

There is considerable individual variation in the extent to which food- and drug-associated cues (conditioned stimuli, CSs) acquire incentive salience, as indicated by whether they elicit approach towards them, and/or act as conditioned reinforcers. Here we asked whether this variation is influenced by properties of the CS itself. In rats, we assessed both the attractiveness and conditioned reinforcing properties of two CSs: a manipulable lever CS versus an auditory (tone) CS. There was considerable individual variation in the extent to which a lever CS acquired incentive motivational properties, as indicated by whether it became attractive (evoked a sign-tracking or goal-tracking conditioned response) or acted as a conditioned reinforcer. However, with a tone CS all rats learned a goal-tracking response, and the tone CS was an equally effective conditioned reinforcer in sign-trackers and goal-trackers. Even when presented in compound (a lever-tone CS), the two elements of the compound differentially acquired motivational properties. In contrast, amphetamine and stress potentiated the conditioned reinforcing properties of both visual and auditory CSs similarly in rats that primarily sign-tracked or goal-tracked. We conclude that variation in the to the ability of CSs to acquire incentive salience, and thus their ability to act as incentive stimuli capable of motivating behavior, is determined in part by properties of the CS itself.

Regulation of expression of relaxin-3 and its receptor RXFP3 in the brain of diet-induced obese rats

An animal model closely related to human obesity is diet-induced obesity in Sprague-Dawley rats. These rats placed on a high-energy (HE) diet show wide distribution in body weight gain with a subset of animals developing diet-induced obesity (DIO) and the remaining animals showing a diet-resistant (DR) phenotype. Once obesity is established, DIO rats strongly defend their increased body weight against caloric restriction. There is evidence that neuropeptide relaxin-3 is involved in food intake regulation, but the levels of expression of relaxin-3 and its receptor have not been yet demonstrated in the DIO model. The present study investigated the brain expression of relaxin-3 and its cognate receptor RXFP3 in DIO and DR rats maintained on an HE diet since weaning. Expression of relaxin-3 and RXFP3 mRNAs was assessed by in situ hybridization in ad libitum, food-deprived (12 h) and refed (1 h) feeding states. The levels of expression of relaxin-3 in the medial portion of the nucleus incertus (NI) were higher in the DIO rats compared to the DR rats in the ad libitum-fed state. Food deprivation increased the levels of expression of relaxin-3 in the medial NI in DR but not DIO rats. The stronger expression of relaxin-3 in the ad libitum-fed state in the DIO rats was accompanied by low expression of the RXFP3 receptor in the paraventricular hypothalamic nucleus (PVN), supraoptic nucleus, central amygdala (CeA), NI, and nucleus of the solitary tract (NTS). Refeeding increased expression of RXFP3 in the paraventricular thalamic nucleus, parvocellular PVN, CeA, NI, and NTS in the DIO rats. These results provide evidence that DIO rats show a constitutive increase in relaxin-3 expression in the medial NI and that refeeding after food deprivation may enhance the orexigenic effects of relaxin-3 in DIO rats by rapid upregulation of the expression of RXFP3 in the specific brain regions involved in food intake regulation.

Modelling individual differences in the form of Pavlovian conditioned approach responses: a dual learning systems approach with factored representations

Reinforcement Learning has greatly influenced models of conditioning, providing powerful explanations of acquired behaviour and underlying physiological observations. However, in recent autoshaping experiments in rats, variation in the form of Pavlovian conditioned responses (CRs) and associated dopamine activity, have questioned the classical hypothesis that phasic dopamine activity corresponds to a reward prediction error-like signal arising from a classical Model-Free system, necessary for Pavlovian conditioning. Over the course of Pavlovian conditioning using food as the unconditioned stimulus (US), some rats (sign-trackers) come to approach and engage the conditioned stimulus (CS) itself - a lever - more and more avidly, whereas other rats (goal-trackers) learn to approach the location of food delivery upon CS presentation. Importantly, although both sign-trackers and goal-trackers learn the CS-US association equally well, only in sign-trackers does phasic dopamine activity show classical reward prediction error-like bursts. Furthermore, neither the acquisition nor the expression of a goal-tracking CR is dopamine-dependent. Here we present a computational model that can account for such individual variations. We show that a combination of a Model-Based system and a revised Model-Free system can account for the development of distinct CRs in rats. Moreover, we show that revising a classical Model-Free system to individually process stimuli by using factored representations can explain why classical dopaminergic patterns may be observed for some rats and not for others depending on the CR they develop. In addition, the model can account for other behavioural and pharmacological results obtained using the same, or similar, autoshaping procedures. Finally, the model makes it possible to draw a set of experimental predictions that may be verified in a modified experimental protocol. We suggest that further investigation of factored representations in computational neuroscience studies may be useful.

Endogenous opiates and behavior: 2012

This paper is the thirty-fifth consecutive installment of the annual review of research concerning the endogenous opioid system. It summarizes papers published during 2012 that studied the behavioral effects of molecular, pharmacological and genetic manipulation of opioid peptides, opioid receptors, opioid agonists and opioid antagonists. The particular topics that continue to be covered include the molecular-biochemical effects and neurochemical localization studies of endogenous opioids and their receptors related to behavior (Section 2), and the roles of these opioid peptides and receptors in pain and analgesia (Section 3); stress and social status (Section 4); tolerance and dependence (Section 5); learning and memory (Section 6); eating and drinking (Section 7); alcohol and drugs of abuse (Section 8); sexual activity and hormones, pregnancy, development and endocrinology (Section 9); mental illness and mood (Section 10); seizures and neurologic disorders (Section 11); electrical-related activity and neurophysiology (Section 12); general activity and locomotion (Section 13); gastrointestinal, renal and hepatic functions (Section 14); cardiovascular responses (Section 15); respiration and thermoregulation (Section 16); and immunological responses (Section 17).

Central amygdala opioid transmission is necessary for increased high-fat intake following 24-h food deprivation, but not following intra-accumbens opioid administration

Previous research has demonstrated a dissociation of certain neural mediators that contribute to the increased consumption of a high-fat diet that follows intra-accumbens (Acb) administration of μ-opioid receptor agonists vs. 24-h food deprivation. These two models, both which induce rapid consumption of the diet, have been shown to involve a distributed corticolimbic circuitry, including the amygdala. Specifically, the central amygdala (CeA) has been shown to be involved in high-fat feeding within both opioid and food-deprivation driven models. The present experiments were conducted to examine the more specific role of CeA opioid transmission in mediating high-fat feeding driven by either intra-Acb administration of the μ-opioid agonist d-Ala2-NMe-Phe4-Glyol5-enkephalin (DAMGO) or 24-h home cage food deprivation. Injection of DAMGO into the Acb (0.25 μg/0.5 μl/side) increased consumption of the high-fat diet, but this feeding was unaffected by administration of opioid antagonist, naltrexone (5 μg/0.25 μl/side) administered into the CeA. In contrast, intra-CeA naltrexone administration attenuated high-fat intake driven by 24-h food deprivation, demonstrating a specific role for CeA opioid transmission in high-fat consumption. Intra-CeA naltrexone administration alone had no effect on baseline feeding levels within either feeding model. These findings suggest that CeA opioid transmission mediates consumption of a palatable high-fat diet driven by short-term negative-energy balance (24-h food deprivation), but not intra-Acb opioid receptor activation.

Individual variation in resisting temptation: implications for addiction

When exposed to the sights, sounds, smells and/or places that have been associated with rewards, such as food or drugs, some individuals have difficulty resisting the temptation to seek out and consume them. Others have less difficulty restraining themselves. Thus, Pavlovian reward cues may motivate maladaptive patterns of behavior to a greater extent in some individuals than in others. We are just beginning to understand the factors underlying individual differences in the extent to which reward cues acquire powerful motivational properties, and therefore, the ability to act as incentive stimuli. Here we review converging evidence from studies in both human and non-human animals suggesting that a subset of individuals are more "cue reactive", in that certain reward cues are more likely to attract these individuals to them and motivate actions to get them. We suggest that those individuals for whom Pavlovian reward cues become especially powerful incentives may be more vulnerable to impulse control disorders, such as binge eating and addiction.

Mapping brain circuits of reward and motivation: in the footsteps of Ann Kelley

Ann Kelley was a scientific pioneer in reward neuroscience. Her many notable discoveries included demonstrations of accumbens/striatal circuitry roles in eating behavior and in food reward, explorations of limbic interactions with hypothalamic regulatory circuits, and additional interactions of motivation circuits with learning functions. Ann Kelley's accomplishments inspired other researchers to follow in her footsteps, including our own laboratory group. Here we describe results from several lines of our research that sprang in part from earlier findings by Kelley and colleagues. We describe hedonic hotspots for generating intense pleasure 'liking', separate identities of 'wanting' versus 'liking' systems, a novel role for dorsal neostriatum in generating motivation to eat, a limbic keyboard mechanism in nucleus accumbens for generating intense desire versus intense dread, and dynamic limbic transformations of learned memories into motivation. We describe how origins for each of these themes can be traced to fundamental contributions by Ann Kelley.

Dopamine or opioid stimulation of nucleus accumbens similarly amplify cue-triggered 'wanting' for reward: entire core and medial shell mapped as substrates for PIT enhancement

Pavlovian cues [conditioned stimulus (CS+)] often trigger intense motivation to pursue and consume related reward [unconditioned stimulus (UCS)]. But cues do not always trigger the same intensity of motivation. Encountering a reward cue can be more tempting on some occasions than on others. What makes the same cue trigger more intense motivation to pursue reward on a particular encounter? The answer may be the level of incentive salience ('wanting') that is dynamically generated by mesocorticolimbic brain systems, influenced especially by dopamine and opioid neurotransmission in the nucleus accumbens (NAc) at that moment. We tested the ability of dopamine stimulation (by amphetamine microinjection) vs. mu opioid stimulation [by d-Ala, nMe-Phe, Glyol-enkephalin (DAMGO) microinjection] of either the core or shell of the NAc to amplify cue-triggered levels of motivation to pursue sucrose reward, measured with a Pavlovian-Instrumental Transfer (PIT) procedure, a relatively pure assay of incentive salience. Cue-triggered 'wanting' in PIT was enhanced by amphetamine or DAMGO microinjections equally, and also equally at nearly all sites throughout the entire core and medial shell (except for a small far-rostral strip of shell). NAc dopamine/opioid stimulations specifically enhanced CS+ ability to trigger phasic peaks of 'wanting' to obtain UCS, without altering baseline efforts when CS+ was absent. We conclude that dopamine/opioid stimulation throughout nearly the entire NAc can causally amplify the reactivity of mesocorticolimbic circuits, and so magnify incentive salience or phasic UCS 'wanting' peaks triggered by a CS+. Mesolimbic amplification of incentive salience may explain why a particular cue encounter can become irresistibly tempting, even when previous encounters were successfully resisted before.

Principles of motivation revealed by the diverse functions of neuropharmacological and neuroanatomical substrates underlying feeding behavior

Circuits that participate in specific subcomponents of feeding (e.g., gustatory perception, peripheral feedback relevant to satiety and energy balance, reward coding, etc.) are found at all levels of the neural axis. Further complexity is conferred by the wide variety of feeding-modulatory neurotransmitters and neuropeptides that act within these circuits. An ongoing challenge has been to refine the understanding of the functional specificity of these neurotransmitters and circuits, and there have been exciting advances in recent years. We focus here on foundational work of Dr. Ann Kelley that identified distinguishable actions of striatal opioid peptide modulation and dopamine transmission in subcomponents of reward processing. We also discuss her work in overlaying these neuropharmacological effects upon anatomical pathways that link the telencephalon (cortex and basal ganglia) with feeding-control circuits in the hypothalamus. Using these seminal contributions as a starting point, we will discuss new findings that expand our understanding of (1) the specific, differentiable motivational processes that are governed by central dopamine and opioid transmission, (2) the manner in which other striatal neuromodulators, specifically acetylcholine, endocannabinoids and adenosine, modulate these motivational processes (including via interactions with opioid systems), and (3) the organization of the cortical-subcortical network that subserves opioid-driven feeding. The findings discussed here strengthen the view that incentive-motivational properties of food are coded by substrates and neural circuits that are distinguishable from those that mediate the acute hedonic experience of food reward. Striatal opioid transmission modulates reward processing by engaging frontotemporal circuits, possibly via a hypothalamic-thalamic axis, that ultimately impinges upon hypothalamic modules dedicated to autonomic function and motor pattern control. We will conclude by discussing implications for understanding disorders of "non-homeostatic" feeding.

The effect of nicotine on sign-tracking and goal-tracking in a Pavlovian conditioned approach paradigm in rats

RATIONALE

Nicotine (NIC) potently increases operant responding for non-NIC reinforcers, and this effect may depend on drug-mediated increases in incentive motivation. According to this hypothesis, NIC should also potently increase approach to Pavlovian-conditioned stimuli associated with rewards.

OBJECTIVE

The present studies explored the effects of NIC on Pavlovian-conditioned approach responses.

METHOD

To do so, liquid dippers were used to deliver an unconditioned stimulus (US; 0.1 ml sucrose) after presentation of a conditioned stimulus (CS; 30 s illumination of a stimulus light)-both the CS and US were presented in receptacles equipped to monitor head entries.

RESULTS

In experiment 1, the CS and US were presented in the same receptacle, but NIC pretreatment (0.4 mg/kg base) did not increase conditioned approach responses. Delivery of the sucrose US was then shifted to receptacle in a different location. All rats learned to approach the new US location (goal-tracking) at similar rates. Approach to the CS receptacle (sign-tracking) declined for saline-pretreated rats, but NIC pretreatment increased sign-tracking. In experiment 2, NIC pretreatment increased sign-tracking when the CS and US were spatially separated during acquisition. In experiment 3, NIC pretreatments were replaced with saline, but the effect of NIC persisted for an additional 24 test sessions.

CONCLUSION

The findings suggest that NIC increases incentive motivation and that this effect is long-lasting, persisting beyond the pharmacological effects of NIC.

Neural response to visual sexual cues in dopamine treatment-linked hypersexuality in Parkinson's disease

Hypersexuality with compulsive sexual behaviour is a significant source of morbidity for patients with Parkinson's disease receiving dopamine replacement therapies. We know relatively little about the pathophysiology of hypersexuality in Parkinson's disease, and it is unknown how visual sexual stimuli, similar to the portrayals of sexuality in the mainstream mass media may affect the brain and behaviour in such susceptible individuals. Here, we have studied a group of 12 patients with Parkinson's disease with hypersexuality using a functional magnetic resonance imaging block design exposing participants to both sexual, other reward-related and neutral visual cues. We hypothesized that exposure to visual sexual cues would trigger increased sexual desire in patients with Parkinson's disease with hypersexuality that would correspond to changes in brain activity in regions linked to dopaminergically stimulated sexual motivation. Patients with Parkinson's disease with hypersexuality were scanned ON and OFF dopamine drugs, and their results were compared with a group of 12 Parkinson's disease control patients without hypersexuality or other impulse control disorders. Exposure to sexual cues significantly increased sexual desire and hedonic responses in the Parkinson's disease hypersexuality group compared with the Parkinson's disease control patients. These behavioural changes corresponded to significant blood oxygen level-dependent signal changes in regions within limbic, paralimbic, temporal, occipital, somatosensory and prefrontal cortices that correspond to emotional, cognitive, autonomic, visual and motivational processes. The functional imaging data showed that the hypersexuality patients' increased sexual desire correlated with enhanced activations in the ventral striatum, and cingulate and orbitofrontal cortices. When the patients with Parkinson's disease with hypersexuality were OFF medication, the functional imaging data showed decreases in activation during the presentation of sexual cues relative to rest. These deactivations were not observed when the patients were ON medication, suggesting that dopamine drugs may release inhibition within local neuronal circuits in the cerebral cortex that may contribute to compulsive sexual behaviour. The findings of this study have implications with respect to the potential influence of cue exposure via exposure to mass media in enhancing libido, which in this group of vulnerable patients can lead to devastating social consequences and occasionally, custodial sentences. Stimulation through exposure to sexual visual cues in patients with Parkinson's disease with hypersexuality provides a motivational impetus for seeking this reward behaviour through activations and deactivations of cerebral cortex.

Reward uncertainty enhances incentive salience attribution as sign-tracking

Conditioned stimuli (CSs) come to act as motivational magnets following repeated association with unconditioned stimuli (UCSs) such as sucrose rewards. By traditional views, the more reliably predictive a Pavlovian CS-UCS association, the more the CS becomes attractive. However, in some cases, less predictability might equal more motivation. Here we examined the effect of introducing uncertainty in CS-UCS association on CS strength as an attractive motivation magnet. In the present study, Experiment 1 assessed the effects of Pavlovian predictability versus uncertainty about reward probability and/or reward magnitude on the acquisition and expression of sign-tracking (ST) and goal-tracking (GT) responses in an autoshaping procedure. Results suggested that uncertainty produced strongest incentive salience expressed as sign-tracking. Experiment 2 examined whether a within-individual temporal shift from certainty to uncertainty conditions could produce a stronger CS motivational magnet when uncertainty began, and found that sign-tracking still increased after the shift. Overall, our results support earlier reports that ST responses become more pronounced in the presence of uncertainty regarding CS-UCS associations, especially when uncertainty combines both probability and magnitude. These results suggest that Pavlovian uncertainty, although diluting predictability, is still able to enhance the incentive motivational power of particular CSs.

The role of dopamine in the accumbens core in the expression of Pavlovian-conditioned responses

The role of dopamine in reward is a topic of debate. For example, some have argued that phasic dopamine signaling provides a prediction-error signal necessary for stimulus-reward learning, whereas others have hypothesized that dopamine is not necessary for learning per se, but for attributing incentive motivational value ('incentive salience') to reward cues. These psychological processes are difficult to tease apart, because they tend to change together. To disentangle them we took advantage of natural individual variation in the extent to which reward cues are attributed with incentive salience, and asked whether dopamine (specifically in the core of the nucleus accumbens) is necessary for the expression of two forms of pavlovian-conditioned approach behavior--one in which the cue acquires powerful motivational properties (sign-tracking) and another closely related one in which it does not (goal-tracking). After acquisition of these conditioned responses (CRs), intra-accumbens injection of the dopamine receptor antagonist flupenthixol markedly impaired the expression of a sign-tracking CR, but not a goal-tracking CR. Furthermore, dopamine antagonism did not produce a gradual extinction-like decline in behavior, but maximally impaired expression of a sign-tracking CR on the very first trial, indicating the effect was not due to new learning (i.e. it occurred in the absence of new prediction-error computations). The data support the view that dopamine in the accumbens core is not necessary for learning stimulus-reward associations, but for attributing incentive salience to reward cues, transforming predictive conditional stimuli into incentive stimuli with powerful motivational properties.