From prediction error to incentive salience: mesolimbic computation of reward motivation

Mechanisms of motivation-cognition interaction: challenges and opportunities

Recent years have seen a rejuvenation of interest in studies of motivation-cognition interactions arising from many different areas of psychology and neuroscience. The present issue of Cognitive, Affective, & Behavioral Neuroscience provides a sampling of some of the latest research from a number of these different areas. In this introductory article, we provide an overview of the current state of the field, in terms of key research developments and candidate neural mechanisms receiving focused investigation as potential sources of motivation-cognition interaction. However, our primary goal is conceptual: to highlight the distinct perspectives taken by different research areas, in terms of how motivation is defined, the relevant dimensions and dissociations that are emphasized, and the theoretical questions being targeted. Together, these distinctions present both challenges and opportunities for efforts aiming toward a more unified and cross-disciplinary approach. We identify a set of pressing research questions calling for this sort of cross-disciplinary approach, with the explicit goal of encouraging integrative and collaborative investigations directed toward them.

Optimal inference with suboptimal models: addiction and active Bayesian inference

When casting behaviour as active (Bayesian) inference, optimal inference is defined with respect to an agent's beliefs - based on its generative model of the world. This contrasts with normative accounts of choice behaviour, in which optimal actions are considered in relation to the true structure of the environment - as opposed to the agent's beliefs about worldly states (or the task). This distinction shifts an understanding of suboptimal or pathological behaviour away from aberrant inference as such, to understanding the prior beliefs of a subject that cause them to behave less 'optimally' than our prior beliefs suggest they should behave. Put simply, suboptimal or pathological behaviour does not speak against understanding behaviour in terms of (Bayes optimal) inference, but rather calls for a more refined understanding of the subject's generative model upon which their (optimal) Bayesian inference is based. Here, we discuss this fundamental distinction and its implications for understanding optimality, bounded rationality and pathological (choice) behaviour. We illustrate our argument using addictive choice behaviour in a recently described 'limited offer' task. Our simulations of pathological choices and addictive behaviour also generate some clear hypotheses, which we hope to pursue in ongoing empirical work.

Modulation of locomotor activation by the rostromedial tegmental nucleus

The rostromedial tegmental nucleus (RMTg) is a strong inhibitor of dopamine neurons in the ventral tegmental area (VTA) reported to influence neurobiological and behavioral responses to reward omission, aversive and fear-eliciting stimuli, and certain drugs of abuse. Insofar as previous studies implicate ventral mesencephalic dopamine neurons as an essential component of locomotor activation, we hypothesized that the RMTg also should modulate locomotion activation. We observed that bilateral infusions into the RMTg of the gamma-aminobutyric acid A (GABAA) agonist, muscimol, indeed activate locomotion. Alternatively, bilateral RMTg infusions of the GABAA receptor antagonist, bicuculline, suppress robust activations of locomotion elicited in two distinct ways: (1) by disinhibitory stimulation of neurons in the lateral preoptic area and (2) by return of rats to an environment previously paired with amphetamine administration. The possibility that suppressive locomotor effects of RMTg bicuculline infusions were due to unintended spread of drug to the nearby VTA was falsified by a control experiment showing that bilateral infusions of bicuculline into the VTA produce activation rather than suppression of locomotion. These results objectively implicate the RMTg in the regulation of locomotor activation. The effect is important because much evidence reported in the literature suggests that locomotor activation can be an involuntary behavioral expression of expectation and/or want without which the willingness to execute adaptive behaviors is impaired.

Reward processing in neurodegenerative disease

Representation of reward value involves a distributed network including cortical and subcortical structures. Because neurodegenerative illnesses target specific anatomic networks that partially overlap with the reward circuit, they would be predicted to have distinct impairments in reward processing. This review presents the existing evidence of reward processing changes in neurodegenerative diseases including mild cognitive impairment (MCI), Alzheimer's disease, frontotemporal dementia, amyotrophic lateral sclerosis (ALS), Parkinson's disease, and Huntington's disease, as well as in healthy aging. Carefully distinguishing the different aspects of reward processing (primary rewards, secondary rewards, reward-based learning, and reward-based decision-making) and using tasks that differentiate the stages of processing reward will lead to improved understanding of this fundamental process and clarify a contributing cause of behavioral change in these illnesses.

Automatic versus Choice-Dependent Value Representations in the Human Brain

The subjective values of choice options can impact on behavior in two fundamentally different types of situations: first, when people explicitly base their actions on such values, and second, when values attract attention despite being irrelevant for current behavior. Here we show with functional magnetic resonance imaging (fMRI) that these two behavioral functions of values are encoded in distinct regions of the human brain. In the medial prefrontal cortex, value-related activity is enhanced when subjective value becomes choice-relevant, and the magnitude of this increase relates directly to the outcome and reliability of the value-based choice. In contrast, activity in the posterior cingulate cortex represents values similarly when they are relevant or irrelevant for the present choice, and the strength of this representation predicts attentional capture by choice-irrelevant values. Our results suggest that distinct components of the brain's valuation network encode value in context-dependent manners that serve fundamentally different behavioral aims.

Central role for the insular cortex in mediating conditioned responses to anticipatory cues

Reward-related circuits are fundamental for initiating feeding on the basis of food-predicting cues, whereas gustatory circuits are believed to be involved in the evaluation of food during consumption. However, accumulating evidence challenges such a rigid separation. The insular cortex (IC), an area largely studied in rodents for its role in taste processing, is involved in representing anticipatory cues. Although IC responses to anticipatory cues are well established, the role of IC cue-related activity in mediating feeding behaviors is poorly understood. Here, we examined the involvement of the IC in the expression of cue-triggered food approach in mice trained with a Pavlovian conditioning paradigm. We observed a significant change in neuronal firing during presentation of the cue. Pharmacological silencing of the IC inhibited food port approach. Such a behavior could be recapitulated by temporally selective inactivation during the cue. These findings represent the first evidence, to our knowledge, that cue-evoked neuronal activity in the mouse IC modulates behavioral output, and demonstrate a causal link between cue responses and feeding behaviors.

The predictive mind and the experience of visual art work

Among the main challenges of the predictive brain/mind concept is how to link prediction at the neural level to prediction at the cognitive-psychological level and finding conceptually robust and empirically verifiable ways to harness this theoretical framework toward explaining higher-order mental and cognitive phenomena, including the subjective experience of aesthetic and symbolic forms. Building on the tentative prediction error account of visual art, this article extends the application of the predictive coding framework to the visual arts. It does so by linking this theoretical discussion to a subjective, phenomenological account of how a work of art is experienced. In order to engage more deeply with a work of art, viewers must be able to tune or adapt their prediction mechanism to recognize art as a specific class of objects whose ontological nature defies predictability, and they must be able to sustain a productive flow of predictions from low-level sensory, recognitional to abstract semantic, conceptual, and affective inferences. The affective component of the process of predictive error optimization that occurs when a viewer enters into dialog with a painting is constituted both by activating the affective affordances within the image and by the affective consequences of prediction error minimization itself. The predictive coding framework also has implications for the problem of the culturality of vision. A person's mindset, which determines what top-down expectations and predictions are generated, is co-constituted by culture-relative skills and knowledge, which form hyperpriors that operate in the perception of art.

Homeostatic reinforcement learning for integrating reward collection and physiological stability

Efficient regulation of internal homeostasis and defending it against perturbations requires adaptive behavioral strategies. However, the computational principles mediating the interaction between homeostatic and associative learning processes remain undefined. Here we use a definition of primary rewards, as outcomes fulfilling physiological needs, to build a normative theory showing how learning motivated behaviors may be modulated by internal states. Within this framework, we mathematically prove that seeking rewards is equivalent to the fundamental objective of physiological stability, defining the notion of physiological rationality of behavior. We further suggest a formal basis for temporal discounting of rewards by showing that discounting motivates animals to follow the shortest path in the space of physiological variables toward the desired setpoint. We also explain how animals learn to act predictively to preclude prospective homeostatic challenges, and several other behavioral patterns. Finally, we suggest a computational role for interaction between hypothalamus and the brain reward system.

DOI:http://dx.doi.org/10.7554/eLife.04811.001

Our survival depends on our ability to maintain internal states, such as body temperature and blood sugar levels, within narrowly defined ranges, despite being subject to constantly changing external forces. This process, which is known as homeostasis, requires humans and other animals to carry out specific behaviors—such as seeking out warmth or food—to compensate for changes in their environment. Animals must also learn to prevent the potential impact of changes that can be anticipated.

A network that includes different regions of the brain allows animals to perform the behaviors that are needed to maintain homeostasis. However, this network is distinct from the network that supports the learning of new behaviors in general. These two systems must, therefore, interact so that animals can learn novel strategies to support their physiological stability, but it is not clear how animals do this.

Keramati and Gutkin have now devised a mathematical model that explains the nature of this interaction, and that can account for many behaviors seen among animals, even those that might otherwise appear irrational. There are two assumptions at the heart of the model. First, it is assumed that animals are capable of guessing the impact of the outcome of their behaviors on their internal state. Second, it is assumed that animals find a behavior rewarding if they believe that the predicted impact of its outcome will reduce the difference between a particular internal state and its ideal value. For example, a form of behavior for a human might be going to the kitchen, and an outcome might be eating chocolate.

Based on these two assumptions, the model shows that animals stabilize their internal state around its ideal value by simply learning to perform behaviors that lead to rewarding outcomes (such as going into the kitchen and eating chocolate). Their theory also explains the physiological importance of a type of behavior known as ‘delay discounting’. Animals displaying this form of behavior regard a positive outcome as less rewarding the longer they have to wait for it. The model proves mathematically that delay discounting is a logical way to optimize homeostasis.

In addition to making a number of predictions that could be tested in experiments, Keramati and Gutkin argue that their model can account for the failure of homeostasis to limit food consumption whenever foods loaded with salt, sugar or fat are freely available.

DOI:http://dx.doi.org/10.7554/eLife.04811.002

Duration of the unconditioned stimulus in appetitive conditioning of honeybees differentially impacts learning, long-term memory strength, and the underlying protein synthesis

This study examines the role of stimulus duration in learning and memory formation of honeybees (Apis mellifera). In classical appetitive conditioning honeybees learn the association between an initially neutral, conditioned stimulus (CS) and the occurrence of a meaningful stimulus, the unconditioned stimulus (US). Thereby the CS becomes a predictor for the US eliciting a conditioned response (CR). Here we study the role of US duration in classical conditioning by examining honeybees conditioned with different US durations. We quantify the CR during acquisition, memory retention, and extinction of the early long-term memory (eLTM), and examine the molecular mechanisms of eLTM by interfering with protein synthesis. We find that the US duration affects neither the probability nor the strength of the CR during acquisition, eLTM retention, and extinction 24 h after conditioning. However, we find that the resistance to extinction 24 h after conditioning is susceptible to protein synthesis inhibition depending on the US duration. We conclude that the US duration does not affect the predictability of the US but modulates the protein synthesis underlying the eLTM's strength. Thus, the US duration differentially impacts learning, eLTM strength, and its underlying protein synthesis.

Inferring reward prediction errors in patients with schizophrenia: a dynamic reward task for reinforcement learning

Abnormalities in the dopamine system have long been implicated in explanations of reinforcement learning and psychosis. The updated reward prediction error (RPE)—a discrepancy between the predicted and actual rewards—is thought to be encoded by dopaminergic neurons. Dysregulation of dopamine systems could alter the appraisal of stimuli and eventually lead to schizophrenia. Accordingly, the measurement of RPE provides a potential behavioral index for the evaluation of brain dopamine activity and psychotic symptoms. Here, we assess two features potentially crucial to the RPE process, namely belief formation and belief perseveration, via a probability learning task and reinforcement-learning modeling. Forty-five patients with schizophrenia [26 high-psychosis and 19 low-psychosis, based on their p1 and p3 scores in the positive-symptom subscales of the Positive and Negative Syndrome Scale (PANSS)] and 24 controls were tested in a feedback-based dynamic reward task for their RPE-related decision making. While task scores across the three groups were similar, matching law analysis revealed that the reward sensitivities of both psychosis groups were lower than that of controls. Trial-by-trial data were further fit with a reinforcement learning model using the Bayesian estimation approach. Model fitting results indicated that both psychosis groups tend to update their reward values more rapidly than controls. Moreover, among the three groups, high-psychosis patients had the lowest degree of choice perseveration. Lumping patients' data together, we also found that patients' perseveration appears to be negatively correlated (p = 0.09, trending toward significance) with their PANSS p1 + p3 scores. Our method provides an alternative for investigating reward-related learning and decision making in basic and clinical settings.

BDNF Val66Met polymorphism influence on striatal blood-level-dependent response to monetary feedback depends on valence and agency

Animal work implicates the brain-derived neurotrophic factor (BDNF) in function of the ventral striatum (VS), a region known for its role in processing valenced feedback. Recent evidence in humans shows that BDNF Val66Met polymorphism modulates VS activity in anticipation of monetary feedback. However, it remains unclear whether the polymorphism impacts the processing of self-attributed feedback differently from feedback attributed to an external agent. In this study, we emphasize the importance of the feedback attribution because agency is central to computational accounts of the striatum and cognitive accounts of valence processing. We used functional magnetic resonance imaging and a task, in which financial gains/losses are either attributable to performance (self-attributed, SA) or chance (externally-attributed, EA) to ask whether BDNF Val66Met polymorphism predicts VS activity. We found that BDNF Val66Met polymorphism influenced how feedback valence and agency information were combined in the VS and in the right inferior frontal junction (IFJ). Specifically, Met carriers' VS response to valenced feedback depended on agency information, while Val/Val carriers' VS response did not. This context-specific modulation of valence effectively amplified VS responses to SA losses in Met carriers. The IFJ response to SA losses also differentiated Val/Val from Met carriers. These results may point to a reduced allocation of attention and altered motivational salience to SA losses in Val/Val compared to Met carriers. Implications for major depressive disorder are discussed.

Incentive salience attribution under reward uncertainty: A Pavlovian model

There is a vast literature on the behavioural effects of partial reinforcement in Pavlovian conditioning. Compared with animals receiving continuous reinforcement, partially rewarded animals typically show (a) a slower development of the conditioned response (CR) early in training and (b) a higher asymptotic level of the CR later in training. This phenomenon is known as the partial reinforcement acquisition effect (PRAE). Learning models of Pavlovian conditioning fail to account for it. In accordance with the incentive salience hypothesis, it is here argued that incentive motivation (or 'wanting') plays a more direct role in controlling behaviour than does learning, and reward uncertainty is shown to have an excitatory effect on incentive motivation. The psychological origin of that effect is discussed and a computational model integrating this new interpretation is developed. Many features of CRs under partial reinforcement emerge from this model.

Impulsivity links reward and threat sensitivities to substance use: a functional model

This study used structural equations modeling and undergraduate student data to examine the effects of reward and threat sensitivities on substance use, along with the extent to which impulsivity explained these effects. Our results suggest that impulsivity may translate inversely related reward and threat sensitivities into substance use, completely mediate the effect between threat sensitivity and substance use, and partially mediate the effect between reward sensitivity and substance use. Our results also suggest that individuals with a combination of higher levels on both reward and threat sensitivities may be most impulsive and vulnerable to heightened substance use. We discuss implications for research at the interface of personality and substance use and also substance abuse prevention and treatment.

Reduced dopamine transporter expression in the amygdala of subjects diagnosed with schizophrenia

A disruption of dopaminergic transmission in the amygdala of subjects with schizophrenia was proposed as a main contributor to pathophysiological and clinical manifestations of this disorder. We tested the hypothesis that the expression of the dopamine transporter (DAT) is decreased in the amygdala of subjects with schizophrenia. In normal control, schizophrenic subjects and bipolar disorder subjects, we measured numerical density of axon varicosities immunoreactive (IR) for DAT in the lateral (LN), basal, accessory basal (ABN), and cortical (CO) nuclei and intercalated cell masses (ITCM) of the amygdala. Tyrosine hydroxylase (TH)-IR and dopamine beta-hydroxylase (DBH)-IR varicosities were measured to test for potential loss of varicosities and serotonin transporter (5HTT)-IR for involvement of the serotoninergic system. Among several potential confounding variables tested, particular emphasis was placed on exposure to therapeutic drugs. In schizophrenic subjects, DAT-IR varicosities were decreased in LN (P = .0002), ABN (P = .013), and CO (P = .0001) in comparison with controls, and in comparison with bipolar disorder subjects in LN (P = .004) and CO (P = .002). DBH-IR varicosities were decreased in ABN (P = .008) and ITCM (P = .017), compared with controls. TH- and 5HTT-IR varicosities were not altered. No changes were detected in bipolar disorder. Taken together with TH and DBH findings, reductions of DAT-IR varicosities point to decreased DAT expression in dopaminergic terminals in the amygdala of subjects with schizophrenia. This DAT decrease may disrupt dopamine uptake, leading to increased dopaminergic synaptic transmission and spillage into the extracellular space with activation of extrasynaptic dopamine receptors. Concurrent decrease of noradrenaline in the ABN may disrupt memory consolidation.

Attribution and expression of incentive salience are differentially signaled by ultrasonic vocalizations in rats

During Pavlovian incentive learning, the affective properties of rewards are thought to be transferred to their predicting cues. However, how rewards are represented emotionally in animals is widely unknown. This study sought to determine whether 50-kHz ultrasonic vocalizations (USVs) in rats may signal such a state of incentive motivation to natural, nutritional rewards. To this end, rats learned to anticipate food rewards and, across experiments, the current physiological state (deprived vs. sated), the type of learning mechanism recruited (Pavlovian vs. instrumental), the hedonic properties of UCS (low vs. high palatable food), and the availability of food reward (continued vs. discontinued) were manipulated. Overall, we found that reward-cues elicited 50-kHz calls as they were signaling a putative affective state indicative of incentive motivation in the rat. Attribution and expression of incentive salience, however, seemed not to be an unified process, and could be teased apart in two different ways: 1) under high motivational state (i.e., hunger), the attribution of incentive salience to cues occurred without being expressed at the USVs level, if reward expectations were higher than the outcome; 2) in all experiments when food rewards were devalued by satiation, reward cues were still able to elicit USVs and conditioned anticipatory activity although reward seeking and consumption were drastically weakened. Our results suggest that rats are capable of representing rewards emotionally beyond apparent, immediate physiological demands. These findings may have translational potential in uncovering mechanisms underlying aberrant and persistent motivation as observed in drug addiction, gambling, and eating disorders.

Role of nicotinic acetylcholine receptors in regulating dopamine neuron activity

Midbrain dopamine (DA) neurons play a central role in a wide range of behaviors, from attention and motivation to motor control and reinforcement. The release of DA is modulated by a number of factors, and its deregulation has been implicated in multiple psychiatric disorders, such as addiction. In particular, nicotinic acetylcholine receptors (nAChRs) are key modulators of DA cells. Nicotine, the main addictive component in tobacco, strongly interacts with these receptors in the midbrain DA systems, resulting in reinforcing effects that are at the core of tobacco addiction. nAChRs are virtually expressed on every cell of the DA system, both at pre-, post- and extra-synaptic locations. The complex issue of interpreting the role of the large portfolio of different nAChR subtypes expressed on ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) neurons, and especially their role in defining functional DAergic subpopulations, is far from being solved. In this review we will try to provide the reader with an integrative view of the nicotinic modulation of DA neurons and its influence at the cellular, systemic and behavioral levels (exploratory behavior), as well as its implication in the reinforcing effects of nicotine.

Neural responses to exclusion predict susceptibility to social influence

PURPOSE

Social influence is prominent across the lifespan, but sensitivity to influence is especially high during adolescence and is often associated with increased risk taking. Such risk taking can have dire consequences. For example, in American adolescents, traffic-related crashes are leading causes of nonfatal injury and death. Neural measures may be especially useful in understanding the basic mechanisms of adolescents' vulnerability to peer influence.

METHODS

We examined neural responses to social exclusion as potential predictors of risk taking in the presence of peers in recently licensed adolescent drivers. Risk taking was assessed in a driving simulator session occurring approximately 1 week after the neuroimaging session.

RESULTS

Increased activity in neural systems associated with the distress of social exclusion and mentalizing during an exclusion episode predicted increased risk taking in the presence of a peer (controlling for solo risk behavior) during a driving simulator session outside the neuroimaging laboratory 1 week later. These neural measures predicted risky driving behavior above and beyond self-reports of susceptibility to peer pressure and distress during exclusion.

CONCLUSIONS

These results address the neural bases of social influence and risk taking; contribute to our understanding of social and emotional function in the adolescent brain; and link neural activity in specific, hypothesized, regions to risk-relevant outcomes beyond the neuroimaging laboratory. Results of this investigation are discussed in terms of the mechanisms underlying risk taking in adolescents and the public health implications for adolescent driving.

Neural responses to exclusion predict susceptibility to social influence

PURPOSE

Social influence is prominent across the lifespan, but sensitivity to influence is especially high during adolescence and is often associated with increased risk taking. Such risk taking can have dire consequences. For example, in American adolescents, traffic-related crashes are leading causes of nonfatal injury and death. Neural measures may be especially useful in understanding the basic mechanisms of adolescents' vulnerability to peer influence.

METHODS

We examined neural responses to social exclusion as potential predictors of risk taking in the presence of peers in recently licensed adolescent drivers. Risk taking was assessed in a driving simulator session occurring approximately 1 week after the neuroimaging session.

RESULTS

Increased activity in neural systems associated with the distress of social exclusion and mentalizing during an exclusion episode predicted increased risk taking in the presence of a peer (controlling for solo risk behavior) during a driving simulator session outside the neuroimaging laboratory 1 week later. These neural measures predicted risky driving behavior above and beyond self-reports of susceptibility to peer pressure and distress during exclusion.

CONCLUSIONS

These results address the neural bases of social influence and risk taking; contribute to our understanding of social and emotional function in the adolescent brain; and link neural activity in specific, hypothesized, regions to risk-relevant outcomes beyond the neuroimaging laboratory. Results of this investigation are discussed in terms of the mechanisms underlying risk taking in adolescents and the public health implications for adolescent driving.

Touch Massage, a Rewarding Experience

This study aims to describe and analyze healthy individuals’ expressed experiences of touch massage (TM). Fifteen healthy participants received whole body touch massage during 60 minutes for two separate occasions. Interviews were analyzed by narrative analysis. Four identifiable storyline was found, Touch massage as an essential need, in this storyline the participants talked about a desire and need for human touch and TM. Another storyline was about, Touch massage as a pleasurable experience and the participants talked about the pleasure of having had TM. In the third storyline Touch massage as a dynamic experience, the informants talked about things that could modulate the experience of receiving TM. In the last storyline, Touch massage influences self-awareness, the participants described how TM affected some of their psychological and physical experiences. Experiences of touch massage was in general described as pleasant sensations and the different storylines could be seen in the light of rewarding experiences.

The problem with value

Neural correlates of value have been extensively reported in a diverse set of brain regions. However, in many cases it is difficult to determine whether a particular neural response pattern corresponds to a value-signal per se as opposed to an array of alternative non-value related processes, such as outcome-identity coding, informational coding, encoding of autonomic and skeletomotor consequences, alongside previously described "salience" or "attentional" effects. Here, I review a number of experimental manipulations that can be used to test for value, and I identify the challenges in ascertaining whether a particular neural response is or is not a value signal. Finally, I emphasize that some non-value related signals may be especially informative as a means of providing insight into the nature of the decision-making related computations that are being implemented in a particular brain region.

The attention deficit hyperactivity disorder phenotype as a summation of deficits in executive functioning and reward sensitivity: does this explain its relationship with oppositional defiant disorder?

OBJECTIVE

Attention deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD) commonly coexist but the relationship between them has not been well explained. We present a view of the ADHD phenotype as a summation of deficits in executive functioning and reward sensitivity and apply this to explain its co-occurrence with ODD.

MODEL

ADHD relates to difficulty in sustaining concentration for completing tasks. This is associated with lack of reward due to lack of achievement. The unfavorable balance of effort to reward in ADHD is exacerbated in ODD, a condition in which lack of reward is responsible for an adverse effect on mood resulting in a negative attitude and oppositional behavior. We have called this model the "Mental effort-reward imbalances model".

CONCLUSION

This model predicts that oppositional symptoms would be highly prevalent in ADHD, to the extent that ODD would be a frequent co-morbidity.

Ventral striatal dopamine synthesis capacity is associated with individual differences in behavioral disinhibition

Pathological gambling, alongside addictive and antisocial disorders, forms part of a broad psychopathological spectrum of externalizing disorders, which share an underlying genetic vulnerability. The shared externalizing propensity is a highly heritable, continuously varying trait. Disinhibitory personality traits such as impulsivity and novelty seeking (NS) function as indicators of this broad shared externalizing tendency, which may reflect, at the neurobiological level, variation in the reactivity of dopaminergic (DAergic) brain reward systems centered on the ventral striatum (VS). Here, we examined whether individual differences in ventral striatal dopamine (DA) synthesis capacity were associated with individual variation in disinhibitory personality traits. Twelve healthy male volunteers underwent 6-[¹⁸F]Fluoro-L-DOPA (FDOPA) positron emission tomography (PET) scanning to measure striatal DA synthesis capacity, and completed a measure of disinhibited personality (NS). We found that levels of ventral, but not dorsal, striatal DA synthesis capacity were significantly correlated with inter-individual variation in disinhibitory personality traits, particularly a propensity for financial extravagance and irresponsibility. Our results are consistent with preclinical models of behavioral disinhibition and addiction proneness, and provide novel insights into the neurobiology of personality based vulnerability to pathological gambling and other externalizing disorders.

Neurophysiology of performance monitoring and adaptive behavior

Successful goal-directed behavior requires not only correct action selection, planning, and execution but also the ability to flexibly adapt behavior when performance problems occur or the environment changes. A prerequisite for determining the necessity, type, and magnitude of adjustments is to continuously monitor the course and outcome of one's actions. Feedback-control loops correcting deviations from intended states constitute a basic functional principle of adaptation at all levels of the nervous system. Here, we review the neurophysiology of evaluating action course and outcome with respect to their valence, i.e., reward and punishment, and initiating short- and long-term adaptations, learning, and decisions. Based on studies in humans and other mammals, we outline the physiological principles of performance monitoring and subsequent cognitive, motivational, autonomic, and behavioral adaptation and link them to the underlying neuroanatomy, neurochemistry, psychological theories, and computational models. We provide an overview of invasive and noninvasive systemic measures, such as electrophysiological, neuroimaging, and lesion data. We describe how a wide network of brain areas encompassing frontal cortices, basal ganglia, thalamus, and monoaminergic brain stem nuclei detects and evaluates deviations of actual from predicted states indicating changed action costs or outcomes. This information is used to learn and update stimulus and action values, guide action selection, and recruit adaptive mechanisms that compensate errors and optimize goal achievement.

Psychopathic traits modulate brain responses to drug cues in incarcerated offenders

Recent neuroscientific evidence indicates that psychopathy is associated with abnormal function and structure in limbic and paralimbic areas. Psychopathy and substance use disorders are highly comorbid, but clinical experience suggests that psychopaths abuse drugs for different reasons than non-psychopaths, and that psychopaths do not typically experience withdrawal and craving upon becoming incarcerated. These neurobiological abnormalities may be related to psychopaths' different motivations for—and symptoms of—drug use. This study examined the modulatory effect of psychopathic traits on the neurobiological craving response to pictorial drug stimuli. Drug-related pictures and neutral pictures were presented and rated by participants while hemodynamic activity was monitored using functional magnetic resonance imaging. These data were collected at two correctional facilities in New Mexico using the Mind Research Network mobile magnetic resonance imaging system. The sample comprised 137 incarcerated adult males and females (93 females) with histories of substance dependence. The outcome of interest was the relation between psychopathy scores (using the Hare Psychopathy Checklist-Revised) and hemodynamic activity associated with viewing drug-related pictures vs. neutral pictures. There was a negative association between psychopathy scores and hemodynamic activity for viewing drug-related cues in the anterior cingulate, posterior cingulate, hippocampus, amygdala, caudate, globus pallidus, and parts of the prefrontal cortex. Psychopathic traits modulate the neurobiological craving response and suggest that individual differences are important for understanding and treating substance abuse.

Dopamine, urges to smoke, and the relative salience of drug versus non-drug reward

When addicted individuals are exposed to drug-related stimuli, dopamine release is thought to mediate incentive salience attribution, increasing attentional bias, craving and drug seeking. It is unclear whether dopamine acts specifically on drug cues versus other rewards, and if these effects correspond with craving and other forms of cognitive bias. Here, we administered the dopamine D2/D3 agonist pramipexole (0.5 mg) to 16 tobacco smokers in a double-blind placebo-controlled crossover design. Visual fixations on smoking and money images were recorded alongside smoking urges and fluency tasks. Pramipexole attenuated a marked bias in initial orienting towards smoking relative to money but did not alter a maintained attentional bias towards smoking. Pramipexole decreased urges to smoke retrospectively after the task but not on a state scale. Fewer smoking words were generated after pramipexole but phonological and semantic fluency were preserved. Although these treatment effects did not correlate with each other, changes in initial orienting towards smoking and money were inversely related to baseline scores. In conclusion, pramipexole can reduce the salience of an addictive drug compared with other rewards and elicit corresponding changes in smoking urges and cognitive bias. These reward-specific and baseline-dependent effects support an 'inverted-U' shaped profile of dopamine in addiction.

Investigating habits: strategies, technologies and models

Understanding habits at a biological level requires a combination of behavioral observations and measures of ongoing neural activity. Theoretical frameworks as well as definitions of habitual behaviors emerging from classic behavioral research have been enriched by new approaches taking account of the identification of brain regions and circuits related to habitual behavior. Together, this combination of experimental and theoretical work has provided key insights into how brain circuits underlying action-learning and action-selection are organized, and how a balance between behavioral flexibility and fixity is achieved. New methods to monitor and manipulate neural activity in real time are allowing us to have a first look “under the hood” of a habit as it is formed and expressed. Here we discuss ideas emerging from such approaches. We pay special attention to the unexpected findings that have arisen from our own experiments suggesting that habitual behaviors likely require the simultaneous activity of multiple distinct components, or operators, seen as responsible for the contrasting dynamics of neural activity in both cortico-limbic and sensorimotor circuits recorded concurrently during different stages of habit learning. The neural dynamics identified thus far do not fully meet expectations derived from traditional models of the structure of habits, and the behavioral measures of habits that we have made also are not fully aligned with these models. We explore these new clues as opportunities to refine an understanding of habits.

Differential effects of acute stress on anticipatory and consummatory phases of reward processing

Anhedonia is one of the core symptoms of depression and has been linked to blunted responses to rewarding stimuli in striatal regions. Stress, a key vulnerability factor for depression, has been shown to induce anhedonic behavior, including reduced reward responsiveness in both animals and humans, but the brain processes associated with these effects remain largely unknown in humans. Emerging evidence suggests that stress has dissociable effects on distinct components of reward processing, as it has been found to potentiate motivation/'wanting' during the anticipatory phase but reduce reward responsiveness/'liking' during the consummatory phase. To examine the impact of stress on reward processing, we used a monetary incentive delay (MID) task and an acute stress manipulation (negative performance feedback) in conjunction with functional magnetic resonance imaging (fMRI). Fifteen healthy participants performed the MID task under no-stress and stress conditions. We hypothesized that stress would have dissociable effects on the anticipatory and consummatory phases in reward-related brain regions. Specifically, we expected reduced striatal responsiveness during reward consumption (mirroring patterns previously observed in clinical depression) and increased striatal activation during reward anticipation consistent with non-human findings. Supporting our hypotheses, significant Phase (Anticipation/Consumption)×Stress (Stress/No-stress) interactions emerged in the putamen, nucleus accumbens, caudate and amygdala. Post hoc tests revealed that stress increased striatal and amygdalar activation during anticipation but decreased striatal activation during consumption. Importantly, stress-induced striatal blunting was similar to the profile observed in clinical depression under baseline (no-stress) conditions in prior studies. Given that stress is a pivotal vulnerability factor for depression, these results offer insight to better understand the etiology of this prevalent disorder.

Vendor differences in alcohol consumption and the contribution of dopamine receptors to Pavlovian-conditioned alcohol-seeking in Long-Evans rats

RATIONALE

Drug-associated environmental stimuli elicit craving in humans and drug-seeking in animals.

OBJECTIVES

We tested the hypothesis that Pavlovian-conditioned alcohol-seeking is mediated by dopamine, using rats from two vendors.

METHODS

Male, Long-Evans rats (220-240 g) from Charles River (St-Constant, QC, Canada) and Harlan Laboratories (Indianapolis, IN, USA) received 21 sessions of intermittent, 24-h access to ethanol (15 %, v/v) and water in the home-cage. Subsequently, rats were trained to discriminate between one conditioned stimulus (CS+) that was paired with ethanol (0.2 ml per CS+) and a second stimulus (CS-) that was not. Entries into a fluid port where ethanol was delivered were recorded. Next, rats were exposed to a different context where cues and ethanol were withheld. At test, responding to the CS+ and CS- without ethanol was assessed in the second, non-alcohol context. Injections (1 ml/kg; s.c.) of the dopamine D1-receptor antagonist SCH 23390 (0, 3.33, and 10 μg/kg) or dopamine D2-receptor antagonist eticlopride (0, 5, and 10 μg/kg) were administered before test.

RESULTS

Home-cage alcohol consumption was higher in Harlan rats than Charles River rats. At test, saline-treated rats responded more to the alcohol-predictive CS+ than the CS-. While SCH 23390 attenuated CS+ responding in rats from both vendors, eticlopride reduced CS+ responding in Harlan rats only. Subsequently, SCH 23390 but not eticlopride attenuated CS+ responding when the CS+ was again paired with ethanol.

CONCLUSIONS

These results indicate important differences in alcohol consumption in Long-Evans rats from different suppliers, and highlight a novel role for dopamine in Pavlovian-conditioned alcohol-seeking.

Acute high fat diet consumption activates the mesolimbic circuit and requires orexin signaling in a mouse model

Overconsumption of palatable energy-dense foods has negative health implications and it is associated with obesity and several eating disorders. Currently, little is known about the neuronal circuitries activated by the acute ingestion of a rewarding stimulus. Here, we used a combination of immunohistochemistry, pharmacology and neuronal tracing analyses to examine the role of the mesolimbic system in general, and the orexin neurons in particular, in a simple experimental test in which naïve mice are allowed to spontaneously eat a pellet of a high fat diet (HFD) for 2 h. We found that acute HFD activates c-Fos expression in several reward-related brain areas, including the ventral tegmental area (VTA), nucleus accumbens, central amygdala and lateral hypothalamic area. We also found that: i- HFD-mediated orosensory stimulation was required for the mesolimbic pathway activation, ii- acute HFD differentially activates dopamine neurons of the paranigral, parabrachial pigmented and interfascicular sub-regions of the VTA, and iii- orexin neurons of the lateral hypothalamic area are responsive to acute HFD. Moreover, orexin signaling blockade, with the orexin 1 receptor antagonist SB-334867, reduces acute HFD consumption and c-Fos induction in the VTA but not in the other mesolimbic nuclei under study. Finally, we found that most orexin neurons responsive to acute HFD innervate the VTA. Our results show that acute HFD consumption recruits the mesolimbic system and that the full manifestation of this eating behavior requires the activation of orexin signaling.

Attentional bias for nondrug reward is magnified in addiction

Attentional biases for drug-related stimuli play a prominent role in addiction, predicting treatment outcomes. Attentional biases also develop for stimuli that have been paired with nondrug rewards in adults without a history of addiction, the magnitude of which is predicted by visual working-memory capacity and impulsiveness. We tested the hypothesis that addiction is associated with an increased attentional bias for nondrug (monetary) reward relative to that of healthy controls, and that this bias is related to working-memory impairments and increased impulsiveness. Seventeen patients receiving methadone-maintenance treatment for opioid dependence and 17 healthy controls participated. Impulsiveness was measured using the Barratt Impulsiveness Scale (BIS-11; Patton, Stanford, & Barratt, 1995), visual working-memory capacity was measured as the ability to recognize briefly presented color stimuli, and attentional bias was measured as the magnitude of response time slowing caused by irrelevant but previously reward-associated distractors in a visual-search task. The results showed that attention was biased toward the distractors across all participants, replicating previous findings. It is important to note, this bias was significantly greater in the patients than in the controls and was negatively correlated with visual working-memory capacity. Patients were also significantly more impulsive than controls as a group. Our findings demonstrate that patients in treatment for addiction experience greater difficulty ignoring stimuli associated with nondrug reward. This nonspecific reward-related bias could mediate the distracting quality of drug-related stimuli previously observed in addiction.

Striatal ups and downs: their roles in vulnerability to addictions in humans

Susceptibility to addictive behaviors has been related to both increases and decreases in striatal function. Both profiles have been reported in humans as well as in animal models. Yet, the mechanisms underlying these opposing effects and the manner in which they relate to the behavioral development and expression of addiction remain unclear. In the present review of human studies, we describe a number of factors that could influence whether striatal hyper- or hypo-function is observed and propose a model that integrates the influence of these opposite responses on the expression of addiction related behaviors. Central to this model is the role played by the presence versus absence of addiction related cues and their ability to regulate responding to abused drugs and other rewards. Striatal function and incentive motivational states are increased in the presence of these cues and decreased in their absence. Alternations between these states might account for the progressive narrowing of interests as addictions develop and point to relevant processes to target in treatment.

Individual variation in resisting temptation: implications for addiction

When exposed to the sights, sounds, smells and/or places that have been associated with rewards, such as food or drugs, some individuals have difficulty resisting the temptation to seek out and consume them. Others have less difficulty restraining themselves. Thus, Pavlovian reward cues may motivate maladaptive patterns of behavior to a greater extent in some individuals than in others. We are just beginning to understand the factors underlying individual differences in the extent to which reward cues acquire powerful motivational properties, and therefore, the ability to act as incentive stimuli. Here we review converging evidence from studies in both human and non-human animals suggesting that a subset of individuals are more "cue reactive", in that certain reward cues are more likely to attract these individuals to them and motivate actions to get them. We suggest that those individuals for whom Pavlovian reward cues become especially powerful incentives may be more vulnerable to impulse control disorders, such as binge eating and addiction.

The effects of amphetamine sensitization on conditioned inhibition during a Pavlovian-instrumental transfer task in rats

RATIONALE

Psychostimulant sensitization heightens behavioral and motivational responses to reward-associated stimuli; however, its effects on stimuli associated with reward absence are less understood.

OBJECTIVES

We examined whether amphetamine sensitization alters performance during Pavlovian-instrumental transfer (PIT) to conditioned excitors and inhibitors. We further sought to characterize the effects of amphetamine sensitization on learning versus performance by exposing rats to amphetamine prior to Pavlovian training or between training and test.

METHODS

Adult male Long-Evans rats were given conditioned inhibition (A+/AX-) and Pavlovian (B+) training, followed by variable-interval instrumental conditioning. Rats were sensitized to D-amphetamine (2 mg/kg daily injections for 7 days) or served as non-exposed controls. Rats were given a PIT test, in which they were presented with stimulus B alone or in compound with the conditioned inhibitor (BX).

RESULTS

During the PIT test, control rats significantly reduced instrumental responding on BX trials (to approximately 50 % of responding to B). Amphetamine sensitization prior to Pavlovian conditioning increased lever pressing on BX trials and reduced lever pressing on B trials compared to controls. Amphetamine sensitization between training and test increased lever pressing on B and BX trials compared to controls. No effects of sensitization were observed on conditioned food cup approach.

CONCLUSIONS

Amphetamine sensitization increases instrumental responding during PIT to a conditioned inhibitor by enhancing the excitation of conditioned stimuli and reducing the inhibition of conditioned inhibitors.

The effects of acute nicotine, chronic nicotine, and withdrawal from chronic nicotine on performance of a cued appetitive response

Nicotine is a widely used addictive drug, with an estimated 73 million Americans 12 years of age or older having used a tobacco product in the last month, despite documented risks to personal health. Nicotine alters cognitive processes, which include effects on attention and impulsivity, a mechanism that may contribute to the addictive properties of the drug. Individuals with a variety of psychological disorders ranging from attention deficit hyperactivity disorder (ADHD) to schizophrenia smoke at a higher rate than the rest of the population and show deficits in impulse control. The present studies evaluated the effects of acute, chronic, and withdrawal from chronic nicotine on an operant task that measured premature and signaled nose pokes, as well as performance efficiency in C57BL/6J mice. Results indicate that acute nicotine (0.09 mg/kg intraperitoneally) does not alter the acquisition of the task, but does significantly increase performance efficiency once the behavior has been learned. In contrast, chronic nicotine (0, 6.3, 12.6, and 36 mg/kg/day subcutaneously) and withdrawal from chronic nicotine had no effect on performance efficiency. These results suggest that initial nicotine use may have beneficial effects on inhibitory control, but these effects are not maintained with chronic nicotine consumption as tolerance develops. The findings may provide an explanation for higher rates of smoking in patients with impulse control issues, as the smoking may represent an initial attempt at self-medication.

How attentional boost interacts with reward: the effect of dopaminergic medications in Parkinson's disease

There is widespread evidence that dopamine is implicated in the regulation of reward and salience. However, it is less known how these processes interact with attention and recognition memory. To explore this question, we used the attentional boost test in patients with Parkinson's disease (PD) before and after the administration of dopaminergic medications. Participants performed a visual letter detection task (remembering rewarded target letters and ignoring distractor letters) while also viewing a series of photos of natural and urban scenes in the background of the letters. The aim of the game was to retrieve the target letter after each trial and to win as much virtual money as possible. The recognition of background scenes was not rewarded. We enrolled 26 drug-naïve, newly diagnosed patients with PD and 25 healthy controls who were evaluated at baseline and follow-up. Patients with PD received dopamine agonists (pramipexole, ropinirole, rotigotine) during the 12-week follow-up period. At baseline, we found intact attentional boost in patients with PD: they were able to recognize target-associated scenes similarly to controls. At follow-up, patients with PD outperformed controls for both target- and distractor-associated scenes, but not when scenes were presented without letters. The alerting, orienting and executive components of attention were intact in PD. Enhanced attentional boost was replicated in a smaller group of patients with PD (n = 15) receiving l-3,4-dihydroxyphenylalanine (L-DOPA). These results suggest that dopaminergic medications facilitate attentional boost for background information regardless of whether the central task (letter detection) is rewarded or not.

Dopamine on D2-like receptors "reboosts" dopamine D1-like receptor-mediated behavioural activation in rats licking for a isotonic NaCl solution

RATIONALE

We recently suggested that dopamine on D1-like receptors is involved in the activation of goal-directed responses and the level of response activation is "reboosted" on the basis of an evaluation process involving D2-like receptors assessing "response efficacy". A main piece of evidence in support of this hypothesis was the observation of an "extinction mimicry" effect in the time course of licking bursts after dopamine D2-like receptor blockade in rats licking for sucrose.

OBJECTIVES

The aim of this study was to determine whether the pattern of licking observed with sucrose as a reward could be reproduced in rats licking for a different reward (0.9% NaCl).

MATERIALS AND METHODS

We investigated the effects of the dopamine D1-like receptor antagonist SCH 23390 (0.01-0.04 mg/kg) and of the dopamine D2-like receptor antagonist raclopride (0.025-0.25 mg/kg) on the microstructure of licking for a 0.9% NaCl solution in 12-h water-deprived rats in 30-min sessions.

RESULTS

As previously observed with sucrose as a reward, raclopride reduced the size of licking bursts and produced on the burst number time course an "extinction mimicry" effect, while SCH 23390 reduced licking exclusively by reducing burst number.

CONCLUSIONS

These results are consistent with the proposed hypothesis and provide support to the use of the study of licking microstructure as a valid model not only for the investigation of the mechanisms governing ingestive behaviour but also for the investigation of the mechanisms underlying behavioural activation and the related evaluation processes.

Cue-evoked cocaine "craving": role of dopamine in the accumbens core

Drug-associated cues can acquire powerful motivational control over the behavior of addicts, and can contribute to relapse via multiple, dissociable mechanisms. Most preclinical models of relapse focus on only one of these mechanisms: the ability of drug cues to reinforce drug-seeking actions following a period of extinction training. However, in addicts, drug cues typically do not follow seeking actions; they precede them. They often produce relapse by evoking a conditioned motivational state ("wanting" or "craving") that instigates and/or invigorates drug-seeking behavior. Here we used a conflict-based relapse model to ask whether individual variation in the propensity to attribute incentive salience to reward cues predicts variation in the ability of a cocaine cue to produce conditioned motivation (craving) for cocaine. Following self-administration training, responding was curtailed by requiring rats to cross an electrified floor to take cocaine. The subsequent response-independent presentation of a cocaine-associated cue was sufficient to reinstate drug-seeking behavior, despite the continued presence of the adverse consequence. Importantly, there were large individual differences in the motivational properties of the cocaine cue, which were predicted by variation in the propensity to attribute incentive salience to a food cue. Finally, a dopamine antagonist injected into the nucleus accumbens core attenuated, and amphetamine facilitated, cue-evoked cocaine seeking, implicating dopamine signaling in cocaine cue-evoked craving. These data provide a promising preclinical approach for studying sources of individual variation in susceptibility to relapse due to conditioned craving and implicate mesolimbic dopamine in this process.

The error in total error reduction

Most models of human and animal learning assume that learning is proportional to the discrepancy between a delivered outcome and the outcome predicted by all cues present during that trial (i.e., total error across a stimulus compound). This total error reduction (TER) view has been implemented in connectionist and artificial neural network models to describe the conditions under which weights between units change. Electrophysiological work has revealed that the activity of dopamine neurons is correlated with the total error signal in models of reward learning. Similar neural mechanisms presumably support fear conditioning, human contingency learning, and other types of learning. Using a computational modeling approach, we compared several TER models of associative learning to an alternative model that rejects the TER assumption in favor of local error reduction (LER), which assumes that learning about each cue is proportional to the discrepancy between the delivered outcome and the outcome predicted by that specific cue on that trial. The LER model provided a better fit to the reviewed data than the TER models. Given the superiority of the LER model with the present data sets, acceptance of TER should be tempered.

Biomarkers in Parkinson's disease (recent update)

Parkinson's disease (PD) is the second most common neurodegenerative disorder mostly affecting the aging population over sixty. Cardinal symptoms including, tremors, muscle rigidity, drooping posture, drooling, walking difficulty, and autonomic symptoms appear when a significant number of nigrostriatal dopaminergic neurons are already destroyed. Hence we need early, sensitive, specific, and economical peripheral and/or central biomarker(s) for the differential diagnosis, prognosis, and treatment of PD. These can be classified as clinical, biochemical, genetic, proteomic, and neuroimaging biomarkers. Novel discoveries of genetic as well as nongenetic biomarkers may be utilized for the personalized treatment of PD during preclinical (premotor) and clinical (motor) stages. Premotor biomarkers including hyper-echogenicity of substantia nigra, olfactory and autonomic dysfunction, depression, hyposmia, deafness, REM sleep disorder, and impulsive behavior may be noticed during preclinical stage. Neuroimaging biomarkers (PET, SPECT, MRI), and neuropsychological deficits can facilitate differential diagnosis. Single-cell profiling of dopaminergic neurons has identified pyridoxal kinase and lysosomal ATPase as biomarker genes for PD prognosis. Promising biomarkers include: fluid biomarkers, neuromelanin antibodies, pathological forms of α-Syn, DJ-1, amyloid β and tau in the CSF, patterns of gene expression, metabolomics, urate, as well as protein profiling in the blood and CSF samples. Reduced brain regional N-acetyl-aspartate is a biomarker for the in vivo assessment of neuronal loss using magnetic resonance spectroscopy and T2 relaxation time with MRI. To confirm PD diagnosis, the PET biomarkers include [(18)F]-DOPA for estimating dopaminergic neurotransmission, [(18)F]dG for mitochondrial bioenergetics, [(18)F]BMS for mitochondrial complex-1, [(11)C](R)-PK11195 for microglial activation, SPECT imaging with (123)Iflupane and βCIT for dopamine transporter, and urinary salsolinol and 8-hydroxy, 2-deoxyguanosine for neuronal loss. This brief review describes the merits and limitations of recently discovered biomarkers and proposes coenzyme Q10, mitochondrial ubiquinone-NADH oxidoreductase, melatonin, α-synculein index, Charnoly body, and metallothioneins as novel biomarkers to confirm PD diagnosis for early and effective treatment of PD.

On the motivational properties of reward cues: Individual differences

Cues associated with rewards, such as food or drugs of abuse, can themselves acquire motivational properties. Acting as incentive stimuli, such cues can exert powerful control over motivated behavior, and in the case of cues associated with drugs, they can goad continued drug-seeking behavior and relapse. However, recent studies reviewed here suggest that there are large individual differences in the extent to which food and drug cues are attributed with incentive salience. Rats prone to approach reward cues (sign-trackers) attribute greater motivational value to discrete localizable cues and interoceptive cues than do rats less prone to approach reward cues (goal-trackers). In contrast, contextual cues appear to exert greater control over motivated behavior in goal-trackers than sign-trackers. It is possible to predict, therefore, before any experience with drugs, in which animals specific classes of drug cues will most likely reinstate drug-seeking behavior. The finding that different individuals may be sensitive to different triggers capable of motivating behavior and producing relapse suggests there may be different pathways to addiction, and has implications for thinking about individualized treatment. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'.

Role of nucleus accumbens dopamine receptor subtypes in the learning and expression of alcohol-seeking behavior

These studies examined the roles of dopamine D1- and D2-like receptors within the nucleus accumbens (Acb) in the acquisition and expression of ethanol-induced (2g/kg) conditioned place preference (CPP) in adult male DBA/2J mice. Bilateral intra-Acb infusions of the D1-like dopamine receptor antagonist SCH23390 (0.05, 0.5μg/side) or the D2-like dopamine receptor antagonist raclopride (0.5-5.0μg/side) were administered 30min before each ethanol conditioning trial (acquisition studies) or before preference tests (expression studies). CPP was conditioned to tactile cues using an unbiased apparatus and procedure. Intra-Acb infusion of SCH23390 prevented CPP acquisition, whereas intra-Acb infusion of raclopride did not. Intra-Acb infusion of both antagonists, however, dose-dependently reduced ethanol-stimulated locomotor activity during conditioning. In contrast, intra-Acb antagonist infusion had no effect on ethanol CPP expression, suggesting that dopamine's role in the Acb is limited to neurobiological processes engaged during the learning of the relationship between contextual cues and ethanol reward. Control experiments showed that intra-Acb injection of SCH23390 alone produced no place conditioning and did not interfere with the acquisition of conditioned place aversion induced by lithium chloride, suggesting that the antagonist's effect on ethanol CPP was not due to a more general detrimental effect on associative learning. Overall, these data suggest that D1-like (but not D2-like) dopamine Acb receptors play an important role in the learning of context-ethanol associations, either by modulating the magnitude of ethanol reward or the rate of learning about ethanol reward.

Development of behavioral preferences for the optimal choice following unexpected reward omission is mediated by a reduction of D2-like receptor tone in the nucleus accumbens

To survive in a dynamic environment, animals must identify changes in resource availability and rapidly apply adaptive strategies to obtain resources that promote survival. We have utilised a behavioral paradigm to assess differences in foraging strategy when resource (reward) availability unexpectedly changes. When reward magnitude was reduced by 50% (receive one reward pellet instead of two), male and female rats developed a preference for the optimal choice by the second session. However, when an expected reward was omitted (receive no reward pellets instead of one), subjects displayed a robust preference for the optimal choice during the very first session. Previous research shows that, when an expected reward is omitted, dopamine neurons phasically decrease their firing rate, which is hypothesised to decrease dopamine release preferentially affecting D2-like receptors. As robust changes in behavioral preference were specific to reward omission, we tested this hypothesis and the functional role of D1- and D2-like receptors in the nucleus accumbens in mediating the rapid development of a behavioral preference for the rewarded option during reward omission in male rats. Blockade of both receptor types had no effect on this behavior; however, holding D2-like, but not D1-like, receptor tone via infusion of dopamine receptor agonists prevented the development of the preference for the rewarded option during reward omission. These results demonstrate that avoiding an outcome that has been tagged with aversive motivational properties is facilitated through decreased dopamine transmission and subsequent functional disruption of D2-like, but not D1-like, receptor tone in the nucleus accumbens.

Dopamine or opioid stimulation of nucleus accumbens similarly amplify cue-triggered 'wanting' for reward: entire core and medial shell mapped as substrates for PIT enhancement

Pavlovian cues [conditioned stimulus (CS+)] often trigger intense motivation to pursue and consume related reward [unconditioned stimulus (UCS)]. But cues do not always trigger the same intensity of motivation. Encountering a reward cue can be more tempting on some occasions than on others. What makes the same cue trigger more intense motivation to pursue reward on a particular encounter? The answer may be the level of incentive salience ('wanting') that is dynamically generated by mesocorticolimbic brain systems, influenced especially by dopamine and opioid neurotransmission in the nucleus accumbens (NAc) at that moment. We tested the ability of dopamine stimulation (by amphetamine microinjection) vs. mu opioid stimulation [by d-Ala, nMe-Phe, Glyol-enkephalin (DAMGO) microinjection] of either the core or shell of the NAc to amplify cue-triggered levels of motivation to pursue sucrose reward, measured with a Pavlovian-Instrumental Transfer (PIT) procedure, a relatively pure assay of incentive salience. Cue-triggered 'wanting' in PIT was enhanced by amphetamine or DAMGO microinjections equally, and also equally at nearly all sites throughout the entire core and medial shell (except for a small far-rostral strip of shell). NAc dopamine/opioid stimulations specifically enhanced CS+ ability to trigger phasic peaks of 'wanting' to obtain UCS, without altering baseline efforts when CS+ was absent. We conclude that dopamine/opioid stimulation throughout nearly the entire NAc can causally amplify the reactivity of mesocorticolimbic circuits, and so magnify incentive salience or phasic UCS 'wanting' peaks triggered by a CS+. Mesolimbic amplification of incentive salience may explain why a particular cue encounter can become irresistibly tempting, even when previous encounters were successfully resisted before.

Presynaptic leptin action suppresses excitatory synaptic transmission onto ventral tegmental area dopamine neurons

BACKGROUND

Leptin is an adipocyte-derived cytokine that can act in the brain to suppress feeding and maintain energy homeostasis. Additionally, leptin activates its receptors in the ventral tegmental area (VTA), a critical site for neuroadaptations to rewarding stimuli, to modulate reward-seeking behaviors. Although leptin can decrease intrinsic excitability of dopamine neurons in the VTA, it is unknown whether leptin can modulate excitatory synaptic transmission in this brain region. Because plasticity of glutamatergic synapses onto VTA neurons can encode predictive information about reward, we hypothesized that leptin can decrease excitatory synaptic transmission onto dopamine neurons.

METHODS

Using whole-cell patch clamp electrophysiology in mouse midbrain slices, we tested the effects of leptin on evoked α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor (AMPAR) or N-methyl-D-aspartate receptor (NMDAR)-mediated excitatory postsynaptic currents (EPSCs) onto VTA dopamine neurons.

RESULTS

Leptin depressed both AMPAR and NMDAR EPSCs in VTA dopamine neurons and reduced frequency but not amplitude of mini EPSCs. Bath application of the MEK1/2 inhibitor U0126 did not alter leptin-induced suppression of AMPAR EPSCs. However, external, but not internal, application of the phosphoinositol 3-kinase (PI3K) or Janus kinase 2 (Jak2) tyrosine kinase inhibitors abolished leptin-induced synaptic depression.

CONCLUSIONS

This study demonstrates that leptin causes a presynaptic inhibition of the probability of glutamate release onto VTA dopamine neurons. This synaptic inhibition requires Jak2 and PI3K activation. Leptin-induced weakening of synaptic strength onto dopamine cells may underlie its inhibitory effects on appetitive behavior for rewarding stimuli.

Substance-specific environmental influences on drug use and drug preference in animals and humans

Epidemiological, clinical, and preclinical evidence indicate that the setting of drug use can exert a powerful modulatory influence on drug reward and that this influence is substance-specific. When heroin and cocaine co-abusers, for example, report on the circumstances of drug use, they indicate distinct settings for the two drugs: heroin being used preferentially at home and cocaine being used preferentially outside the home. Similar results were obtained in laboratory rats. These findings will be interpreted in the light of a novel model of drug reward, based on the emotional appraisal of central and peripheral drug effects as a function of environmental context. I argue here that drug addiction research has not paid sufficient attention to the substance-specific aspects of drug abuse and this may have contributed to the present dearth of effective treatments. Pharmacological and cognitive-behavioral therapy, for example, should be tailored so as to allow the addict to anticipate, and cope with, the risks associated, in a substance-specific manner, to the different settings of drug use.