Phasic Dopamine Transmission Reflects Initiation Vigor and Exerted Effort in an Action- and Region-Specific Manner

Free will: An Example of the Dopaminergic System

Neuroscience has convinced people that much of their behavior is determined by causes unknown to them and beyond their control. However, are advances in neuroscience truly a prerequisite for such beliefs? Robert Kane's theory of ultimate responsibility is libertarian theory. Its innovative nature makes it possible to discuss the neurophysiological basis of its postulates. Using the functions of the midbrain dopaminergic system as an example, this article provides an overview of this neurophysiological basis. According to Kane, if we are to be ultimately responsible for our wills as well as for our actions, some actions in our lives must lack sufficient motives and causes. These are self-forming actions. Dopamine is hypothesized to mediate self-forming action execution. Dopamine not only mediates action but also ensures synaptic plasticity in the brain, that is, learning from action; hence, dopamine changes the acting individual and provides the formation of our own wills. The basal ganglia, which are the main target of dopamine in the brain, act through parallel pathways and are involved in decision-making processes. Dopamine is also involved in the regulation of the neurodynamical properties of prefrontal cortex networks with random spiking noise. It can be assumed that the activity of the dopaminergic system is closely related to the physiological basis of free will.

Dissociable encoding of motivated behavior by parallel thalamo-striatal projections

The successful pursuit of goals requires the coordinated execution and termination of actions that lead to positive outcomes. This process is thought to rely on motivational states that are guided by internal drivers, such as hunger or fear. However, the mechanisms by which the brain tracks motivational states to shape instrumental actions are not fully understood. The paraventricular nucleus of the thalamus (PVT) is a midline thalamic nucleus that shapes motivated behaviors via its projections to the nucleus accumbens (NAc)1-8 and monitors internal state via interoceptive inputs from the hypothalamus and brainstem3,9-14. Recent studies indicate that the PVT can be subdivided into two major neuronal subpopulations, namely PVTD2(+) and PVTD2(-), which differ in genetic identity, functionality, and anatomical connectivity to other brain regions, including the NAc4,15,16. In this study, we used fiber photometry to investigate the in vivo dynamics of these two distinct PVT neuronal types in mice performing a reward foraging-like behavioral task. We discovered that PVTD2(+) and PVTD2(-) neurons encode the execution and termination of goal-oriented actions, respectively. Furthermore, activity in the PVTD2(+) neuronal population mirrored motivation parameters such as vigor and satiety. Similarly, PVTD2(-) neurons, also mirrored some of these parameters but to a much lesser extent. Importantly, these features were largely preserved when activity in PVT projections to the NAc was selectively assessed. Collectively, our results highlight the existence of two parallel thalamo-striatal projections that participate in the dynamic regulation of goal pursuits and provide insight into the mechanisms by which the brain tracks motivational states to shape instrumental actions.

A reinforcement learning model with choice traces for a progressive ratio schedule

The progressive ratio (PR) lever-press task serves as a benchmark for assessing goal-oriented motivation. However, a well-recognized limitation of the PR task is that only a single data point, known as the breakpoint, is obtained from an entire session as a barometer of motivation. Because the breakpoint is defined as the final ratio of responses achieved in a PR session, variations in choice behavior during the PR task cannot be captured. We addressed this limitation by constructing four reinforcement learning models: a simple Q-learning model, an asymmetric model with two learning rates, a perseverance model with choice traces, and a perseverance model without learning. These models incorporated three behavioral choices: reinforced and non-reinforced lever presses and void magazine nosepokes, because we noticed that male mice performed frequent magazine nosepokes during PR tasks. The best model was the perseverance model, which predicted a gradual reduction in amplitudes of reward prediction errors (RPEs) upon void magazine nosepokes. We confirmed the prediction experimentally with fiber photometry of extracellular dopamine (DA) dynamics in the ventral striatum of male mice using a fluorescent protein (genetically encoded GPCR activation-based DA sensor: GRABDA2m). We verified application of the model by acute intraperitoneal injection of low-dose methamphetamine (METH) before a PR task, which increased the frequency of magazine nosepokes during the PR session without changing the breakpoint. The perseverance model captured behavioral modulation as a result of increased initial action values, which are customarily set to zero and disregarded in reinforcement learning analysis. Our findings suggest that the perseverance model reveals the effects of psychoactive drugs on choice behaviors during PR tasks.

Slow-rising and fast-falling dopaminergic dynamics jointly adjust negative prediction error in the ventral striatum

The greater the reward expectations are, the more different the brain's physiological response will be. Although it is well-documented that better-than-expected outcomes are encoded quantitatively via midbrain dopaminergic (DA) activity, it has been less addressed experimentally whether worse-than-expected outcomes are expressed quantitatively as well. We show that larger reward expectations upon unexpected reward omissions are associated with the preceding slower rise and following larger decrease (DA dip) in the DA concentration at the ventral striatum of mice. We set up a lever press task on a fixed ratio (FR) schedule requiring five lever presses as an effort for a food reward (FR5). The mice occasionally checked the food magazine without a reward before completing the task. The percentage of this premature magazine entry (PME) increased as the number of lever presses approached five, showing rising expectations with increasing proximity to task completion, and hence greater reward expectations. Fibre photometry of extracellular DA dynamics in the ventral striatum using a fluorescent protein (genetically encoded GPCR activation-based DA sensor: GRABDA2m ) revealed that the slow increase and fast decrease in DA levels around PMEs were correlated with the PME percentage, demonstrating a monotonic relationship between the DA dip amplitude and degree of expectations. Computational modelling of the lever press task implementing temporal difference errors and state transitions replicated the observed correlation between the PME frequency and DA dip amplitude in the FR5 task. Taken together, these findings indicate that the DA dip amplitude represents the degree of reward expectations monotonically, which may guide behavioural adjustment.

Inhibitory Pedunculopontine Neurons Gate Dopamine-Mediated Motor Actions of Unsigned Valence

BACKGROUND

The pedunculopontine nucleus (PPN) maintains a bidirectional connectivity with the basal ganglia that supports their shared roles in the selection and execution of motor actions. Previous studies identified a role for PPN neurons in goal-directed behavior, but the cellular substrates underlying this function have not been elucidated. We recently revealed the existence of a monosynaptic GABAergic input from the PPN that inhibits dopamine neurons of the substantia nigra. Activation of this pathway interferes with the execution of learned motor sequences when the actions are rewarded, even though the inhibition of dopamine neurons did not shift the value of the action, hence suggesting executive control over the gating of behavior.

OBJECTIVE

To test the attributes of the inhibition of dopamine neurons by the PPN in the context of goal-directed behavior regardless of whether the outcome is positive or negative.

METHODS

We delivered optogenetic stimulation to PPN GABAergic axon terminals in the substantia nigra during a battery of behavioral tasks with positive and negative valence.

RESULTS

Inhibition of dopamine neurons by PPN optogenetic activation during an appetitive task impaired the initiation and overall execution of the behavioral sequence without affecting the consumption of reward. During an active avoidance task, the same activation impaired the ability of mice to avoid a foot shock, but their escape response was unaffected. In addition, responses to potential threats were significantly attenuated.

CONCLUSION

Our results show that PPN GABAergic neurons modulate learned, goal-directed behavior of unsigned valence without affecting overall motor behavior.

Ventral Tegmental Area M5 Muscarinic Receptors Mediate Effort-Choice Responding and Nucleus Accumbens Dopamine in a Sex-Specific Manner

Optimization of effort-related choices is impaired in depressive disorders. Acetylcholine (ACh) and dopamine (DA) are linked to depressive disorders, and modulation of ACh tone in the ventral tegmental area (VTA) affects mood-related behavioral responses in rats. However, it is unknown if VTA ACh mediates effort-choice behaviors. Using a task of effort-choice, rats can choose to lever press on a fixed-ratio 5 (FR5) schedule for a more-preferred food or consume freely available, less-preferred food. VTA administration of physostigmine (1 μg and 2 μg/side), a cholinesterase inhibitor, reduced FR5 responding for the more-preferred food while leaving consumption of the less-preferred food intact. VTA infusion of the M5 muscarinic receptor negative allosteric modulator VU6000181 (3 μM, 10 μM, 30 μM/side) did not affect lever pressing or chow consumption. However, VU6000181 (30 μM/side) coadministration with physostigmine (2 μg/side) attenuated physostigmine-induced decrease in lever pressing in female and male rats and significantly elevated lever pressing above vehicle baseline levels in male rats. In in vivo voltammetry experiments, VTA infusion of combined physostigmine and VU6000181 did not significantly alter evoked phasic DA release in the nucleus accumbens core (NAc) in female rats. In male rats, combined VTA infusion of physostigmine and VU6000181 increased phasic evoked DA release in the NAc compared with vehicle, physostigmine, or VU6000181 infusion alone. These data indicate a critical role and potential sex differences of VTA M5 receptors in mediating VTA cholinergic effects on effort choice behavior and regulation of DA release. SIGNIFICANCE STATEMENT: Effort-choice impairments are observed in depressive disorders, which are often treatment resistant to currently available thymoleptics. The role of ventral tegmental area (VTA) acetylcholine muscarinic M5 receptors, in a preclinical model of effort-choice behavior, is examined. Using the selective negative allosteric modulator of the M5 receptor VU6000181, we show the role of VTA M5 receptors on effort-choice and regulation of dopamine release in the nucleus accumbens core. This study supports M5 receptors as therapeutic targets for depression.

Nucleus Accumbens D1 Receptor-Expressing Spiny Projection Neurons Control Food Motivation and Obesity

BACKGROUND

Obesity is a chronic relapsing disorder that is caused by an excess of caloric intake relative to energy expenditure. There is growing recognition that food motivation is altered in people with obesity. However, it remains unclear how brain circuits that control food motivation are altered in obese animals.

METHODS

Using a novel behavioral assay that quantifies work during food seeking, in vivo and ex vivo cell-specific recordings, and a synaptic blocking technique, we tested the hypothesis that activity of circuits promoting appetitive behavior in the core of the nucleus accumbens (NAc) is enhanced in the obese state, particularly during food seeking.

RESULTS

We first confirmed that mice made obese with ad libitum exposure to a high fat diet work harder than lean mice to obtain food, consistent with an increase in food motivation in obese mice. We observed greater activation of D₁ receptor-expressing NAc spiny projection neurons (NAc D1^SPNs) during food seeking in obese mice relative to lean mice. This enhanced activity was not observed in D₂ receptor-expressing neurons (D2^SPNs). Consistent with these in vivo findings, both intrinsic excitability and excitatory drive onto D1^SPNs were enhanced in obese mice relative to lean mice, and these measures were selective for D1^SPNs. Finally, blocking synaptic transmission from D1^SPNs, but not D2^SPNs, in the NAc core decreased physical work during food seeking and, critically, attenuated high fat diet-induced weight gain.

CONCLUSIONS

These experiments demonstrate the necessity of NAc core D1^SPNs in food motivation and the development of diet-induced obesity, establishing these neurons as a potential therapeutic target for preventing obesity.

Intra-NAc insulin reduces the motivation for food and food intake without altering cue-triggered food-seeking

Insulin receptors are expressed throughout the adult brain, and insulin from the periphery reaches the central nervous system. In humans and rodents, actions of insulin in the brain decrease food intake. Furthermore, insulin receptor activation alters dopamine and glutamate transmission within mesolimbic regions that influence food-seeking and feeding including the nucleus accumbens (NAc). Here we determined how intra-NAc insulin affects conditioned approach (a measure of cue-triggered food-seeking), free food intake, and the motivation to obtain food in hungry rats using Pavlovian and instrumental approaches. Intra-NAc insulin did not affect conditioned approach but did reduce home cage chow intake immediately following conditioned approach testing. Consistent with reduced chow intake, intra-NAc insulin also reduced the motivation to work for flavored food pellets (assessed by a progressive ratio procedure). This effect was partially reversed by insulin receptor blockade and was not driven by insulin-induced sickness or malaise. Taken together, these data show that insulin within the NAc does not alter behavioral responses to a food cue, but instead reduces the motivation to work for and consume food in hungry animals. These data are discussed in light of insulin's role in the regulation of feeding, and its dysregulation by obesity.

A self-adjusting, progressive shock strength procedure to investigate resistance to punishment: Characterization in male and female rats

Indifference to harmful consequences is one of the main characteristics of compulsive behaviors and addiction. Animal models that provide a rapid and effective measure of resistance to punishment could be critical for the investigation of mechanisms underlying these maladaptive behaviors. Here, analogous to the progressive ratio (PR) procedure widely used to evaluate appetitive motivation as the response requirement is increased, we developed a self-adjusting, progressive shock strength (PSS) procedure. The PSS provides, within a single session, a break point that quantifies the propensity to work for a reward in spite of receiving electric footshock that progressively increases in duration. In both male and female rats, the PSS break point was sensitive to 1) hunger; and 2) changes in the qualitative, but not quantitative, incentive value of the reward. In systematic comparisons between PSS and PR procedures in the same rats, we found that both measures are sensitive to manipulations of motivational states, but they are not intercorrelated, suggesting that they measure overlapping but partially distinct processes. Importantly, the PSS procedure represents a refinement in the 3Rs principles of animal research because animals can control the strength of shock that they are willing to tolerate. This self-adjusting PSS procedure may represent a useful tool to investigate mechanisms underlying maladaptive behavior that persists in certain individuals despite harmful consequences.

Does Traumatic Brain Injury Cause Risky Substance Use or Substance Use Disorder?

There is a high co-occurrence of risky substance use among adults with traumatic brain injury (TBI), although it is unknown if the neurologic sequelae of TBI can promote this behavior. We propose that to conclude that TBI can cause risky substance use, it must be determined that TBI precedes risky substance use, that confounders with the potential to increase the likelihood of both TBI and risky substance use must be ruled out, and that there must be a plausible mechanism of action. In this review, we address these factors by providing an overview of key clinical and preclinical studies and list plausible mechanisms by which TBI could increase risky substance use. Human and animal studies have identified an association between TBI and risky substance use, although the strength of this association varies. Factors that may limit detection of this relationship include differential variability due to substance, sex, age of injury, and confounders that may influence the likelihood of both TBI and risky substance use. We propose possible mechanisms by which TBI could increase substance use that include damage-associated neuroplasticity, chronic changes in neuroimmune signaling, and TBI-associated alterations in brain networks.

A hunger for warmth

Humans and animals alike perform behaviors-like putting on a sweater or building a warm nest-to regulate body temperature. In this issue of Neuron, Jung et al. (2022) reveal a parabrachial nucleus-to-lateral hypothalamus circuit that regulates thermoregulatory behavior, a circuit distinct from that which governs motivated feeding behavior.

5-HT2C receptor perturbation has bidirectional influence over instrumental vigour and restraint

The serotonin (5-HT) system, particularly the 5-HT2C receptor, has consistently been implicated in behavioural control. However, while some studies have focused on the role 5-HT2C receptors play in regulating motivation to work for reward, others have highlighted its importance in response restraint. To date, it is unclear how 5-HT transmission at this receptor regulates the balance of response invigoration and restraint in anticipation of future reward. In addition, it remains to be established how 5-HT2C receptors gate the influence of internal versus cue-driven processes over reward-guided actions. To elucidate these issues, we investigated the effects of administering the 5-HT2C receptor antagonist SB242084, both systemically and directly into the nucleus accumbens core (NAcC), in rats performing a Go/No-Go task for small or large rewards. The results were compared to the administration of d-amphetamine into the NAcC, which has previously been shown to promote behavioural activation. Systemic perturbation of 5-HT2C receptors-but crucially not intra-NAcC infusions-consistently boosted rats' performance and instrumental vigour on Go trials when they were required to act. Concomitantly, systemic administration also reduced their ability to withhold responding for rewards on No-Go trials, particularly late in the holding period. Notably, these effects were often apparent only when the reward on offer was small. By contrast, inducing a hyperdopaminergic state in the NAcC with d-amphetamine strongly impaired response restraint on No-Go trials both early and late in the holding period, as well as speeding action initiation. Together, these findings suggest that 5-HT2C receptor transmission, outside the NAcC, shapes the vigour of ongoing goal-directed action as well as the likelihood of responding as a function of expected reward.

A genetically defined insula-brainstem circuit selectively controls motivational vigor

The anterior insular cortex (aIC) plays a critical role in cognitive and motivational control of behavior, but the underlying neural mechanism remains elusive. Here, we show that aIC neurons expressing Fezf2 (aICFezf2), which are the pyramidal tract neurons, signal motivational vigor and invigorate need-seeking behavior through projections to the brainstem nucleus tractus solitarii (NTS). aICFezf2 neurons and their postsynaptic NTS neurons acquire anticipatory activity through learning, which encodes the perceived value and the vigor of actions to pursue homeostatic needs. Correspondingly, aIC → NTS circuit activity controls vigor, effort, and striatal dopamine release but only if the action is learned and the outcome is needed. Notably, aICFezf2 neurons do not represent taste or valence. Moreover, aIC → NTS activity neither drives reinforcement nor influences total consumption. These results pinpoint specific functions of aIC → NTS circuit for selectively controlling motivational vigor and suggest that motivation is subserved, in part, by aIC's top-down regulation of dopamine signaling.

Dopamine release and its control over early Pavlovian learning differs between the NAc core and medial NAc shell

Dopamine neurons respond to cues to reflect the value of associated outcomes. These cue-evoked dopamine responses can encode the relative rate of reward in rats with extensive Pavlovian training. Specifically, a cue that always follows the previous reward by a short delay (high reward rate) evokes a larger dopamine response in the nucleus accumbens (NAc) core relative to a distinct cue that always follows the prior reward by a long delay (low reward rate). However, it was unclear if these reward rate dopamine signals are evident during early Pavlovian training sessions and across NAc subregions. To address this, we performed fast-scan cyclic voltammetry recordings of dopamine levels to track the pattern of cue- and reward-evoked dopamine signals in the NAc core and medial NAc shell. We identified regional differences in the progression of cue-evoked dopamine signals across training. However, the dopamine response to cues did not reflect the reward rate in either the NAc core or the medial NAc shell during early training sessions. Pharmacological experiments found that dopamine-sensitive conditioned responding emerged in the NAc core before the medial NAc shell. Together, these findings illustrate regional differences in NAc dopamine release and its control over behavior during early Pavlovian learning.

Distinct regions of the striatum underlying effort, movement initiation and effort discounting

The ventral striatum is believed to encode the subjective value of cost-benefit options; however, this effect has notably been absent during choices that involve physical effort. Previous work in freely moving animals has revealed opposing striatal signals, with greater response to increasing effort demands and reduced responses to rewards requiring effort. Yet, the relationship between these conflicting signals remains unknown. Using functional magnetic resonance imaging with a naturalistic maze-navigation paradigm, we identified functionally segregated regions within the ventral striatum that separately encoded effort activation, movement initiation and effort discounting of rewards. In addition, activity in regions associated with effort activation and discounting oppositely predicted striatal encoding of effort during effort-based decision-making. Our results suggest that the dorsomedial region hitherto associated with action may instead represent the cost of effort and raise fundamental questions regarding the interpretation of striatal 'reward' signals in the context of effort demands. This has implications for uncovering the neural architecture underlying motivated behaviour.

Pharmacological evidence for the implication of noradrenaline in effort

The trade-off between effort and reward is one of the main determinants of behavior, and its alteration is at the heart of major disorders such as depression or Parkinson's disease. Monoaminergic neuromodulators are thought to play a key role in this trade-off, but their relative contribution remains unclear. Rhesus monkeys (Macaca mulatta) performed a choice task requiring a trade-off between the volume of fluid reward and the amount of force to be exerted on a grip. In line with a causal role of noradrenaline in effort, decreasing noradrenaline levels with systemic clonidine injections (0.01 mg/kg) decreased exerted force and enhanced the weight of upcoming force on choices, without any effect on reward sensitivity. Using computational modeling, we showed that a single variable ("effort") could capture the amount of resources necessary for action and control both choices (as a variable for decision) and force production (as a driving force). Critically, the multiple effects of noradrenaline manipulation on behavior could be captured by a specific modulation of this single variable. Thus, our data strongly support noradrenaline's implication in effort processing.

Glucagon-Like Peptide-1 Receptor Signaling in the Ventral Tegmental Area Reduces Alcohol Self-Administration in Male Rats

BACKGROUND

The misuse and abuse of alcohol is a major public health issue. However, available treatments are limited with variable efficacy. Recently, preclinical studies show that glucagon-like-peptide-1 (GLP-1) and its analogue Exendin-4 (Ex4) potently reduce a range of alcohol intake behaviors, thus highlighting its potential as a treatment for alcohol use disorders. However, the neural mechanisms and sites of action mediating the effects of Ex4 on alcohol intake behaviors remain to be characterized. This study examined the ventral tegmental area (VTA) as a site of action for the effects of GLP-1 on alcohol intake.

METHODS

Male Long-Evans rats were given intermittent access to 20% alcohol and trained to nose poke for 20% alcohol. Rats received intra-VTA injections of Ex4 (vehicle, 0.01, 0.05 μg), and the effects of VTA Ex4 on alcohol self-administration, motivation, and relapse were assessed.

RESULTS

When compared to vehicle treatment, intra-VTA Ex4 (0.01, 0.05 μg) delivery significantly reduced alcohol self-administration, an effect that was particularly prominent in high alcohol drinkers. However, VTA Ex4 did not reduce reacquisition of alcohol self-administration after extinction nor the motivation to obtain alcohol. Importantly, the lower dose of Ex4 (0.01 μg) used had no effect on food intake or locomotor activity, suggesting that the reduction in alcohol self-administration observed was not secondary to caloric intake or motor deficits.

CONCLUSIONS

Together, these findings provide support for the VTA as a key site of action for GLP-1 on alcohol self-administration but not the reacquisition of alcohol self-administration or motivation to work for alcohol.

Rat Paraventricular Neurons Encode Predictive and Incentive Information of Reward Cues

The paraventricular nucleus of the thalamus (PVT) has been implicated in cue-induced motivated behaviors. Although reward-associated cues (conditioned stimuli, CSs) contain different types of information including predictive information of future reward delivery and incentive (motivational) value of the reward, it remains unknown whether PVT neurons represent predictive and incentive information of CSs. It is suggested that neural activity just after the onset of CSs (early activity) and that just before reward delivery (late activity) might more strongly represent predictive and incentive information, respectively. In this study, rats were trained to lick a tube, which was protruded close to their mouth just after a CS, to obtain a reward (sucrose or water) (cue-induced licking task). Auditory and visual CSs were used: each elemental cue (CS) predicted reward or non-reward outcome, while simultaneous presentation of the two elemental cues (configural cues) predicted the opposite reward outcome. We recorded PVT neurons in the cue-induced licking task, and report that half of the CS-responsive PVT neurons responded selectively to the CSs predicting reward outcome regardless of physical property of the cues (CS+-selective). In addition, the early activity of the CS+-selective neurons discriminated reward/non-reward association (predictive information) and was less sensitive to reward value and motivation reflected by lick latency (incentive information), while the late activity of the CS+-selective neurons was correlated with reward value and motivation rather than reward/non-reward association. Early and late population activity of the CS+-selective neurons also represented predictive and incentive information of the CSs, respectively. On the other hand, activity of more than half of the PVT neurons was correlated with individual licking during licking to acquire reward. Taken together, the results suggest that the PVT neurons engage in different neural processes involved in cue-induced motivated behaviors: CS encoding to determine reward availability and form motivation for reward-seeking behavior, and hedonic mouth movements during reward consumption.

Heterogeneity in striatal dopamine circuits: Form and function in dynamic reward seeking

The striatal dopamine system has long been studied in the context of reward learning, motivation, and movement. Given the prominent role dopamine plays in a variety of adaptive behavioral states, as well as diseases like addiction, it is essential to understand the full complexity of dopamine neurons and the striatal systems they target. A growing number of studies are uncovering details of the heterogeneity in dopamine neuron subpopulations. Here, we review that work to synthesize current understanding of dopamine system heterogeneity across three levels, anatomical organization, functions in behavior, and modes of action, wherein we focus on signaling profiles and local mechanisms for modulation of dopamine release. Together, these studies reveal new and emerging dimensions of the striatal dopamine system, informing its contribution to dynamic motivational and decision-making processes.

Preadolescent dopamine receptor antagonism increases postadolescent reward-related operant behaviors that may depend on dopamine receptor hypersensitivity

The dopaminergic system has a long history of being associated with reward-related activities but the developmental consequences of blocking dopamine receptor function on reward-based associative learning has been less studied. To this end, male, Long Evans rats were systemically (i.p.) treated with the dopamine receptor (DAr) antagonist, flupenthixol (0.25 mg/kg), or saline, from postnatal day (P)18 - 24 (preadolescence) then trained on an operant conditioning task from P41 - P45 (postadolescent) without drug treatment. The preadolescent flupenthixol group showed elevated active lever responses and locomotor activity during the drug-free test. Another group of rats was given flupenthixol prior to each acquisition session from P41 - 45 which significantly suppressed both active lever presses and locomotor activity. Separate groups of rats were treated with flupenthixol or saline from P18 - 24 then treated with apomorphine or saline on P41 followed by assessment of c-Fos labeling in the nucleus accumbens. Early flupenthixol treatment was associated with more apomorphine-induced c-Fos labeling in the nucleus accumbens shell than the early saline-apomorphine group, indicating a sensitized response. These findings suggest that preadolescent dopamine receptor blockade may lead to a sensitized postadolescent dopaminergic response that underlies enhanced behavioral responses in the presence of rewarding stimuli.

Acute Stress Enhances Associative Learning via Dopamine Signaling in the Ventral Lateral Striatum

Acute stress transiently increases vigilance, enhancing the detection of salient stimuli in one's environment. This increased perceptual sensitivity is thought to promote the association of rewarding outcomes with relevant cues. The mesolimbic dopamine system is critical for learning cue-reward associations. Dopamine levels in the ventral striatum are elevated following exposure to stress. Together, this suggests that the mesolimbic dopamine system could mediate the influence of acute stress on cue-reward learning. To address this possibility, we examined how a single stressful experience influenced learning in an appetitive pavlovian conditioning task. Male rats underwent an episode of restraint prior to the first conditioning session. This acute stress treatment augmented conditioned responding in subsequent sessions. Voltammetry recordings of mesolimbic dopamine levels demonstrated that acute stress selectively increased reward-evoked dopamine release in the ventral lateral striatum (VLS), but not in the ventral medial striatum. Antagonizing dopamine receptors in the VLS blocked the stress-induced enhancement of conditioned responding. Collectively, these findings illustrate that stress engages dopamine signaling in the VLS to facilitate appetitive learning.SIGNIFICANCE STATEMENT Acute stress influences learning about aversive and rewarding outcomes. Dopamine neurons are sensitive to stress and critical for reward learning. However, it is unclear whether stress regulates reward learning via dopamine signaling. Using fast-scan cyclic voltammetry as rats underwent pavlovian conditioning, we demonstrate that a single stressful experience increases reward-evoked dopamine release in the ventral lateral striatum. This enhanced dopamine signal accompanies a long-lasting increase in conditioned behavioral responding. These findings highlight that the ventral lateral striatum is a node for mediating the effect of stress on reward processing.

Pattern of dopamine signaling during aversive events predicts active avoidance learning

Learning to avoid aversive outcomes is an adaptive strategy to limit one's future exposure to stressful events. However, there is considerable variance in active avoidance learning across a population. The mesolimbic dopamine system contributes to behaviors elicited by aversive stimuli, although it is unclear if the heterogeneity in active avoidance learning is explained by differences in dopamine transmission. Furthermore, it is not known how dopamine signals evolve throughout active avoidance learning. To address these questions, we performed voltammetry recordings of dopamine release in the ventral medial striatum throughout training on inescapable footshock and signaled active avoidance tasks. This approach revealed differences in the pattern of dopamine signaling during the aversive cue and the safety period that corresponded to subsequent task performance. Dopamine transmission throughout the footshock bout did not predict performance but rather was modulated by the prior stress exposure. Additionally, we demonstrate that dopamine encodes a safety prediction error signal, which illustrates that ventral medial striatal dopamine release conveys a learning signal during both appetitive and aversive conditions.

Dopamine Signaling Is Critical for Supporting Cue-Driven Behavioral Control

Mesolimbic dopamine has been implicated in reward learning. Fischbach-Weiss and Janak (this issue) use optogenetics to attenuate dopamine signaling and study its role in cue-driven motivated behavior.

Synchronicity: The Role of Midbrain Dopamine in Whole-Brain Coordination

Midbrain dopamine seems to play an outsized role in motivated behavior and learning. Widely associated with mediating reward-related behavior, decision making, and learning, dopamine continues to generate controversies in the field. While many studies and theories focus on what dopamine cells encode, the question of how the midbrain derives the information it encodes is poorly understood and comparatively less addressed. Recent anatomical studies suggest greater diversity and complexity of afferent inputs than previously appreciated, requiring rethinking of prior models. Here, we elaborate a hypothesis that construes midbrain dopamine as implementing a Bayesian selector in which individual dopamine cells sample afferent activity across distributed brain substrates, comprising evidence to be evaluated on the extent to which stimuli in the on-going sensorimotor stream organizes distributed, parallel processing, reflecting implicit value. To effectively generate a temporally resolved phasic signal, a population of dopamine cells must exhibit synchronous activity. We argue that synchronous activity across a population of dopamine cells signals consensus across distributed afferent substrates, invigorating responding to recognized opportunities and facilitating further learning. In framing our hypothesis, we shift from the question of how value is computed to the broader question of how the brain achieves coordination across distributed, parallel processing. We posit the midbrain is part of an “axis of agency” in which the prefrontal cortex (PFC), basal ganglia (BGS), and midbrain form an axis mediating control, coordination, and consensus, respectively.

Blockade of muscarinic acetylcholine receptors facilitates motivated behaviour and rescues a model of antipsychotic-induced amotivation

Disruptions to motivated behaviour are a highly prevalent and severe symptom in a number of neuropsychiatric and neurodegenerative disorders. Current treatment options for these disorders have little or no effect upon motivational impairments. We assessed the contribution of muscarinic acetylcholine receptors to motivated behaviour in mice, as a novel pharmacological target for motivational impairments. Touchscreen progressive ratio (PR) performance was facilitated by the nonselective muscarinic receptor antagonist scopolamine as well as the more subtype-selective antagonists biperiden (M1) and tropicamide (M4). However, scopolamine and tropicamide also produced increases in non-specific activity levels, whereas biperiden did not. A series of control tests suggests the effects of the mAChR antagonists were sensitive to changes in reward value and not driven by changes in satiety, motor fatigue, appetite or perseveration. Subsequently, a sub-effective dose of biperiden was able to facilitate the effects of amphetamine upon PR performance, suggesting an ability to enhance dopaminergic function. Both biperiden and scopolamine were also able to reverse a haloperidol-induced deficit in PR performance, however only biperiden was able to rescue the deficit in effort-related choice (ERC) performance. Taken together, these data suggest that the M1 mAChR may be a novel target for the pharmacological enhancement of effort exertion and consequent rescue of motivational impairments. Conversely, M4 receptors may inadvertently modulate effort exertion through regulation of general locomotor activity levels.

Coordination of rapid cholinergic and dopaminergic signaling in striatum during spontaneous movement

Interplay between dopaminergic and cholinergic neuromodulation in the striatum is crucial for movement control, with prominent models proposing pro-kinetic and anti-kinetic effects of dopamine and acetylcholine release, respectively. However, the natural, movement-related signals of striatum cholinergic neurons and their relationship to simultaneous variations in dopamine signaling are unknown. Here, functional optical recordings in mice were used to establish rapid cholinergic signals in dorsal striatum during spontaneous movements. Bursts across the cholinergic population occurred at transitions between movement states and were marked by widespread network synchronization which diminished during sustained locomotion. Simultaneous cholinergic and dopaminergic recordings revealed distinct but coordinated sub-second signals, suggesting a new model where cholinergic population synchrony signals rapid changes in movement states while dopamine signals the drive to enact or sustain those states.

Sign Tracking and Goal Tracking Are Characterized by Distinct Patterns of Nucleus Accumbens Activity

During Pavlovian conditioning, if a cue (e.g., lever extension) predicts reward delivery in a different location (e.g., a food magazine), some individuals will come to approach and interact with the cue, a behavior known as sign tracking (ST), and others will approach the site of reward, a behavior known as goal tracking (GT). In rats, the acquisition of ST versus GT behavior is associated with distinct profiles of dopamine release in the nucleus accumbens (NAc), but it is unknown whether it is associated with different patterns of accumbens neural activity. Therefore, we recorded from individual neurons in the NAc core during the acquisition, maintenance, and extinction of ST and GT behavior. Even though NAc dopamine is specifically important for the acquisition and expression of ST, we found that cue-evoked excitatory responses encode the vigor of both ST and GT behavior. In contrast, among sign trackers only, there was a prominent decrease in reward-related activity over the course of training, which may reflect the decreasing reward prediction error encoded by phasic dopamine. Finally, both behavior and cue-evoked activity were relatively resistant to extinction in sign trackers, as compared with goal trackers, although a subset of neurons in both groups retained their cue-evoked responses. Overall, the results point to the convergence of multiple forms of reward learning in the NAc.

The Psychopharmacology of Effort-Related Decision Making: Dopamine, Adenosine, and Insights into the Neurochemistry of Motivation

Effort-based decision making is studied using tasks that offer choices between high-effort options leading to more highly valued reinforcers versus low-effort/low-reward options. These tasks have been used to study the involvement of neural systems, including mesolimbic dopamine and related circuits, in effort-related aspects of motivation. Moreover, such tasks are useful as animal models of some of the motivational symptoms that are seen in people with depression, schizophrenia, Parkinson's disease, and other disorders. The present review will discuss the pharmacology of effort-related decision making and will focus on the use of these tasks for the development of drug treatments for motivational dysfunction. Research has identified pharmacological conditions that can alter effort-based choice and serve as models for depression-related symptoms (e.g., the vesicular monoamine transport-2 inhibitor tetrabenazine and proinflammatory cytokines). Furthermore, tests of effort-based choice have identified compounds that are particularly useful for stimulating high-effort work output and reversing the deficits induced by tetrabenazine and cytokines. These studies indicate that drugs that act by facilitating dopamine transmission, as well as adenosine A_2A antagonists, are relatively effective at reversing effort-related impairments. Studies of effort-based choice may lead to the identification of drug targets that could be useful for treating motivational treatments that are resistant to commonly used antidepressants such as serotonin transport inhibitors.

Operant Costs Modulate Dopamine Release to Self-Administered Cocaine

The costs associated with obtaining illicit drugs can fluctuate depending upon the relative drug availability. As a consequence of the changing costs, the effort that one must exert to obtain drugs is dynamic. Considerable evidence illustrates a critical role for dopamine in the ventral medial striatum in mediating drug reinforcement. However, little is known regarding how dopamine release is affected by changes in the costs associated with earning drugs. We used fast-scan cyclic voltammetry to determine how changes in the operant requirement affected dopamine release to self-administered cocaine in male rats. Dopamine release to cocaine infusions increased across trials during self-administration sessions using a fixed-ratio reinforcement schedule with a low operant requirement. However, increasing the operant requirement abolished the within-session elevation in dopamine release to drug rewards. This effect was not due to underlying changes in preinfusion dopamine levels and was not explained by cocaine levels in the brain. This within-session increase in dopamine release to cocaine infusions reemerged when the operant requirement was lowered. Under a progressive ratio reinforcement schedule, there was no increase in dopamine release to drug rewards across trials, which contrasts with prior studies demonstrating an increase in dopamine release to food rewards. Collectively, these findings illustrate that the influence of operant costs on reward-evoked dopamine release depends upon type of reward that can be earned (e.g., food or drug).SIGNIFICANCE STATEMENT The mesolimbic dopamine system is involved with mediating drug reinforcement. Although the costs associated with earning drugs are dynamic, no studies to date have examined how dopamine release to drug rewards is affected by changing costs. By performing fast-scan cyclic voltammetry recordings in rats self-administering cocaine, the present work demonstrates that changing the operant costs reversibly modulates the dopamine response to cocaine rewards. Furthermore, these findings highlight that the influence of costs on dopamine release to drug rewards differs from the established effect of costs on dopamine release to food rewards.

Inhibition of endocannabinoid degradation rectifies motivational and dopaminergic deficits in the Q175 mouse model of Huntington's disease

Prominent motor deficits (e.g., chorea) that typify Huntington's disease (HD) arise following a prolonged prodromal stage characterized by psychiatric disturbances. Apathy, a disorder of motivation characterized by diminished goal-directed behavior, is one of the earliest and most common psychiatric symptoms in HD, but the underlying neurobiology is unclear and treatment options are limited. Alterations in the endocannabinoid (eCB) and dopamine systems represent prominent pathophysiological markers in HD that-similar to motivational deficits-present early and decline across disease progression. Whether changes in dopamine and eCB systems are associated with specific behavioral impairments in HD and whether these deficits are amenable to viable treatments is unknown. Here, we show that dopaminergic encoding of effortful drive progressively declines with age in an HD mouse model, and is restored by elevating tissue levels of the eCB 2-arachidonoylglycerol (2-AG) through targeted inhibition of its enzymatic degradation. This work supports aberrant dopaminergic encoding of reward as a neurobiological correlate of apathy in HD, and indicates that cannabinoid receptor-based therapies may benefit neuropsychiatric care for HD.

Heterogeneous dopamine signals support distinct features of motivated actions: implications for learning and addiction

Despite decades of research, investigations into effective neural and pharmacological therapies for many drugs of abuse, such as cocaine, have produced no FDA-approved approaches. This difficulty derives from the complexity of substance use disorders, which encompass a variety of behavioral, psychological, and neural circuit-based changes that occur as a result of repeated experience with the drug. Dopamine signaling has been demonstrated to play a key role in several aspects of drug abuse—from mediating its reinforcing properties and drug-seeking to triggering relapse—while also mediating a number of important aspects of normal (nondrug related) motivated behaviors and actions. Real-time recording methods such as in vivo voltammetry, electrophysiology, and calcium imaging demonstrate that the signaling properties of dopamine for motivationally relevant stimuli are highly dynamic and spatiotemporally circumscribed within afferent target regions. In this review, we identify the origins and functional consequences of heterogeneous dopamine release in the limbic system, and how these properties are persistently altered in the drug-experienced brain. We propose that these spatiotemporally parallel dopaminergic signals are simultaneously available to the animal, but that these circuits are impaired following prolonged drug experience by disrupting the location and content of dopamine signals in afferent target regions. These findings are discussed in the context of relapse and pathways to discovering new treatments for addiction disorders.

Mice learn to avoid regret

Regret can be defined as the subjective experience of recognizing that one has made a mistake and that a better alternative could have been selected. The experience of regret is thought to carry negative utility. This typically takes two distinct forms: augmenting immediate postregret valuations to make up for losses, and augmenting long-term changes in decision-making strategies to avoid future instances of regret altogether. While the short-term changes in valuation have been studied in human psychology, economics, neuroscience, and even recently in nonhuman-primate and rodent neurophysiology, the latter long-term process has received far less attention, with no reports of regret avoidance in nonhuman decision-making paradigms. We trained 31 mice in a novel variant of the Restaurant Row economic decision-making task, in which mice make decisions of whether to spend time from a limited budget to achieve food rewards of varying costs (delays). Importantly, we tested mice longitudinally for 70 consecutive days, during which the task provided their only source of food. Thus, decision strategies were interdependent across both trials and days. We separated principal commitment decisions from secondary reevaluation decisions across space and time and found evidence for regret-like behaviors following change-of-mind decisions that corrected prior economically disadvantageous choices. Immediately following change-of-mind events, subsequent decisions appeared to make up for lost effort by altering willingness to wait, decision speed, and pellet consumption speed, consistent with past reports of regret in rodents. As mice were exposed to an increasingly reward-scarce environment, we found they adapted and refined distinct economic decision-making strategies over the course of weeks to maximize reinforcement rate. However, we also found that even without changes in reinforcement rate, mice transitioned from an early strategy rooted in foraging to a strategy rooted in deliberation and planning that prevented future regret-inducing change-of-mind episodes from occurring. These data suggest that mice are learning to avoid future regret, independent of and separate from reinforcement rate maximization.

Regret describes a unique postdecision phenomenon in which losses are realized as a fault of one’s own actions. Regret is often hypothesized to have an inherent negative utility, and humans will often incur costs so as to avoid the risk of future regret. However, current models of nonhuman decision-making are based on reward maximization hypotheses. We recently found that rats express regret behaviorally and neurophysiologically on neuroeconomic foraging tasks; however, it remains unknown whether nonhuman animals will change strategies so as to avoid regret, even in the absence of changes in the achieved rate of reinforcement. Here, we provide the first evidence that mice change strategies to avoid future regret, independent of and separate from reinforcement rate maximization. Our data suggest mice accomplish this by shifting from a foraging decision-making strategy that produces change-of-mind decisions after investment mistakes to one rooted in deliberation that learns to plan ahead.

Dopamine, Effort-Based Choice, and Behavioral Economics: Basic and Translational Research

Operant behavior is not only regulated by factors related to the quality or quantity of reinforcement, but also by the work requirements inherent in performing instrumental actions. Moreover, organisms often make effort-related decisions involving economic choices such as cost/benefit analyses. Effort-based decision making is studied using behavioral procedures that offer choices between high-effort options leading to relatively preferred reinforcers vs. low effort/low reward choices. Several neural systems, including the mesolimbic dopamine (DA) system and other brain circuits, are involved in regulating effort-related aspects of motivation. Considerable evidence indicates that mesolimbic DA transmission exerts a bi-directional control over exertion of effort on instrumental behavior tasks. Interference with DA transmission produces a low-effort bias in animals tested on effort-based choice tasks, while increasing DA transmission with drugs such as DA transport blockers tends to enhance selection of high-effort options. The results from these pharmacology studies are corroborated by the findings from recent articles using optogenetic, chemogenetic and physiological techniques. In addition to providing important information about the neural regulation of motivated behavior, effort-based choice tasks are useful for developing animal models of some of the motivational symptoms that are seen in people with various psychiatric and neurological disorders (e.g., depression, schizophrenia, Parkinson’s disease). Studies of effort-based decision making may ultimately contribute to the development of novel drug treatments for motivational dysfunction.

Dopamine and Proximity in Motivation and Cognitive Control

Cognitive control - the ability to override a salient or prepotent action to execute a more deliberate one - is required for flexible, goal-directed behavior, and yet it is subjectively costly: decision-makers avoid allocating control resources, even when doing so affords more valuable outcomes. Dopamine likely offsets effort costs just as it does for physical effort. And yet, dopamine can also promote impulsive action, undermining control. We propose a novel hypothesis that reconciles opposing effects of dopamine on cognitive control: during action selection, striatal dopamine biases benefits relative to costs, but does so preferentially for "proximal" motor and cognitive actions. Considering the nature of instrumental affordances and their dynamics during action selection facilitates a parsimonious interpretation and conserved corticostriatal mechanisms across physical and cognitive domains.

Enhancing excitability of dopamine neurons promotes motivational behaviour through increased action initiation

Motivational deficits are a key symptom in multiple psychiatric disorders, including major depressive disorder, schizophrenia and addiction. A likely neural substrate for these motivational deficits is the brain dopamine (DA) system. In particular, DA signalling in the nucleus accumbens, which originates from DA neurons in the ventral tegmental area (VTA), has been identified as a crucial substrate for effort-related and activational aspects of motivation. Unravelling how VTA DA neuronal activity relates to motivational behaviours is required to understand how motivational deficits in psychiatry can be specifically targeted. In this study, we therefore used designer receptors exclusively activated by designer drugs (DREADD) in TH:Cre rats, in order to determine the effects of chemogenetic DA neuron activation on different aspects of motivational behaviour. We found that chemogenetic activation of DA neurons in the VTA, but not substantia nigra, significantly increased responding for sucrose under a progressive ratio schedule of reinforcement. More specifically, high effort exertion was characterized by increased initiations of reward-seeking actions. This effect was dependent on effort requirements and instrumental contingencies, but was not affected by sucrose pre-feeding. Together, these findings indicate that VTA DA neuronal activation drives motivational behaviour by facilitating action initiation. With this study, we show that enhancing excitability of VTA DA neurons is a viable strategy to improve motivational behaviour.

A transient dopamine signal encodes subjective value and causally influences demand in an economic context

The mesolimbic dopamine system is strongly implicated in motivational processes. Currently accepted theories suggest that transient mesolimbic dopamine release events energize reward seeking and encode reward value. During the pursuit of reward, critical associations are formed between the reward and cues that predict its availability. Conditioned by these experiences, dopamine neurons begin to fire upon the earliest presentation of a cue, and again at the receipt of reward. The resulting dopamine concentration scales proportionally to the value of the reward. In this study, we used a behavioral economics approach to quantify how transient dopamine release events scale with price and causally alter price sensitivity. We presented sucrose to rats across a range of prices and modeled the resulting demand curves to estimate price sensitivity. Using fast-scan cyclic voltammetry, we determined that the concentration of accumbal dopamine time-locked to cue presentation decreased with price. These data confirm and extend the notion that dopamine release events originating in the ventral tegmental area encode subjective value. Using optogenetics to augment dopamine concentration, we found that enhancing dopamine release at cue made demand more sensitive to price and decreased dopamine concentration at reward delivery. From these observations, we infer that value is decreased because of a negative reward prediction error (i.e., the animal receives less than expected). Conversely, enhancing dopamine at reward made demand less sensitive to price. We attribute this finding to a positive reward prediction error, whereby the animal perceives they received a better value than anticipated.

Inhibiting Mesolimbic Dopamine Neurons Reduces the Initiation and Maintenance of Instrumental Responding

Mesolimbic dopamine perturbations modulate performance of reward-seeking behavior, with tasks requiring high effort being especially vulnerable to disruption of dopamine signaling. Previous work primarily investigated long-term perturbations such as receptor antagonism and dopamine depletion, which constrain the ability to assess dopamine contributions to effort expenditure in isolation from other behavior events, such as reward consumption. Also unclear is if dopamine is required for both initiation and maintenance when a sequence of multiple instrumental responses is required. Here we used optogenetic inhibition of midbrain TH+  neurons to probe the role of dopamine neuron activity during instrumental responding for reward by varying the time epoch of neural inhibition relative to the time of response initiation. Within a fixed-ratio procedure, requiring eight nosepoke responses per reinforcer delivery, or a progressive ratio (PR) procedure, in which within-session response requirements increased exponentially, inhibiting dopamine neurons while mice were engaged in response bouts decreased the probability of continued responding. If inhibition occurred during each attempted bout, the effect was to decrease total responses, and thus amount of rewards earned, over a session. In contrast, if inhibition was applied only during some bouts, mice increased the number of bouts initiated to earn control levels of reward. Inhibiting dopamine neurons while mice were not responding decreased the probability of initiating an instrumental response but had no effect on the amount of effort exerted over the entire session. We conclude that midbrain dopamine signaling promotes initiation of instrumental responding and maintains motivation to continue ongoing bouts of effortful responses.

Cross-talk between the epigenome and neural circuits in drug addiction

Drug addiction is a behavioral disorder characterized by dysregulated learning about drugs and associated cues that result in compulsive drug seeking and relapse. Learning about drug rewards and predictive cues is a complex process controlled by a computational network of neural connections interacting with transcriptional and molecular mechanisms within each cell to precisely guide behavior. The interplay between rapid, temporally specific neuronal activation, and longer-term changes in transcription is of critical importance in the expression of appropriate, or in the case of drug addiction, inappropriate behaviors. Thus, these factors and their interactions must be considered together, especially in the context of treatment. Understanding the complex interplay between epigenetic gene regulation and circuit connectivity will allow us to formulate novel therapies to normalize maladaptive reward behaviors, with a goal of modulating addictive behaviors, while leaving natural reward-associated behavior unaffected.

Hypocretin receptor 1 blockade produces bimodal modulation of cocaine-associated mesolimbic dopamine signaling

RATIONALE

Cocaine addiction is a chronic psychiatric disorder characterized by pathological motivation to obtain cocaine and behavioral and neurochemical hypersensitivity to cocaine-associated cues. These features of cocaine addiction are thought to be driven by aberrant phasic dopamine signaling. We previously demonstrated that blockade of the hypocretin receptor 1 (HCRTr1) attenuates cocaine self-administration and reduces cocaine-induced enhancement of dopamine signaling. Despite this evidence, the effects of HCRTr1 blockade on endogenous phasic dopamine release are unknown.

OBJECTIVE

In the current studies, we assessed whether blockade of HCRTr1 alters spontaneous and cue-evoked dopamine release in the nucleus accumbens core of freely moving rats.

METHODS

We first validated the behavioral and neurochemical effects of the novel, highly selective, HCRTr1 antagonist RTIOX-276 using cocaine self-administration and fast-scan cyclic voltammetry (FSCV) in anesthetized rats. We then used FSCV in freely moving rats to examine whether RTIOX-276 impacts spontaneous and cue-evoked dopamine release. Finally, we used ex vivo slice FSCV to determine whether the effects of RTIOX-276 on dopamine signaling involve dopamine terminal adaptations.

RESULTS

Doses of RTIOX-276 that attenuate the motivation for cocaine reduce spontaneous dopamine transient amplitude and cue-evoked dopamine release. Further, these doses attenuated cocaine-induced dopamine uptake inhibition at the level of dopamine terminals.

CONCLUSION

Our results provide support for the standing hypothesis that HCRTr1 blockade suppresses endogenous phasic dopamine signals, likely via actions at dopamine cell bodies. These results also elucidate a second process through which HCRTr1 blockade attenuates the effects of cocaine by reducing cocaine sensitivity at dopamine terminals.

Chemogenetic Activation of Midbrain Dopamine Neurons Affects Attention, but not Impulsivity, in the Five-Choice Serial Reaction Time Task in Rats

Attentional impairments and exaggerated impulsivity are key features of psychiatric disorders, such as attention-deficit/hyperactivity disorder, schizophrenia, and addiction. These deficits in attentional performance and impulsive behaviors have been associated with aberrant dopamine (DA) signaling, but it remains unknown whether these deficits result from enhanced DA neuronal activity in the midbrain. Here, we took a novel approach by testing the impact of chemogenetically activating DA neurons in the ventral tegmental area (VTA) or substantia nigra pars compacta (SNc) on attention and impulsivity in the five-choice serial reaction time task (5-CSRTT) in rats. We found that activation of DA neurons in both the VTA and SNc impaired attention by increasing trial omissions. In addition, SNc DA neuron activation decreased attentional accuracy. Surprisingly, enhanced DA neuron activity did not affect impulsive action in this task. These results show that enhanced midbrain DA neuronal activity induces deficits in attentional performance, but not impulsivity. Furthermore, DA neurons in the VTA and SNc have different roles in regulating attention. These findings contribute to our understanding of the neural substrates underlying attention deficits and impulsivity, and provide valuable insights to improve treatment of these symptoms.

Brain and Emotion

Progress in understanding the relation between brain profiles and emotions is being slowed by the belief in a collection of basic emotional states, with the names: fear, anger, joy, disgust, and sadness, that do not specify the species or age of the experiencing agent, the origin of the state, or the evidence used to infer it. This article evaluates critically the premise that decontextualized emotional words refer to natural kinds. It also suggests that investigators set aside the currently popular words and search for relations, in humans and animals, between patterns of measures to varied incentives presented in distinctive contexts.

Reward Dependent Invigoration Relates to Theta Oscillations and Is Predicted by Dopaminergic Midbrain Integrity in Healthy Elderly

Motivation can have invigorating effects on behavior via dopaminergic neuromodulation. While this relationship has mainly been established in theoretical models and studies in younger subjects, the impact of structural declines of the dopaminergic system during healthy aging remains unclear. To investigate this issue, we used electroencephalography (EEG) in healthy young and elderly humans in a reward-learning paradigm. Specifically, scene images were initially encoded by combining them with cues predicting monetary reward (high vs. low reward). Subsequently, recognition memory for the scenes was tested. As a main finding, we can show that response times (RTs) during encoding were faster for high reward predicting images in the young but not elderly participants. This pattern was resembled in power changes in the theta-band (4–7 Hz). Importantly, analyses of structural MRI data revealed that individual reward-related differences in the elderlies’ response time could be predicted by the structural integrity of the dopaminergic substantia nigra (SN; as measured by magnetization transfer (MT)). These findings suggest a close relationship between reward-based invigoration, theta oscillations and age-dependent changes of the dopaminergic system.

Forgetting in Reinforcement Learning Links Sustained Dopamine Signals to Motivation

It has been suggested that dopamine (DA) represents reward-prediction-error (RPE) defined in reinforcement learning and therefore DA responds to unpredicted but not predicted reward. However, recent studies have found DA response sustained towards predictable reward in tasks involving self-paced behavior, and suggested that this response represents a motivational signal. We have previously shown that RPE can sustain if there is decay/forgetting of learned-values, which can be implemented as decay of synaptic strengths storing learned-values. This account, however, did not explain the suggested link between tonic/sustained DA and motivation. In the present work, we explored the motivational effects of the value-decay in self-paced approach behavior, modeled as a series of ‘Go’ or ‘No-Go’ selections towards a goal. Through simulations, we found that the value-decay can enhance motivation, specifically, facilitate fast goal-reaching, albeit counterintuitively. Mathematical analyses revealed that underlying potential mechanisms are twofold: (1) decay-induced sustained RPE creates a gradient of ‘Go’ values towards a goal, and (2) value-contrasts between ‘Go’ and ‘No-Go’ are generated because while chosen values are continually updated, unchosen values simply decay. Our model provides potential explanations for the key experimental findings that suggest DA's roles in motivation: (i) slowdown of behavior by post-training blockade of DA signaling, (ii) observations that DA blockade severely impairs effortful actions to obtain rewards while largely sparing seeking of easily obtainable rewards, and (iii) relationships between the reward amount, the level of motivation reflected in the speed of behavior, and the average level of DA. These results indicate that reinforcement learning with value-decay, or forgetting, provides a parsimonious mechanistic account for the DA's roles in value-learning and motivation. Our results also suggest that when biological systems for value-learning are active even though learning has apparently converged, the systems might be in a state of dynamic equilibrium, where learning and forgetting are balanced.

Dopamine (DA) has been suggested to have two reward-related roles: (1) representing reward-prediction-error (RPE), and (2) providing motivational drive. Role(1) is based on the physiological results that DA responds to unpredicted but not predicted reward, whereas role(2) is supported by the pharmacological results that blockade of DA signaling causes motivational impairments such as slowdown of self-paced behavior. So far, these two roles are considered to be played by two different temporal patterns of DA signals: role(1) by phasic signals and role(2) by tonic/sustained signals. However, recent studies have found sustained DA signals with features indicative of both roles (1) and (2), complicating this picture. Meanwhile, whereas synaptic/circuit mechanisms for role(1), i.e., how RPE is calculated in the upstream of DA neurons and how RPE-dependent update of learned-values occurs through DA-dependent synaptic plasticity, have now become clarified, mechanisms for role(2) remain unclear. In this work, we modeled self-paced behavior by a series of ‘Go’ or ‘No-Go’ selections in the framework of reinforcement-learning assuming DA's role(1), and demonstrated that incorporation of decay/forgetting of learned-values, which is presumably implemented as decay of synaptic strengths storing learned-values, provides a potential unified mechanistic account for the DA's two roles, together with its various temporal patterns.

Optogenetically-induced tonic dopamine release from VTA-nucleus accumbens projections inhibits reward consummatory behaviors

Recent optogenetic studies demonstrated that phasic dopamine release in the nucleus accumbens may play a causal role in multiple aspects of natural and drug reward-related behaviors. The role of tonic dopamine release in reward consummatory behavior remains unclear. The current study used a combinatorial viral-mediated gene delivery approach to express ChR2 on mesolimbic dopamine neurons in rats. We used optical activation of this dopamine circuit to mimic tonic dopamine release in the nucleus accumbens and to explore the causal relationship between this form of dopamine signaling within the ventral tegmental area (VTA)-nucleus accumbens projection and consumption of a natural reward. Using a two bottle choice paradigm (sucrose vs. water), the experiments revealed that tonic optogenetic stimulation of mesolimbic dopamine transmission significantly decreased reward consummatory behaviors. Specifically, there was a significant decrease in the number of bouts, licks and amount of sucrose obtained during the drinking session. Notably, activation of VTA dopamine cell bodies or dopamine terminals in the nucleus accumbens resulted in identical behavioral consequences. No changes in water intake were evident under the same experimental conditions. Collectively, these data demonstrate that tonic optogenetic stimulation of VTA-nucleus accumbens dopamine release is sufficient to inhibit reward consummatory behavior, possibly by preventing this circuit from engaging in phasic activity that is thought to be essential for reward-based behaviors.

Inhibitory Input from the Lateral Hypothalamus to the Ventral Tegmental Area Disinhibits Dopamine Neurons and Promotes Behavioral Activation

Projections from the lateral hypothalamus (LH) to the ventral tegmental area (VTA), containing both GABAergic and glutamatergic components, encode conditioned responses and control compulsive reward-seeking behavior. GABAergic neurons in the LH have been shown to mediate appetitive and feeding-related behaviors. Here we show that the GABAergic component of the LH-VTA pathway supports positive reinforcement and place preference, while the glutamatergic component mediates place avoidance. In addition, our results indicate that photoactivation of these projections modulates other behaviors, such as social interaction and perseverant investigation of a novel object. We provide evidence that photostimulation of the GABAergic LH-VTA component, but not the glutamatergic component, increases dopamine (DA) release in the nucleus accumbens (NAc) via inhibition of local VTA GABAergic neurons. Our study clarifies how GABAergic LH inputs to the VTA can contribute to generalized behavioral activation across multiple contexts, consistent with a role in increasing motivational salience. VIDEO ABSTRACT.