Decreases in Cued Reward Seeking After Reward-Paired Inhibition of Mesolimbic Dopamine

Frontostriatal circuit dysfunction leads to cognitive inflexibility in neuroligin-3 R451C knockin mice

Cognitive and behavioral rigidity are observed in various psychiatric diseases, including in autism spectrum disorder (ASD). However, the underlying mechanism remains to be elucidated. In this study, we found that neuroligin-3 (NL3) R451C knockin mouse model of autism (KI mice) exhibited deficits in behavioral flexibility in choice selection tasks. Single-unit recording of medium spiny neuron (MSN) activity in the nucleus accumbens (NAc) revealed altered encoding of decision-related cue and impaired updating of choice anticipation in KI mice. Additionally, fiber photometry demonstrated significant disruption in dynamic mesolimbic dopamine (DA) signaling for reward prediction errors (RPEs), along with reduced activity in medial prefrontal cortex (mPFC) neurons projecting to the NAc in KI mice. Interestingly, NL3 re-expression in the mPFC, but not in the NAc, rescued the deficit of flexible behaviors and simultaneously restored NAc-MSN encoding, DA dynamics, and mPFC-NAc output in KI mice. Taken together, this study reveals the frontostriatal circuit dysfunction underlying cognitive inflexibility and establishes a critical role of the mPFC NL3 deficiency in this deficit in KI mice. Therefore, these findings provide new insights into the mechanisms of cognitive and behavioral inflexibility and potential intervention strategies.

Striatal dopamine integrates cost, benefit, and motivation

Striatal dopamine (DA) release has long been linked to reward processing, but it remains controversial whether DA release reflects costs or benefits and how these signals vary with motivation. Here, we measure DA release in the nucleus accumbens (NAc) and dorsolateral striatum (DLS) while independently varying costs and benefits and apply behavioral economic principles to determine a mouse's level of motivation. We reveal that DA release in both structures incorporates both reward magnitude and sunk cost. Surprisingly, motivation was inversely correlated with reward-evoked DA release. Furthermore, optogenetically evoked DA release was also heavily dependent on sunk cost. Our results reconcile previous disparate findings by demonstrating that striatal DA release simultaneously encodes cost, benefit, and motivation but in distinct manners over different timescales. Future work will be necessary to determine whether the reduction in phasic DA release in highly motivated animals is due to changes in tonic DA levels.

Aberrant neural computation of social controllability in nicotine-dependent humans

Social controllability, defined as the ability to exert influence when interacting with others, is crucial for optimal decision-making. Inability to do so might contribute to maladaptive behaviors such as drug use, which often takes place in social settings. Here, we examined nicotine-dependent humans using fMRI, as they made choices that could influence the proposals from simulated partners. Computational modeling revealed that smokers under-estimated the influence of their actions and self-reported a reduced sense of control, compared to non-smokers. These findings were replicated in a large independent sample of participants recruited online. Neurally, smokers showed reduced tracking of forward projected choice values in the ventromedial prefrontal cortex, and impaired computation of social prediction errors in the midbrain. These results demonstrate that smokers were less accurate in estimating their personal influence when the social environment calls for control, providing a neurocomputational account for the social cognitive deficits in this population.

Temporal Dynamics Underlying Prelimbic Prefrontal Cortical Regulation of Action Selection and Outcome Evaluation during Risk/Reward Decision-Making

Risk/reward decision-making is a dynamic process that includes periods of deliberation before action selection and evaluation of the action outcomes that bias subsequent choices. Inactivation of the prelimbic (PL) cortex has revealed its integral role in updating decision biases in the face of changes in probabilistic reward contingencies, yet how phasic PL signals during different phases of the decision process influence choice remains unclear. We used temporally specific optogenetic inhibition to selectively disrupt PL activity coinciding with action selection and outcome phases to examine how these signals influence choice. Male rats expressing the inhibitory opsin eArchT within PL excitatory neurons were well trained on a probabilistic discounting task, entailing choice between small/certain versus large/risky rewards, the probability of which varied over a session (50-12.5%). During testing, brief light pulses suppressed PL activity before choice or after different outcomes. Prechoice suppression reduced bias toward more preferred/higher utility options and disrupted how recent outcomes influenced subsequent choice. Inhibition during risky losses induced a similar profile, but here, the impact of reward omissions were either amplified or diminished, relative to the context of the estimated profitability of the risky option. Inhibition during large or small reward receipt reduced risky choice when this option was more profitable, suggesting these signals can both reinforce rewarded risky choices and also act as a relative value comparator signal that augments incentive for larger rewards. These findings reveal multifaceted contributions by the PL in implementing decisions and integrating action-outcome feedback to assign context to the decision space.SIGNIFICANCE STATEMENT The PL prefrontal cortex plays an integral role in guiding risk/reward decisions, but how activity in this region during different phases of the decision process influences choice is unclear. By using temporally specific optogenetic manipulations of this activity, the present study unveiled previously uncharacterized and differential contributions by PL in implementing decision policies and how evaluation of decision outcomes shape subsequent choice. These findings provide novel insight into the dynamic processes engaged by the PL that underlie action selection in situations involving reward uncertainty that may aid in understanding the mechanism underlying normal and aberrant decision-making processes.

Nucleus accumbens D1-receptors regulate and focus transitions to reward-seeking action

It is well established that dopamine transmission is integral in mediating the influence of reward expectations on reward-seeking actions. However, the precise causal role of dopamine transmission in moment-to-moment reward-motivated behavioral control remains contentious, particularly in contexts where it is necessary to refrain from responding to achieve a beneficial outcome. To examine this, we manipulated dopamine transmission pharmacologically as rats performed a Go/No-Go task that required them to either make or withhold action to gain either a small or large reward. D1R Stimulation potentiated cue-driven action initiation, including fast impulsive actions on No-Go trials. By contrast, D1R blockade primarily disrupted the successful completion of Go trial sequences. Surprisingly, while after global D1R blockade this was characterized by a general retardation of reward-seeking actions, nucleus accumbens core (NAcC) D1R blockade had no effect on the speed of action initiation or impulsive actions. Instead, fine-grained analyses showed that this manipulation decreased the precision of animals' goal-directed actions, even though they usually still followed the appropriate response sequence. Strikingly, such "unfocused" responding could also be observed off-drug, particularly when only a small reward was on offer. These findings suggest that the balance of activity at NAcC D1Rs plays a key role in enabling the rapid activation of a focused, reward-seeking state to enable animals to efficiently and accurately achieve their goal.

Deconstructing and reconstructing behaviour relevant to mental health disorders: The benefits of a psychological approach, with a focus on addiction

RUTHERFORD, L.G. and Milton, A.L. Deconstructing and reconstructing behaviour relevant to mental health disorders: what can psychology offer? NEUROSCI BIOBEHAV REV XX(X)XXX-XXX, 2021. - Current treatments for mental health disorders are successful only for some patients, and there is an unmet clinical need for new treatment development. One challenge for treatment development has been how best to model complex human conditions in animals, where mechanism can be more readily studied with a range of neuroscientific techniques. We suggest that an approach to modelling based on associative animal learning theory provides a good framework for deconstructing complex mental health disorders such that they can be studied in animals. These individual simple models can subsequently be used in combination to 'reconstruct' a more complex model of the mental health disorder of interest. Using examples primarily from the field of drug addiction, we explore the 'psychological approach' and suggest that in addition to facilitating translation and backtranslation of tasks between animal models and patients, it is also highly concordant with the concept of triangulation.

Throwing open the doors of perception: The role of dopamine in visual processing

Animals form associations between visual cues and behaviours. Although dopamine is known to be critical in many areas of the brain to bind sensory information with appropriate responses, dopamine's role in the visual system is less well understood. Visual signals, which indicate the likely occurrence of a rewarding or aversive stimulus or indicate the context within which such stimuli may arrive, modulate activity in the superior colliculus and alter behaviour. However, such signals primarily originate in cortical and basal ganglia circuits, and evidence of direct signalling from midbrain dopamine neurons to superior colliculus is lacking. Instead, hypothalamic A13 dopamine neurons innervate the superior colliculus, and dopamine receptors are differentially expressed in the superior colliculus, with D1 receptors in superficial layers and D2 receptors in deep layers. However, it remains unknown if A13 dopamine neurons control behaviours through their effect on afferents within the superior colliculus. We propose that A13 dopamine neurons may play a critical role in processing information in the superior colliculus, modifying behavioural responses to visual cues, and propose some testable hypotheses regarding dopamine's effect on visual perception.

Dopamine signals as temporal difference errors: recent advances

In the brain, dopamine is thought to drive reward-based learning by signaling temporal difference reward prediction errors (TD errors), a 'teaching signal' used to train computers. Recent studies using optogenetic manipulations have provided multiple pieces of evidence supporting that phasic dopamine signals function as TD errors. Furthermore, novel experimental results have indicated that when the current state of the environment is uncertain, dopamine neurons compute TD errors using 'belief states' or a probability distribution over potential states. It remains unclear how belief states are computed but emerging evidence suggests involvement of the prefrontal cortex and the hippocampus. These results refine our understanding of the role of dopamine in learning and the algorithms by which dopamine functions in the brain.

Cocaine experience abolishes the motivation suppressing effect of CRF in the ventral midbrain

Stress affects dopamine-dependent behaviors in part through the actions of corticotropin releasing factor (CRF) in the ventral tegmental area (VTA). For example, acute stress engages CRF signaling in the VTA to suppress the motivation to work for food rewards. In contrast, acute stress promotes drug-seeking behavior through the actions of CRF in the VTA. These diverging behavioral effects in food- and drug-based tasks could indicate that CRF modulates goal-directed actions in a reinforcer-specific manner. Alternatively, prior drug experience could functionally alter how CRF in the VTA regulates dopamine-dependent behavior. To address these possibilities, we examined how intra-VTA injections of CRF influenced cocaine intake and whether prior drug experience alters how CRF modulates the motivation for food rewards. Our results demonstrate that intra-VTA injections of CRF had no effect on drug intake when self-administering cocaine under a progressive ratio reinforcement schedule. We also found that a prior history of either contingent or noncontingent cocaine infusions abolished the capacity for CRF to reduce the motivation for food rewards. Furthermore, voltammetry recordings in the nucleus accumbens illustrate that CRF in the VTA had no effect on cocaine-evoked dopamine release. These results collectively illustrate that exposure to abused substances functionally alters how neuropeptides act within the VTA to influence motivated behavior.

Learning from Action: Reconsidering Movement Signaling in Midbrain Dopamine Neuron Activity

Animals infer when and where a reward is available from experience with informative sensory stimuli and their own actions. In vertebrates, this is thought to depend upon the release of dopamine from midbrain dopaminergic neurons. Studies of the role of dopamine have focused almost exclusively on their encoding of informative sensory stimuli; however, many dopaminergic neurons are active just prior to movement initiation, even in the absence of sensory stimuli. How should current frameworks for understanding the role of dopamine incorporate these observations? To address this question, we review recent anatomical and functional evidence for action-related dopamine signaling. We conclude by proposing a framework in which dopaminergic neurons encode subjective signals of action initiation to solve an internal credit assignment problem.

Temporally restricted dopaminergic control of reward-conditioned movements

Midbrain dopamine (DA) neurons encode both reward- and movement-related events and are implicated in disorders of reward processing as well as movement. Consequently, disentangling the contribution of DA neurons in reinforcing versus generating movements is challenging and has led to lasting controversy. In this study, we dissociated these functions by parametrically varying the timing of optogenetic manipulations in a Pavlovian conditioning task and examining the influence on anticipatory licking before reward delivery. Inhibiting both ventral tegmental area and substantia nigra pars compacta DA neurons in the post-reward period had a significantly greater behavioral effect than inhibition in the pre-reward period of the task. Furthermore, the contribution of DA neurons to behavior decreased linearly as a function of elapsed time after reward. Together, the results indicate a temporally restricted role of DA neurons primarily related to reinforcing stimulus-reward associations and suggest that directly generating movements is a comparatively less important function.

Opposite Consequences of Tonic and Phasic Increases in Accumbal Dopamine on Alcohol-Seeking Behavior

Despite many years of work on dopaminergic mechanisms of alcohol addiction, much of the evidence remains mostly correlative in nature. Fortunately, recent technological advances have provided the opportunity to explore the causal role of alterations in neurotransmission within circuits involved in addictive behaviors. Here, we address this critical gap in our knowledge by integrating an optogenetic approach and an operant alcohol self-administration paradigm to assess directly how accumbal dopamine (DA) release dynamics influences the appetitive (seeking) component of alcohol-drinking behavior. We show that appetitive reward-seeking behavior in rats trained to self-administer alcohol can be shaped causally by ventral tegmental area-nucleus accumbens (VTA-NAc) DA neurotransmission. Our findings reveal that phasic patterns of DA release within this circuit enhance a discrete measure of alcohol seeking, whereas tonic patterns of stimulation inhibit this behavior. Moreover, we provide mechanistic evidence that tonic-phasic interplay within the VTA-NAc DA circuit underlies these seemingly paradoxical effects.

•

VTA-NAc DA transmission can bidirectionally modulate motivated behavior

•

Optogenetic increases in phasic DA release in the NAc enhance alcohol seeking

•

Optogenetic increases in tonic DA release in the NAc inhibit alcohol seeking

•

Phasic DA release can be decreased by the concurrent tonic activation

Nucleus accumbens shell dopamine mediates outcome value, but not predicted value, in a magnitude decision-making task

Effective decision-making depends on an animal's ability to predict and select the outcome of greatest value, and the nucleus accumbens (NAc) and its dopaminergic input play a key role in this process. We previously reported that rapid dopamine release in the NAc shell preferentially tracks the "preferred" (i.e., large reward) option during cues that predict the ability to respond for rewards of different sizes, as well as during reward delivery itself. The present study assessed whether shell dopamine release at these discrete times selectively mediated choice behavior for rewards of different magnitudes using optogenetics. Here, using Long Evans TH:Cre^± rats we employed selective optogenetic stimulation of dopamine terminals in the NAc shell during either reward-predictive cues (experiment 1) or reward delivery (experiment 2) in a magnitude-based decision-making task. We found that in TH:Cre^± rats, but not littermate controls, optical stimulation during low-magnitude reward delivery during forced choice trials was sufficient to bias preference for this option when given a choice. In contrast, optical stimulation of shell dopamine terminals during low-magnitude reward-predictive cues in forced choice trials did not shift free choice behavior in TH:Cre^± rats or controls. The findings indicate that preferential dopamine signaling in the NAc shell during reward outcome (delivery), but not reward-predictive cues are sufficient to influence choice behavior in our task supporting a causal role of dopamine in the NAc shell in reward outcome value, but not value-based predictive strategies.

Dopamine Signaling Is Critical for Supporting Cue-Driven Behavioral Control

Mesolimbic dopamine has been implicated in reward learning. Fischbach-Weiss and Janak (this issue) use optogenetics to attenuate dopamine signaling and study its role in cue-driven motivated behavior.