A Neural Circuit Mechanism for the Involvements of Dopamine in Effort-Related Choices: Decay of Learned Values, Secondary Effects of Depletion, and Calculation of Temporal Difference Error

Mesolimbic dopamine ramps reflect environmental timescales

Mesolimbic dopamine activity occasionally exhibits ramping dynamics, reigniting debate on theories of dopamine signaling. This debate is ongoing partly because the experimental conditions under which dopamine ramps emerge remain poorly understood. Here, we show that during Pavlovian and instrumental conditioning, mesolimbic dopamine ramps are only observed when the inter-trial interval is short relative to the trial period. These results constrain theories of dopamine signaling and identify a critical variable determining the emergence of dopamine ramps.

Glutamate inputs send prediction error of reward, but not negative value of aversive stimuli, to dopamine neurons

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs), but the mechanisms underlying RPE computation, particularly the contributions of different neurotransmitters, remain poorly understood. Here, we used a genetically encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons in mice. We found that glutamate inputs exhibit virtually all of the characteristics of RPE rather than conveying a specific component of RPE computation, such as reward or expectation. Notably, whereas glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli into more positive responses, whereas excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.

Glutamate inputs send prediction error of reward but not negative value of aversive stimuli to dopamine neurons

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs) but the mechanisms underlying RPE computation, particularly contributions of different neurotransmitters, remain poorly understood. Here we used a genetically-encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons. We found that glutamate inputs exhibit virtually all of the characteristics of RPE, rather than conveying a specific component of RPE computation such as reward or expectation. Notably, while glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli toward more positive responses, while excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.

Phasic Dopamine Changes and Hebbian Mechanisms during Probabilistic Reversal Learning in Striatal Circuits: A Computational Study

Cognitive flexibility is essential to modify our behavior in a non-stationary environment and is often explored by reversal learning tasks. The basal ganglia (BG) dopaminergic system, under a top-down control of the pre-frontal cortex, is known to be involved in flexible action selection through reinforcement learning. However, how adaptive dopamine changes regulate this process and learning mechanisms for training the striatal synapses remain open questions. The current study uses a neurocomputational model of the BG, based on dopamine-dependent direct (Go) and indirect (NoGo) pathways, to investigate reinforcement learning in a probabilistic environment through a task that associates different stimuli to different actions. Here, we investigated: the efficacy of several versions of the Hebb rule, based on covariance between pre- and post-synaptic neurons, as well as the required control in phasic dopamine changes crucial to achieving a proper reversal learning. Furthermore, an original mechanism for modulating the phasic dopamine changes is proposed, assuming that the expected reward probability is coded by the activity of the winner Go neuron before a reward/punishment takes place. Simulations show that this original formulation for an automatic phasic dopamine control allows the achievement of a good flexible reversal even in difficult conditions. The current outcomes may contribute to understanding the mechanisms for active control of dopamine changes during flexible behavior. In perspective, it may be applied in neuropsychiatric or neurological disorders, such as Parkinson’s or schizophrenia, in which reinforcement learning is impaired.

Diminished positive affect and traumatic stress: A biobehavioral review and commentary on trauma affective neuroscience

Post-traumatic stress manifests in disturbed affect and emotion, including exaggerated severity and frequency of negative valence emotions, e.g., fear, anxiety, anger, shame, and guilt. However, another core feature of common post-trauma psychopathologies, i.e. post-traumatic stress disorder (PTSD) and major depression, is diminished positive affect, or reduced frequency and intensity of positive emotions and affective states such as happiness, joy, love, interest, and desire/capacity for interpersonal affiliation. There remains a stark imbalance in the degree to which the neuroscience of each affective domain has been probed and characterized in PTSD, with our knowledge of post-trauma diminished positive affect remaining comparatively underdeveloped. This remains a prominent barrier to realizing the clinical breakthroughs likely to be afforded by the increasing availability of neuroscience assessment and intervention tools. In this review and commentary, the author summarizes the modest extant neuroimaging literature that has probed diminished positive affect in PTSD using reward processing behavioral paradigms, first briefly reviewing and outlining the neurocircuitry implicated in reward and positive emotion and its interrelationship with negative emotion and negative valence circuitry. Specific research guidelines are then offered to best and most efficiently develop the knowledge base in this area in a way that is clinically translatable and will exert a positive impact on routine clinical care. The author concludes with the prediction that the development of an integrated, bivalent theoretical and predictive model of how trauma impacts affective neurocircuitry to promote post-trauma psychopathology will ultimately lead to breakthroughs in how trauma treatments are conceptualized mechanistically and developed pragmatically.