Dopamine error signal to actively cope with lack of expected reward

Natural phasic inhibition of dopamine neurons signals cognitive rigidity

When animals unexpectedly fail, their dopamine neurons undergo phasic inhibition that canonically drives extinction learning, a cognitive-flexibility mechanism for discarding outdated strategies. However, the existing evidence equates natural and artificial phasic inhibition, despite their spatiotemporal differences. Addressing this gap, we targeted a GABAA-receptor antagonist precisely to dopamine neurons, yielding three unexpected findings. First, this intervention blocked natural phasic inhibition selectively, leaving tonic activity unaffected. Second, blocking natural phasic inhibition accelerated extinction learning, opposite to canonical mechanisms. Third, our approach selectively benefitted perseverative mice, restoring rapid extinction without affecting new reward learning. Our findings reveal that extinction learning is rapid by default and slowed by natural phasic inhibition, challenging foundational learning theories, while delineating a synaptic mechanism and therapeutic target for cognitive rigidity.

Prediction error in dopamine neurons during associative learning

Dopamine neurons have long been thought to facilitate learning by broadcasting reward prediction error (RPE), a teaching signal used in machine learning, but more recent work has advanced alternative models of dopamine's computational role. Here, I revisit this critical issue and review new experimental evidences that tighten the link between dopamine activity and RPE. First, I introduce the recent observation of a gradual backward shift of dopamine activity that had eluded researchers for over a decade. I also discuss several other findings, such as dopamine ramping, that were initially interpreted to conflict but later found to be consistent with RPE. These findings improve our understanding of neural computation in dopamine neurons.

Sensorimotor-linked reward modulates smooth pursuit eye movements in monkeys

Reward is essential for shaping behavior. Using sensory cues to imply forthcoming rewards, previous studies have demonstrated powerful effects of rewards on behavior. Nevertheless, the impact of reward on the sensorimotor transformation, particularly when reward is linked to behavior remains uncertain. In this study, we investigated how reward modulates smooth pursuit eye movements in monkeys. Three distinct associations between reward and eye movements were conducted in independent blocks. Results indicated that reward increased eye velocity during the steady-state pursuit, rather than during the initiation. The influence depended on the particular association between behavior and reward: a faster eye velocity was linked with reward. Neither rewarding slower eye movements nor randomizing rewards had a significant effect on behavior. The findings support the existence of distinct mechanisms involved in the initiation and steady-state phases of pursuit, and contribute to a deeper understanding of how reward interacts with these two periods of pursuit.

Dopamine release in human associative striatum during reversal learning

The dopaminergic system is firmly implicated in reversal learning but human measurements of dopamine release as a correlate of reversal learning success are lacking. Dopamine release and hemodynamic brain activity in response to unexpected changes in action-outcome probabilities are here explored using simultaneous dynamic [11C]Raclopride PET-fMRI and computational modelling of behavior. When participants encounter reversed reward probabilities during a card guessing game, dopamine release is observed in associative striatum. Individual differences in absolute reward prediction error and sensitivity to errors are associated with peak dopamine receptor occupancy. The fMRI response to perseverance errors at the onset of a reversal spatially overlap with the site of dopamine release. Trial-by-trial fMRI correlates of absolute prediction errors show a response in striatum and association cortices, closely overlapping with the location of dopamine release, and separable from a valence signal in ventral striatum. The results converge to implicate striatal dopamine release in associative striatum as a central component of reversal learning, possibly signifying the need for increased cognitive control when new stimuli-responses should be learned.

Heat-Killed L. helveticus Enhances Positive Mood States: A Randomized, Double-Blind, Placebo-Controlled Study

When mood states are impaired, daily life is severely disrupted. To maintain a specific mood state, both positive and negative moods must be controlled; however, methods to maintain a positive mood have not been fully established. Previous studies have suggested that heat-killed L. helveticus MCC1848 has the potential to improve positive moods. This study aimed to test the efficacy of heat-killed L. helveticus MCC1848 in maintaining and improving a positive mood with PANAS, a questionnaire specifically designed to assess positive and negative mood, as the primary endpoint. Healthy Japanese nursing students (n = 46) were randomized to receive heat-killed L. helveticus MCC1848 (5 billion/day) or placebo powder for four weeks. Mood state was assessed before and two and four weeks after the intervention began; ingestion of heat-killed L. helveticus MCC1848 significantly improved PANAS 'Positive Affect' compared to the placebo. These results indicate that heat-killed L. helveticus MCC1848 is effective in enhancing positive mood.

Opponent learning with different representations in the cortico-basal ganglia pathways can develop obsession-compulsion cycle

Obsessive-compulsive disorder (OCD) has been suggested to be associated with impairment of model-based behavioral control. Meanwhile, recent work suggested shorter memory trace for negative than positive prediction errors (PEs) in OCD. We explored relations between these two suggestions through computational modeling. Based on the properties of cortico-basal ganglia pathways, we modeled human as an agent having a combination of successor representation (SR)-based system that enables model-based-like control and individual representation (IR)-based system that only hosts model-free control, with the two systems potentially learning from positive and negative PEs in different rates. We simulated the agent's behavior in the environmental model used in the recent work that describes potential development of obsession-compulsion cycle. We found that the dual-system agent could develop enhanced obsession-compulsion cycle, similarly to the agent having memory trace imbalance in the recent work, if the SR- and IR-based systems learned mainly from positive and negative PEs, respectively. We then simulated the behavior of such an opponent SR+IR agent in the two-stage decision task, in comparison with the agent having only SR-based control. Fitting of the agents' behavior by the model weighing model-based and model-free control developed in the original two-stage task study resulted in smaller weights of model-based control for the opponent SR+IR agent than for the SR-only agent. These results reconcile the previous suggestions about OCD, i.e., impaired model-based control and memory trace imbalance, raising a novel possibility that opponent learning in model(SR)-based and model-free controllers underlies obsession-compulsion. Our model cannot explain the behavior of OCD patients in punishment, rather than reward, contexts, but it could be resolved if opponent SR+IR learning operates also in the recently revealed non-canonical cortico-basal ganglia-dopamine circuit for threat/aversiveness, rather than reward, reinforcement learning, and the aversive SR + appetitive IR agent could actually develop obsession-compulsion if the environment is modeled differently.