A neural substrate of prediction and reward

Brain Activation Associated With Response to Psychotherapies for Late-Life Depression: A Task-Based fMRI Study

BACKGROUND

The course of late-life depression is associated with functioning of multiple brain networks. Understanding the brain mechanisms associated with response to psychotherapy can inform treatment development and a personalized treatment approach. This study examined how activation of key regions of the salience network, default mode network and reward systems is associated with response to psychotherapies for late-life depression.

METHODS

Thirty-three older adults with major depressive disorder were randomized to 9 weeks of Engage or Problem-Solving Therapy for late-life depression. Participants completed a Probabilistic Reversal Learning task in the MRI at baseline and Week 6. We focused on focal activation in regions of interest selected a priori: the subgenual cingulate cortex (sgACC; DMN); the dorsal anterior cingulate cortex (dACC; salience network and reward system); and the nucleus accumbens (NAcc; reward system). We applied mixed-effects regression models to examine whether brain activation was associated with psychotherapy response.

RESULTS

We found that at baseline, low activation of the dACC and the sgACC was associated with lower depression severity over 6 weeks of psychotherapy. In addition, we observed significant time*activation interactions, such that after 6 weeks of psychotherapy, lower dACC activation and higher NAcc and sgACC activation were each associated with lower depression severity. Further, we found that baseline slower response to negative feedback and faster response to positive feedback was associated with lower depression severity over 6 weeks of psychotherapy.

CONCLUSIONS

Our findings suggest that activation of reward, salience, and DMN regions may serve as markers of response during psychotherapy for late-life depression. Engagement of these networks may be linked to treatment outcome. Personalized psychotherapies can target individuals' brain profiles to improve outcomes for older adults with major depression.

ARTICLE SUMMARY

This study examined whether activation of regions of the reward, salience and default mode networks is associated with response to psychotherapies for late-life depression. We found that baseline low activation of the dACC and the sgACC was associated with lower depression severity during psychotherapy. We also found that at week 6, lower dACC activation and higher NAcc and sgACC activation were linked with lower depression severity. These regions may represent promising brain mechanisms for future personalized interventions.

The effect of reward learning on inhibitory control in internet gaming disorder: Evidence from behavioral and ERP

Reward dysregulation and deficits in inhibitory control significantly contribute to the development of internet gaming disorder (IGD). While prior research demonstrates that reward history influences individuals' inhibitory control, it remains unclear whether this effect extends to individuals with IGD. The primary aim of this study was to investigate whether individuals with IGD exhibit impairments in reward learning and whether prior reward learning influences their inhibitory control, using both behavioral and event-related potential (ERP) measures. This study first employed a probability selection task to examine potential impairments in reward learning among individuals with IGD. Next, a stop-signal task incorporating reward- and punishment-associated stimuli was used to further investigate the behavioral and electroencephalographic effects of prior reward learning on subsequent inhibitory control. Results revealed that during the reward-learning phase, the IGD group exhibited significantly longer response times than the control group in both the learning and transfer phases. Additionally, the feedback-related negativity amplitude in the IGD group was significantly lower than that in the control group. Conversely, the P3 wave amplitude induced by positive and negative feedback in the IGD group were significantly higher than in the control group. In the inhibitory control phase following reward learning, the Nogo-P3 wave amplitude in response to reward cues was significantly greater in the IGD group than in the control group. Moreover, within the IGD group, the Nogo-P3 wave amplitude evoked by reward cues was significantly larger than the amplitude evoked by loss cues. These findings suggest that reward learning is impaired in individuals with IGD and that stimuli with a prior reward history may compromise inhibitory control, potentially serving as a critical factor in addiction development in this population.

Reward prediction-error promotes the neural encoding of episodic learning

Reward prediction-error carries significant implications for learning, facilitating the process by influencing prior knowledge and shaping future expectations and decisions. However, the electrophysiological mechanism through which reward prediction-error impacts learning remains incompletely understood. This study aimed to investigate the neural characteristics of reward prediction-error and its effect on recognition memory using Event-Related Potentials (ERPs). Behavioral results indicate that unsigned reward prediction-error indeed enhances recognition performance, with reaction times being slower in "remember" responses compared to correct predictions. The ERP findings conform to a three-stage model of reward prediction-error, suggesting that physical salience is swiftly detected (N1), followed by the processing of positive reward prediction-error (Feedback-Related Negativity, FRN), and ultimately, unsigned reward prediction-error or outcome evaluation (P300). Moreover, early physical salience signals were associated with subsequent "know" responses, while later unsigned reward prediction-error signals predicted subsequent recognition performance. This study not only revealed the neural processing mechanisms of reward prediction-error but also explored its impact on recognition performance, particularly familiarity or recollection processing.

From valence encoding to motivated behavior: A focus on the nucleus accumbens circuitry

How do our brains determine whether something is good or bad? The brain's ability to evaluate stimuli as positive or negative - by attributing valence - is fundamental to survival and decision-making. Different brain regions have been associated with valence encoding, including the nucleus accumbens (NAc). The NAc is predominantly composed of GABAergic medium spiny neurons (MSNs), which segregate into two distinct populations based on their dopamine receptor expression: D1-receptor-expressing (D1-MSNs) and D2-receptor-expressing neurons (D2-MSNs). Classical models propose a binary functional role, where D1-MSNs exclusively mediated reward and positive valence, while D2-MSNs processed aversion and negative valence. However, we now recognize that NAc MSN subpopulations operate in a more complex manner than previously thought, often working cooperatively rather than antagonistically in valence-related behaviors. This review synthesizes our current knowledge of valence-encoding neurocircuitry, with emphasis on the NAc. We examine electrophysiological, calcium imaging, optogenetic, chemogenetic and pharmacological studies detailing the contribution of NAc medium spiny neurons for rewarding and aversive responses. Finally, we explore emerging technical innovations that promise to advance our understanding of how the mammalian brain encodes valence and translates it into behavior.

Social interaction as a unique form of reward - Insights from healthy ageing and frontotemporal dementia

The drive for positive social interactions, or "social rewards", is an important motivator of human behaviour, conferring several adaptive benefits. Social motivation fluctuates across the lifespan, reflecting changes in goals and priorities at different developmental stages. In older adulthood, for instance, priorities tend to shift toward maintaining emotional wellbeing and resources over seeking novel gains. Contemporary theories of social interaction must account for such motivational shifts, addressing the enhancement of social processing in ageing and its decline in dementia. Here, we propose a framework to track the evolution of social motivation across the lifespan, focusing on three mechanisms: (i) social interactions as rewards, (ii) learning from social interactions, and (iii) the effort required for social interactions. We posit that social rewards hold equivalent or increased value later in life, enhancing older adults' social connections. Conversely, social rewards become devalued in neurodegenerative disorders such as frontotemporal dementia (FTD), resulting in social withdrawal. This integrative framework serves as a foundation for understanding adaptive and maladaptive trajectories of social motivation throughout the adult lifespan.

Cognition from genes to ecology: individual differences incognition and its potential role in a social network

There have now been many reports of intra-colony differences in how individuals learn on a variety of conditioning tasks in both honey bees and bumble bees. Yet the fundamental mechanistic and adaptive bases for this variation have yet to be fully described. This review summarizes a long series of investigations with the honey bee (Apis mellifera) that had the objective of describing the factors that contribute to this variation. Selection on haploid drones for extremes in learning performance revealed that genotype accounted for much of the variance. Neither age nor behavioral caste consistently accounted for observed variation on different conditioning protocols until genotype was controlled. Two subsequent Quantitative Trait Locus mapping studies identified a locus in the honey bee genome with a significant effect on the learning phenotype. Pharmacological and reverse genetic approaches, combined with neurophysiological analyses, confirmed that a biogenic amine receptor for tyramine affects expression of the trait. This work allowed for development of a hypothetical model of how that receptor functions in the brain to produce broad pleiotropic effects on behavior. Subsequent work used genotype as a treatment condition for evaluation of the variation under quasi-natural conditions, which revealed that individual variation reflects how foragers weigh known and novel resources in decision making. This work, together with other studies of individual differences, suggests a unifying framework for understanding how and why individuals differ in cognitive abilities.

Region-specific nucleus accumbens dopamine signals encode distinct aspects of avoidance learning

Avoidance learning-learning to avoid bad outcomes-is an essential survival behavior. Dopamine signals are widely observed in response to aversive stimuli, indicating they could play a role in learning about how to avoid these stimuli.1,2,3,4,5 However, it is unclear what computations dopamine signals perform to support avoidance learning. Furthermore, substantial heterogeneity in dopamine responses to aversive stimuli has been observed across nucleus accumbens (NAc) subregions.3,6,7,8 To understand how heterogeneous dopamine responses to aversive stimuli contribute to avoidance learning, we recorded NAc core (Core) and NAc ventromedial shell (vmShell) dopamine during a task in which mice could avoid a footshock punishment by moving to the opposite side of a 2-chamber apparatus during a 5-s warning cue. Both signals evolved substantially-but differently-with learning. We found that Core and vmShell dopamine signals responded oppositely to shocks at the beginning of training and oppositely to warning cues as cue-shock associations developed in mid-training. Core dopamine responses strengthen with learning and are especially evident during expert performance. vmShell dopamine responses to cues and shocks were present during early learning but were not sustained during expert performance. Our data support a model in which Core dopamine encodes prediction errors that guide the consolidation of avoidance learning, while vmShell dopamine guides initial cue-shock associations by signaling aversive salience.

Interpretable deep learning for deconvolutional analysis of neural signals

The widespread adoption of deep learning to model neural activity often relies on "black-box" approaches that lack an interpretable connection between neural activity and network parameters. Here, we propose using algorithm unrolling, a method for interpretable deep learning, to design the architecture of sparse deconvolutional neural networks and obtain a direct interpretation of network weights in relation to stimulus-driven single-neuron activity through a generative model. We introduce our method, deconvolutional unrolled neural learning (DUNL), and demonstrate its versatility by applying it to deconvolve single-trial local signals across multiple brain areas and recording modalities. We uncover multiplexed salience and reward prediction error signals from midbrain dopamine neurons, perform simultaneous event detection and characterization in somatosensory thalamus recordings, and characterize the heterogeneity of neural responses in the piriform cortex and across striatum during unstructured, naturalistic experiments. Our work leverages advances in interpretable deep learning to provide a mechanistic understanding of neural activity.

Functional dissection of a neuronal brain circuit mediating higher-order associative learning

A central feature characterizing the neural architecture of many species' brains is their capacity to form associative chains through learning. In elementary forms of associative learning, stimuli coinciding with reward or punishment become attractive or repulsive. Notably, stimuli previously learned as attractive or repulsive can themselves serve as reinforcers, establishing a cascading effect whereby they become associated with additional stimuli. When this iterative process is perpetuated, it results in higher-order associations. Here, we use odor conditioning in Drosophila and computational modeling to dissect the architecture of neuronal networks underlying higher-order associative learning. We show that the responsible circuit, situated in the mushroom bodies of the brain, is characterized by parallel processing of odor information and by recurrent excitatory and inhibitory feedback loops that empower odors to gain control over the dopaminergic valence-signaling system. Our findings establish a paradigmatic framework of a neuronal circuit diagram enabling the acquisition of associative chains.

Alpha-2 agonism in the locus coeruleus impairs learning driven by negative prediction error

Refining previous learning when environmental contingencies change is a critical adaptive function. Studies have shown that systemic noradrenaline (NA) manipulations, as well as optogenetic manipulations of the locus coeruleus (LC), the primary source of forebrain NA, can improve long-term retention of appetitive extinction. To determine whether the contribution of NA is specific to extinction or extends to other forms of learning where reward is less than expected, we suppressed LC activity with clonidine, an α2A-adrenergic receptor agonist, in two tasks: compound extinction, where two previously rewarded cues are presented together and no longer rewarded, and overexpectation, where animals are presented with two previously rewarded cues but receive a single reward rather than the expected two. In compound extinction, we found no differences between groups in training, extinction, or a spontaneous recovery test. However, animals that received clonidine reacquired responding to the previously extinguished cue significantly faster than saline-treated animals, suggesting weakened extinction learning. In overexpectation testing, the saline group responded significantly less to a stimulus that had undergone overexpectation relative to a control stimulus, indicating that they had recalibrated their estimation of reward magnitude following training where reward was less than expected. In contrast, clonidine-treated animals did not differ in responding to the overexpectation versus control stimuli, suggesting that clonidine impaired learning resulting from overexpectation. These results demonstrate that activity of the LC is important for learning to reduce responding in both extinction and overexpectation paradigms.

Aversion-induced dopamine reductions predict drug-taking and escape behaviors

Dopamine release in the nucleus accumbens core (NAcC) has long been associated with the promotion of motivated behavior. However, inhibited dopamine signaling can increase behavior in certain settings, such as during drug self-administration. While aversive environmental stimuli can reduce dopamine, it is unclear whether such stimuli reliably engage this mechanism in different contexts. Here we compared the physiological and behavioral responses to the same aversive stimulus in different designs to determine if there is uniformity in the manner that aversive stimuli are encoded and promote behavior. NAcC dopamine was measured using fiber photometry in male and female rats during cocaine self-administration sessions in which an acutely aversive 90 dB white noise was intermittently presented. In a separate group of rats, aversion-induced changes in dopamine were measured during an escape design in which operant responses terminated aversive white noise. Aversive white noise significantly reduced NAcC dopamine and increased cocaine self-administration in both male and female rats. The same relationship was observed in the escape design, in which white noise reduced dopamine and promoted the performance of escape behavior. In both designs, the magnitude of the dopamine reduction predicted behavioral performance. While prior research demonstrated that pharmacologically reduced dopamine signaling can promote intake, this report demonstrates that this physiological mechanism is naturally engaged by aversive environmental stimuli and is generalizable to non-drug contexts. These findings illustrate a common physiological signature in response to aversion that may promote both adaptive and maladaptive behavior.

Primate thalamic nuclei select abstract rules and shape prefrontal dynamics

Flexible behavior depends on abstract rules to generalize beyond specific instances and outcome monitoring to adjust actions. Cortical circuits are posited to read out rules from high-dimensional representations of task-relevant variables in prefrontal cortex (PFC). We instead hypothesized that converging inputs from PFC, directly or via basal ganglia (BGs), enable the thalamus to select rules. We measured activity across PFC and connected thalamic nuclei of monkeys applying rules. Abstract rule information first appeared in ventroanterior thalamus (VA)-the main thalamic hub between BG and PFC. Mediodorsal thalamus (MD) also represented rule information before PFC, persisting to help maintain activation of relevant PFC cell ensembles. MD, a major recipient of midbrain dopamine input, was the first to represent information about behavioral outcomes. A PFC-BG-thalamus model reproduced key findings, and thalamic-lesion modeling disrupted PFC rule representations. This suggests that the thalamus selects high-level cognitive information from PFC and monitors behavioral outcomes of these selections.

A mechanism linking dopamine's roles in reinforcement, movement and motivation

Dopamine neurons (DANs) play seemingly distinct roles in reinforcement, 1-3 motivation, 4,5 and movement, 6,7 and DA-modulating therapies relieve symptoms across a puzzling spectrum of neurologic and psychiatric symptoms. 8 Yet, the mechanistic relationship among these roles is unknown. Here, we show DA's tripartite roles are causally linked by a process in which phasic striatal DA rapidly and persistently recalibrates the propensity to move, a measure of vigor. Using a self-timed movement task, we found that single exposures to reward-related DA transients (both endogenous and exogenously-induced) exerted one-shot updates to movement timing-but in a surprising fashion. Rather than reinforce specific movement times, DA transients quantitatively changed movement timing on the next trial, with larger transients leading to earlier movements (and smaller to later), consistent with a stochastic search process that calibrates the frequency of movement. Both abrupt and gradual changes in external and internal contingencies-such as timing criterion, reward content, and satiety state-caused changes to the amplitude of DA transients that causally altered movement timing. The rapidity and bidirectionality of the one-shot effects are difficult to reconcile with gradual synaptic plasticity, and instead point to more flexible cellular mechanisms, such as DA-dependent modulation of neuronal excitability. Our findings shed light on how natural reinforcement, as well as DA-related disorders such as Parkinson's disease, could affect behavioral vigor.

The possible effect of inflammation on non-suicidal self-injury in adolescents with depression: a mediator of connectivity within corticostriatal reward circuitry

Non-suicidal self-injury (NSSI) in adolescent depression is a prevalent and clinically significant behavior linked to dysregulated peripheral inflammation and corticostriatal circuitry dysfunction. However, the neuroimmune mechanisms bridging these systems remain poorly understood. Here, we combined peripheral cytokine profiling with static/dynamic functional connectivity (sFC/dFC) analysis to investigate the potential influence of inflammaton on corticostriatal circuit related to NSSI. A set of peripheral blood inflammatory markers and resting-state functional magnetic resonance imaging (rs-fMRI) were collected in depression with NSSI (NSSI+), depression without NSSI (NSSI-), and healthy controls (HC). We first ascertain group differences in level of pro- and anti-inflammatory cytokines. And using ventral/dorsal striatal seeds, we compared whole-brain, voxel-wise sFC and dFC differences across three groups. Further, we tested the mediation effects of connectivity in the association between inflammatory markers and NSSI frequency. NSSI+ group exhibited elevated pro-inflammatory cytokines (C-reactive protein (CRP), interleukin (IL)-1, and IL-6) whereas reduced anti-inflammatory cytokines (IL-10), compared to NSSI- and HC. Neuroimaging analysis revealed corticostriatal dysconnectivity mainly characterized by static hyperconnectivity between dorsal striatum and thalamus, dynamic instability in dorsal striatum-lingual pathways, and dynamic rigidity in ventral striatum-prefrontal/temporal/occipital gyrus circuits. Critically, sFC of dorsal striatum-thalamus and dFC of dorsal striatum-lingual gyrus mediated the prospective association between altered CRP and NSSI frequency, establishing corticostriatal circuits as conduits for inflammatory effects on NSSI. By bridging molecular psychiatry with circuit neuroscience, this work advances precision management of NSSI in adolescent depression, prioritizing biomarker-driven strategies to disrupt neuroimmune maladaptation.

An active inference account of stuttering behavior

This paper presents an interpretation of stuttering behavior, based on the principles of the active inference framework. Stuttering is a neurodevelopmental disorder characterized by speech disfluencies such as repetitions, prolongations, and blocks. The principles of active inference, a theory of predictive processing and sentient behavior, can be used to conceptualize stuttering as a disruption in perception-action cycling underlying speech production. The theory proposed here posits that stuttering arises from aberrant sensory precision and prediction error dynamics, inhibiting syllable initiation. Relevant to this theory, two hypothesized mechanisms are proposed: (1) a mistiming in precision dynamics, and (2) excessive attentional focus. Both highlight the role of neural oscillations, prediction error, and hierarchical integration in speech production. This framework also explains the contextual variability of stuttering behaviors, including adaptation effects and fluency-inducing conditions. Reframing stuttering as a synaptopathy integrates neurobiological, psychological, and behavioral dimensions, suggesting disruptions in precision-weighting mediated by neuromodulatory systems. This active inference perspective provides a unified account of stuttering and sets the stage for innovative research and therapeutic approaches.

Managing Expectations and Predicting Willingness to Pay in Novel Healthy Foods Development in East Africa

This study explores the factors influencing consumer willingness to pay (WTP) for novel, healthy, and locally produced food products in East Africa, focusing on sensory experiences and packaging design. Conducted in Tanzania, Uganda, and Kenya, the research includes two complementary studies: Study A examines sensory evaluations (taste, texture, aroma, color, and general acceptance) as predictors of WTP, while Study B assesses the impact of visual packaging features (e.g., typography, illustrations, and product windows) on consumer perceptions and WTP. Study A highlights that general acceptance (GA) is the strongest predictor of WTP, driven primarily by taste, texture, and aroma, while visual sensory cues play a secondary role. In contrast, Study B demonstrates that packaging design features, such as product visibility and ingredient-focused imagery, significantly influence WTP, with health messaging increasing perceived value but locality cues reducing it, likely due to cultural biases against packaged local products. The results reveal a critical difference: WTP is more stable and predictable in sensory evaluations but more volatile in response to packaging designs, driven by consumer expectations. These findings underscore the importance of aligning sensory and visual attributes to understand consumer expectations and enhance WTP for innovative food products in emerging markets.

Exploring Habits in Anorexia Nervosa: Promise, Pitfalls, and Progress

PURPOSE OF REVIEW

Habits, characterized by automaticity and insensitivity to outcomes, may be key to the persistence of maladaptive behaviors in anorexia nervosa (AN). This review examines the status of habit research in AN, focusing on insights from task-based assessments.

RECENT FINDINGS

Findings indicate dysfunction in the frontostriatal circuits associated with habitual and goal-directed behaviors, with some studies linking neural disturbances to habit measures or clinically relevant behaviors. Heightened habitual tendencies in AN have consistently been reported using self-reports, while research utilizing experimental paradigms has yielded mixed results and efforts to capture real-world habits in AN remain limited. Some experimental paradigms appear more sensitive than others, but all face challenges associated with studying habits in the lab. Promising new approaches will need to be adopted and efforts made to capture real-world habits. Understanding which habits are problematic, when in illness and for whom they dominate, could make good on the promise of habit-focused treatments for AN.

Premovement neuronal activity in the primary motor cortex is associated with the initiation of ipsilateral hand movements in monkeys

The primary motor cortex (M1) is believed to be a cortical center for the execution of limb movements. Although M1 neurons mainly project to the spinal cord on the contralateral side, some M1 neurons project to the ipsilateral side via the uncrossed corticospinal pathway. Moreover, some M1 neurons are activated during ipsilateral forelimb movements. However, the extent to which M1 neurons are involved in ipsilateral movement execution has not been determined. Therefore, we investigated the involvement of M1 neurons in the initiation of ipsilateral and contralateral hand movements by examining trial-by-trial correlations between premovement neuronal spikes and hand movement reaction times in monkeys. Overall, the activity of M1 neurons was more strongly correlated with the reaction times for contralateral hand movements than those for ipsilateral hand movements. However, the activity of some M1 neurons was correlated with reaction times for ipsilateral hand movements, and these correlations were as strong as those between the activity of other M1 neurons and reaction times for contralateral hand movements. This finding suggests that one subset of M1 neurons sends motor commands for ipsilateral hand movements to the same extent as another subset of M1 neurons sends motor commands for contralateral hand movements.

Eligibility traces as a synaptic substrate for learning

Animals can learn to associate a behavior or a stimulus with a delayed reward, this is essential for survival. A mechanism proposed for bridging this gap are synaptic eligibility traces, which are slowly decaying tags, which can lead to synaptic plasticity if followed by rewards. Recently, experiments have demonstrated the existence of synaptic eligibility traces in diverse neural systems, depending on either neuromodulators or plateau potentials. Evidence for both eligibility trace-dependent potentiation and depression of synaptic efficacies has emerged. We discuss the commonalities and differences of these different results. We show why the existence of both potentiation and depression is important because these opposing forces can lead to a synaptic stopping rule. Without a stopping rule, synapses would saturate at their upper bound thus leading to a loss of selectivity and representational power. We discuss the possible underlying mechanisms of the eligibility traces as well as their functional and theoretical significance.

The neurobiology of overeating

Food intake serves to maintain energy homeostasis; however, overeating can result in obesity, which is associated with serious health complications. In this review, we explore the intricate relationship between overeating, obesity, and the underlying neurobiological mechanisms. We review the homeostatic and hedonic feeding systems, highlighting the role of the hypothalamus and reward systems in controlling food intake and energy balance. Dysregulation in both these systems leads to overeating, as seen in genetic syndromes and environmental models affecting appetite regulation when consuming highly palatable food. The concept of "food addiction" is examined, drawing parallels to drug addiction. We discuss the cellular substrate for addiction-related behavior and current pharmacological obesity treatments-in particular, GLP-1 receptor agonists-showcasing synaptic plasticity in the context of overeating and palatable food exposure. A comprehensive model integrating insights from addiction research is proposed to guide effective interventions for maladaptive feeding behaviors. Ultimately, unraveling the neurobiological basis of overeating holds promise for addressing the pressing public health issue of obesity.

An axonal brake on striatal dopamine output by cholinergic interneurons

Depolarization of axons is necessary for somatic action potentials to trigger axonal neurotransmitter release. Here we show that striatal cholinergic interneurons (ChIs) and nicotinic receptors (nAChRs) on mouse dopamine axons interrupt this relationship. After nAChR-mediated depolarization, dopamine release by subsequent depolarization events was suppressed for ~100 ms. This suppression was not due to depletion of dopamine or acetylcholine, but to a limited reactivation of dopamine axons after nAChR-mediated depolarization, and is more prominent in dorsal than in ventral striatum. In vivo, nAChRs predominantly depressed dopamine release, as nAChR antagonism in dorsal striatum elevated dopamine detected with optic-fiber photometry of dopamine sensor GRABDA2m and promoted conditioned place preference. Our findings reveal that ChIs acting via nAChRs transiently limit the reactivation of dopamine axons for subsequent action potentials in dopamine neurons and therefore generate a dynamic inverse scaling of dopamine release according to ChI activity.

Dopamine in the tail of the striatum facilitates avoidance in threat-reward conflicts

Responding appropriately to potential threats before they materialize is critical to avoiding disastrous outcomes. Here we examine how threat-coping behavior is regulated by the tail of the striatum (TS) and its dopamine input. Mice were presented with a potential threat (a moving object) while pursuing rewards. Initially, the mice failed to obtain rewards but gradually improved in later trials. We found that dopamine in TS promoted avoidance of the threat, even at the expense of reward acquisition. Furthermore, the activity of dopamine D1 receptor-expressing neurons promoted threat avoidance and prediction. In contrast, D2 neurons suppressed threat avoidance and facilitated overcoming the potential threat. Dopamine axon activation in TS not only potentiated the responses of dopamine D1 receptor-expressing neurons to novel sensory stimuli but also boosted them acutely. These results demonstrate that an opponent interaction of D1 and D2 neurons in the TS, modulated by dopamine, dynamically regulates avoidance and overcoming potential threats.

Pre-choice midbrain fluctuations affect self-control in food choice: A functional magnetic resonance imaging (fMRI) study

Recent studies have shown that spontaneous pre-stimulus fluctuations in brain activity affect higher-order cognitive processes, including risky decision-making, cognitive flexibility, and aesthetic judgments. However, there is currently no direct evidence to suggest that pre-choice activity influences value-based decisions that require self-control. We examined the impact of fluctuations in pre-choice activity in key regions of the reward system on self-control in food choice. In the functional magnetic resonance imaging (fMRI) scanner, 49 participants made 120 food choices that required self-control in high and low working memory load conditions. The task was designed to ensure that participants were cognitively engaged and not thinking about upcoming choices. We defined self-control success as choosing a food item that was healthier over one that was tastier. The brain regions of interest (ROIs) were the ventral tegmental area (VTA), putamen, nucleus accumbens (NAc), and caudate nucleus. For each participant and condition, we calculated the mean activity in the 3-s interval preceding the presentation of food stimuli in successful and failed self-control trials. These activities were then used as predictors of self-control success in a fixed-effects logistic regression model. The results indicate that increased pre-choice VTA activity was linked to a higher probability of self-control success in a subsequent food-choice task within the low-load condition, but not in the high-load condition. We posit that pre-choice fluctuations in VTA activity change the reference point for immediate (taste) reward evaluation, which may explain our finding. This suggests that the neural context of decisions may be a key factor influencing human behavior.

The Computational Bottleneck of Basal Ganglia Output (and What to Do About it)

What the basal ganglia do is an oft-asked question; answers range from the selection of actions to the specification of movement to the estimation of time. Here, I argue that how the basal ganglia do what they do is a less-asked but equally important question. I show that the output regions of the basal ganglia create a stringent computational bottleneck, both structurally, because they have far fewer neurons than do their target regions, and dynamically, because of their tonic, inhibitory output. My proposed solution to this bottleneck is that the activity of an output neuron is setting the weight of a basis function, a function defined by that neuron's synaptic contacts. I illustrate how this may work in practice, allowing basal ganglia output to shift cortical dynamics and control eye movements via the superior colliculus. This solution can account for troubling issues in our understanding of the basal ganglia: why we see output neurons increasing their activity during behavior, rather than only decreasing as predicted by theories based on disinhibition, and why the output of the basal ganglia seems to have so many codes squashed into such a tiny region of the brain.

Dopamine prediction error signaling in a unique nigrostriatal circuit is critical for associative fear learning

Learning by experience that certain cues in the environment predict danger is crucial for survival. How dopamine (DA) circuits drive this form of associative learning is not fully understood. Here, in male mice, we demonstrate that DA neurons projecting to a unique subregion of the dorsal striatum, the posterior tail of the striatum (TS), encode a prediction error (PE) signal during associative fear learning. These DA neurons are necessary specifically during acquisition of fear learning, but not once the fear memory is formed, and are not required for forming cue-reward associations. Notably, temporally-precise inhibition or excitation of DA terminals in TS impairs or enhances fear learning, respectively. Furthermore, neuronal activity in TS is crucial for the acquisition of associative fear learning and learning-induced activity patterns in TS critically depend on DA input. Together, our results reveal that DA PE signaling in a non-canonical nigrostriatal circuit is important for driving associative fear learning.

Hedonic eating is controlled by dopamine neurons that oppose GLP-1R satiety

Hedonic eating is defined as food consumption driven by palatability without physiological need. However, neural control of palatable food intake is poorly understood. We discovered that hedonic eating is controlled by a neural pathway from the peri-locus ceruleus to the ventral tegmental area (VTA). Using photometry-calibrated optogenetics, we found that VTA dopamine (VTADA) neurons encode palatability to bidirectionally regulate hedonic food consumption. VTADA neuron responsiveness was suppressed during food consumption by semaglutide, a glucagon-like peptide receptor 1 (GLP-1R) agonist used as an antiobesity drug. Mice recovered palatable food appetite and VTADA neuron activity during repeated semaglutide treatment, which was reversed by consumption-triggered VTADA neuron inhibition. Thus, hedonic food intake activates VTADA neurons, which sustain further consumption, a mechanism that opposes appetite reduction by semaglutide.

Human social behavior and oxytocin: Molecular and neuronal mechanisms

Oxytocin (OT) is a hormone that is crucial for regulating various human social behaviors, such as maternal instinct, empathy, and trust. Its secretion in the brain is triggered by social stimuli. Recent research demonstrated impaired regulation of OT secretion and reduced social behaviors in patients with arginine vasopressin deficiency (central diabetes insipidus). OT interacts with other hormones to regulate human trust. Moreover, it has been shown to generate feelings of attachment and trust toward other and familiar consumer brands, thereby, potentially impacting personal consumption, which is a significant aspect of economic activity. This review provided insights into the molecular and neural mechanisms of OT in regulating human social behavior, including both social and economic activities.

Volitional Regulation and Transferable Patterns of Midbrain Oscillations

Dopaminergic brain areas are crucial for cognition and their dysregulation is linked to neuropsychiatric disorders typically treated with pharmacological interventions. These treatments often have side effects and variable effectiveness, underscoring the need for alternatives. We introduce the first demonstration of neurofeedback using local field potentials (LFP) from the ventral tegmental area (VTA). This approach leverages the real-time temporal resolution of LFP and ability to target deep brain. In our study, two male rhesus macaque monkeys (Macaca mulatta) learned to regulate VTA beta power using a customized normalized metric to stably quantify VTA LFP signal modulation. The subjects demonstrated flexible and specific control with different strategies for specific frequency bands, revealing new insights into the plasticity of VTA neurons contributing to oscillatory activity that is functionally relevant to many aspects of cognition. Excitingly, the subjects showed transferable patterns, a key criterion for clinical applications beyond training settings. This work provides a foundation for neurofeedback-based treatments, which may be a promising alternative to conventional approaches and open new avenues for understanding and managing neuropsychiatric disorders.

Temporal fMRI Dynamics Map Dopamine Physiology

Spatial variations in dopamine function are linked to cognition and substance use disorders but are challenging to characterize with current methods. Because dopamine influences blood vessel dilation, we hypothesized that hemodynamic latency, which reflects BOLD signal timing, could serve as an indirect marker of dopamine physiology. Across four datasets, we found a topography of hemodynamic latencies that precisely distinguished the nucleus accumbens, a dopaminergic region implicated in motivation and substance abuse, from other striatal regions. Using PET, genetics, and pharmacology, we found that hemodynamic latencies are robustly related to dopamine function and dopamine-linked behavior. In individuals with cocaine use disorder, we observed a spatial gradient of altered hemodynamic latencies in the striatum. This pattern independently predicted nicotine use, revealing a conserved physiological profile associated with addictive substance use. Hemodynamic latencies map regional, individual, and pathological differences linked to dopamine, opening new avenues for indirectly assessing the role of dopamine in healthy cognition and disease.

The macaque medial prefrontal cortex simultaneously represents self and others' reward prediction error

Learning the causal structures of social environments involves predicting significant events (e.g., rewards) and detecting prediction errors for each agent. Whether the brain can simultaneously compute reward prediction errors for self (S-RPE) and others (O-RPE), and which neurons are responsible, is unclear. Here, we condition two monkeys with identical visual stimuli predicting different reward outcomes and find that dorsomedial prefrontal neurons represent both S-RPE and O-RPE simultaneously. Neuronal signatures of RPE are agent and sign specific, forming distinct populations for positive and negative S-RPE and O-RPE. A linear decoder trained on neurons encoding O-RPE, but not S-RPE, successfully discriminates RPE. Further investigation identifies coexisting actual reward and prediction confirmation signals for others. These results highlight the presence of neuronal mechanisms in the primate brain that update the value of environmental stimuli simultaneously for oneself and others, enabling primates to comprehend the causal structure of the world from the perspective of others.

Reward expectation and receipt differentially modulate the spiking of accumbens D1+ and D2+ neurons

The nucleus accumbens (NAc) helps govern motivation to pursue reward. Two distinct sets of NAc projection neurons-expressing dopamine D1 vs. D2 receptors-are thought to promote and suppress motivated behaviors, respectively. However, support for this conceptual framework is limited: in particular, the spiking patterns of these distinct cell types during motivated behavior have been largely unknown. Using optogenetic tagging, we recorded the spiking of identified D1+ and D2+ neurons in the NAc core as unrestrained rats performed an operant task in which motivation to initiate work tracks recent reward rate. D1+ neurons preferentially increased firing as rats initiated trials and fired more when reward expectation was higher. By contrast, D2+ cells preferentially increased firing later in the trial, especially in response to reward delivery-a finding not anticipated from current theoretical models. Our results provide new evidence for the specific contribution of NAc D1+ cells to self-initiated approach behavior and will spur updated models of how D2+ cells contribute to learning.

Spatial localization of hippocampal replay requires dopamine signaling

Sequenced reactivations of hippocampal neurons called replays, concomitant with sharp-wave ripples in the local field potential, are critical for the consolidation of episodic memory, but whether replays depend on the brain's reward or novelty signals is unknown. Here, we combined chemogenetic silencing of dopamine neurons in ventral tegmental area (VTA) and simultaneous electrophysiological recordings in dorsal hippocampal CA1, in freely behaving male rats experiencing changes to reward magnitude and environmental novelty. Surprisingly, VTA silencing did not prevent ripple increases where reward was increased, but caused dramatic, aberrant ripple increases where reward was unchanged. These increases were associated with increased reverse-ordered replays. On familiar tracks this effect disappeared, and ripples tracked reward prediction error (RPE), indicating that non-VTA reward signals were sufficient to direct replay. Our results reveal a novel dependence of hippocampal replay on dopamine, and a role for a VTA-independent RPE signal that is reliable only in familiar environments.

Methamphetamine-induced adaptation of learning rate dynamics depend on baseline performance

The ability to calibrate learning according to new information is a fundamental component of an organism's ability to adapt to changing conditions. Yet, the exact neural mechanisms guiding dynamic learning rate adjustments remain unclear. Catecholamines appear to play a critical role in adjusting the degree to which we use new information over time, but individuals vary widely in the manner in which they adjust to changes. Here, we studied the effects of a low dose of methamphetamine (MA), and individual differences in these effects, on probabilistic reversal learning dynamics in a within-subject, double-blind, randomized design. Participants first completed a reversal learning task during a drug-free baseline session to provide a measure of baseline performance. Then they completed the task during two sessions, one with MA (20 mg oral) and one with placebo (PL). First, we showed that, relative to PL, MA modulates the ability to dynamically adjust learning from prediction errors. Second, this effect was more pronounced in participants who performed moderately low at baseline. These results present novel evidence for the involvement of catecholaminergic transmission on learning flexibility and highlights that baseline performance modulates the effect of the drug.

Prospective contingency explains behavior and dopamine signals during associative learning

Associative learning depends on contingency, the degree to which a stimulus predicts an outcome. Despite its importance, the neural mechanisms linking contingency to behavior remain elusive. In the present study, we examined the dopamine activity in the ventral striatum-a signal implicated in associative learning-in a Pavlovian contingency degradation task in mice. We show that both anticipatory licking and dopamine responses to a conditioned stimulus decreased when additional rewards were delivered uncued, but remained unchanged if additional rewards were cued. These results conflict with contingency-based accounts using a traditional definition of contingency or a new causal learning model (ANCCR), but can be explained by temporal difference (TD) learning models equipped with an appropriate intertrial interval state representation. Recurrent neural networks trained within a TD framework develop state representations akin to our best 'handcrafted' model. Our findings suggest that the TD error can be a measure that describes both contingency and dopaminergic activity.

A unified neural representation model for spatial and conceptual computations

The hippocampus and entorhinal cortex encode spaces by spatially local and hexagonal grid activity patterns (place cells and grid cells), respectively. In addition, the same brain regions also implicate neural representations for nonspatial, semantic concepts (concept cells). These observations suggest that neurocomputational mechanisms for spatial knowledge and semantic concepts are related in the brain. However, the exact relationship remains to be understood. Here, we show a mathematical correspondence between a value function for goal-directed spatial navigation and an information measure for word embedding models in natural language processing. Based on this relationship, we integrate spatial and semantic computations into a neural representation model called "disentangled successor information" (DSI). DSI generates biologically plausible neural representations: spatial representations like place cells and grid cells, and concept-specific word representations which resemble concept cells. Furthermore, with DSI representations, we can perform inferences of spatial contexts and words by a common computational framework based on simple arithmetic operations. This computation can be biologically interpreted by partial modulations of cell assemblies of nongrid cells and concept cells. Our model offers a theoretical connection of spatial and semantic computations and suggests possible computational roles of hippocampal and entorhinal neural representations.

Investigating the cortical effect of false positive feedback on motor learning in motor imagery based rehabilitative BCI training

BACKGROUND

Motor imagery-based brain-computer interface (MI-BCI) is a promising solution for neurorehabilitation. Many studies proposed that reducing false positive (FP) feedback is crucial for inducing neural plasticity by BCI technology. However, the effect of FP feedback on cortical plasticity induction during MI-BCI training is yet to be investigated.

OBJECTIVE

This study aims to validate the hypothesis that FP feedback affects the cortical plasticity of the user's MI during MI-BCI training by first comparing two different asynchronous MI-BCI paradigms (with and without FP feedback), and then comparing its effectiveness with that of conventional motor learning methods (passive and active training).

METHODS

Twelve healthy volunteers and four patients with stroke participated in the study. We implemented two electroencephalogram-driven asynchronous MI-BCI systems with different feedback conditions. The feedback was provided by a hand exoskeleton robot performing hand open/close task. We assessed the hemodynamic responses in two different feedback conditions and compared them with two conventional motor learning methods using functional near-infrared spectroscopy with an event-related design. The cortical effects of FP feedback were analyzed in different paradigms, as well as in the same paradigm via statistical analysis.

RESULTS

The MI-BCI without FP feedback paradigm induced higher cortical activation in MI, focusing on the contralateral motor area, compared to the paradigm with FP feedback. Additionally, within the same paradigm providing FP feedback, the task period immediately following FP feedback elicited a lower hemodynamic response in the channel located over the contralateral motor area compared to the MI-BCI paradigm without FP feedback (p = 0.021 for healthy people; p = 0.079 for people with stroke). In contrast, task trials where there was no FP feedback just before showed a higher hemodynamic response, similar to the MI-BCI paradigm without FP feedback (p = 0.099 for healthy people, p = 0.084 for people with stroke).

CONCLUSIONS

FP feedback reduced cortical activation for the users during MI-BCI training, suggesting a potential negative effect on cortical plasticity. Therefore, minimizing FP feedback may enhance the effectiveness of rehabilitative MI-BCI training by promoting stronger cortical activation and plasticity, particularly in the contralateral motor area.

Fluorescence detection of dopamine signaling to the primate striatum in relation to stimulus-reward associations

Dopamine (DA) signals to the striatum play critical roles in shaping and sustaining stimulus-reward associations. In primates, however, the dynamics of the DA signals remain unknown since conventional methods are not necessarily appropriate in terms of the spatiotemporal resolution or chemical specificity sufficient for detecting the DA signals. In our study, fiber photometry with a fluorescent DA sensor was employed to identify reward-related DA transients in the monkey striatum. This technique, which directly monitors local DA release, reveals a reward prediction error signal in the anterior putamen originating from midbrain DA neurons. Further, DA transients in the head of the caudate nucleus exhibit a value-based response to reward-predicting stimuli. These signals have been found to arise from two separate groups of DA neurons in the substantia nigra pars compacta. The present results demonstrate that fluorescence DA monitoring is applicable to detect DA signals in the primate striatum for investigating their roles.

Striatal dopamine represents valence on dynamic regional scales

Adaptive decision making relies on dynamic updating of learned associations where environmental cues come to predict valenced stimuli, such as food or threat. Cue-guided behavior depends on a network of brain systems, including dopaminergic projections to the striatum. Critically, it remains unclear how dopamine signaling across the striatum encodes multi-valent, dynamic learning contexts, where positive and negative associations must be rapidly disambiguated. To understand this, we employed a Pavlovian discrimination paradigm, where cues predicting food or threat were intermingled during conditioning sessions, and their meaning was serially reversed across training. We found that male and female rats readily distinguished these cues and updated their behavior rapidly upon valence reversal. Using fiber photometry, we recorded dopamine signaling in three major striatal subregions - the dorsolateral striatum (DLS), the nucleus accumbens (NAc) core, and the nucleus accumbens medial shell - finding that valence was represented uniquely across all three regions, indicative of local signals biased for value and salience. Further, ambiguity introduced by cue reversals reshaped striatal dopamine on different timelines: nucleus accumbens signals updated more readily than those in the DLS. Together, these results indicate that striatal dopamine flexibly encodes stimulus valence according to region-specific rules, and these signals are dynamically modulated by changing contingencies in the resolution of ambiguity about the meaning of environmental cues.Significance Statement Adaptive decision making relies on updating learned associations to disambiguate predictions of reward or threat. This cue-guided behavior depends on striatal dopamine, but it remains unclear how dopamine signaling encodes multi-valent, dynamic learning contexts. Here, we employed a paradigm where cues predicting positive and negative outcomes were intermingled, and their meaning was serially reversed across time. We recorded dopamine signaling, finding heterogeneous patterns of valence encoding across striatal subregions, and cue reversal reshaped subregional signals on different timelines. Our results suggest that dopamine flexibly encodes dynamic learning contexts to resolve ambiguity about the meaning of environmental cues.

Cross-species dissection of the modular role of the ventral tegmental area in depressive disorders

Depressive disorders, including major depressive disorder (MDD), represent one of the most prevalent set of disorders worldwide. MDD is characterized by a range of cognitive, behavioral, and neurobiological changes that contribute to the vast array of symptom profiles that make this disorder particularly difficult to treat. A multitude of established evidence suggests a role for the dopamine system, stemming in part from the ventral tegmental area (VTA), in mediating symptoms and behavioral changes that underlie depression. Developments in cutting-edge technologies in pre-clinical models of depressive phenotypes, such as retrograde tracing, electrophysiological recordings, immunohistochemistry, and molecular profiling, have allowed a deeper characterization of singular VTA neuron molecular, physiological, and projection properties. These developments have highlighted that the VTA is not a homogenous cell population but instead comprises vast cellular diversity that underscores its modular role across various functions related to reward processing, aversion, salience processing, learning and motivation. In this review, we begin by introducing the various cell types and brain regions that comprise the VTA circuitry. Then, we introduce the role of the VTA in reward processing as it compares to aversion processing. Next, we characterize distinct neural pathways within the VTA circuitry to understand the effects of chronic social and non-social stress and tie together how these neurobiological changes manifest into specific behavioral phenotypes. Finally, we relate these preclinical findings to clinical findings to parse the heterogeneity of depressive phenotypes and explain the efficacy of recent novel pharmacological interventions that may target the VTA in MDD.

Oxytocin signaling in the ventral tegmental area mediates social isolation-induced craving for social interaction

BACKGROUND

Social interaction is crucial for mental health across animal species. Social experiences, especially in early-life stages, strongly influence brain function and social behavior later in life. Acute social isolation (SI) increases motivation to seek social interaction, but little is known about its underlying neuronal and circuitry mechanisms. Here, we focus on oxytocin signaling in the ventral tegmental area (VTA), a vital node of the brain's reward network, as a potential mechanism for SI-induced craving for social interaction.

METHODS

Adolescent (4-week-old) or adult (14-week-old) male C57BL/6J mice underwent a 1-week SI. Free interaction, object exploration, three-chamber social approach, and habituation tests were used to assess social and non-social behavior changes. Viral vectors were used to decipher the underlying neural circuitry, and chemogenetic techniques were applied to modify neuronal activity.

RESULTS

We found that in male C57BL/6J mice, SI during adolescence, but not adulthood, leads to increased craving for social interaction and object exploration, accompanied by impaired social habituation, social novelty preference, and social recognition memory (SRM). SI-induced craving for social interaction and SRM deficit is still observed upon regrouping. Through cell-type-specific manipulations with designer receptors exclusively activated by designer drugs (DREADD), we show that oxytocin neurons in the paraventricular nucleus of the hypothalamus (PVN) are crucial for SI-induced social behavior changes. Chemogenetic activation of PVN oxytocin neurons recapitulates social behavior changes observed in SI mice, whereas chemogenetic inhibition of oxytocin neurons prevents social behavior changes caused by SI. Moreover, we found that dopaminergic neurons in the VTA mediate SI-induced craving for social interaction through their projections to the medial prefrontal cortex (mPFC), but not to the nucleus accumbens. Injection of a specific oxytocin receptor antagonist L368,899 into the VTA or chemical lesions of dopaminergic axon terminals in the mPFC with local application of 6-hydroxydopamine ameliorates SI-induced social behavior changes.

CONCLUSIONS

These findings suggest that adolescent SI has enduring effects on social behaviors in male mice through an oxytocinergic modulation of the VTA-to-mPFC dopaminergic circuit activity.

How the dynamic interplay of cortico-basal ganglia-thalamic pathways shapes the time course of deliberation and commitment

Although the cortico-basal ganglia-thalamic (CBGT) network is identified as a central circuit for decision-making, the dynamic interplay of multiple control pathways within this network in shaping decision trajectories remains poorly understood. Here we develop and apply a novel computational framework - CLAW (Circuit Logic Assessed via Walks) - for tracing the instantaneous flow of neural activity as it progresses through CBGT networks engaged in a virtual decision-making task. Our CLAW analysis reveals that the complex dynamics of network activity is functionally dissectible into two critical phases: deliberation and commitment. These two phases are governed by distinct contributions of underlying CBGT pathways, with indirect and pallidostriatal pathways influencing deliberation, while the direct pathway drives action commitment. We translate CBGT dynamics into the evolution of decision-related policies, based on three previously identified control ensembles (responsiveness, pliancy, and choice) that encapsulate the relationship between CBGT activity and the evidence accumulation process. Our results demonstrate two contrasting strategies for decision-making. Fast decisions, with direct pathway dominance, feature an early response in both boundary height and drift rate, leading to a rapid collapse of decision boundaries and a clear directional bias. In contrast, slow decisions, driven by indirect and pallidostriatal pathway dominance, involve delayed changes in both decision policy parameters, allowing for an extended period of deliberation before commitment to an action. These analyses provide important insights into how the CBGT circuitry can be tuned to adopt various decision strategies and how the decision-making process unfolds within each regime.

A Neural Circuit Framework for Economic Choice: From Building Blocks of Valuation to Compositionality in Multitasking

Value-guided decisions are at the core of reinforcement learning and neuroeconomics, yet the basic computations they require remain poorly understood at the mechanistic level. For instance, how does the brain implement the multiplication of reward magnitude by probability to yield an expected value? Where within a neural circuit is the indifference point for comparing reward types encoded? How do learned values generalize to novel options? Here, we introduce a biologically plausible model that adheres to Dale's law and is trained on five choice tasks, offering potential answers to these questions. The model captures key neurophysiological observations from the orbitofrontal cortex of monkeys and generalizes to novel offer values. Using a single network model to solve diverse tasks, we identified compositional neural representations-quantified via task variance analysis and corroborated by curriculum learning. This work provides testable predictions that probe the neural basis of decision making and its disruption in neuropsychiatric disorders.

Dual neuromodulatory dynamics underlie birdsong learning

Although learning in response to extrinsic reinforcement is theorized to be driven by dopamine signals that encode the difference between expected and experienced rewards1,2, skills that enable verbal or musical expression can be learned without extrinsic reinforcement. Instead, spontaneous execution of these skills is thought to be intrinsically reinforcing3,4. Whether dopamine signals similarly guide learning of these intrinsically reinforced behaviours is unknown. In juvenile zebra finches learning from an adult tutor, dopamine signalling in a song-specialized basal ganglia region is required for successful song copying, a spontaneous, intrinsically reinforced process5. Here we show that dopamine dynamics in the song basal ganglia faithfully track the learned quality of juvenile song performance on a rendition-by-rendition basis. Furthermore, dopamine release in the basal ganglia is driven not only by inputs from midbrain dopamine neurons classically associated with reinforcement learning but also by song premotor inputs, which act by means of local cholinergic signalling to elevate dopamine during singing. Although both cholinergic and dopaminergic signalling are necessary for juvenile song learning, only dopamine tracks the learned quality of song performance. Therefore, dopamine dynamics in the basal ganglia encode performance quality during self-directed, long-term learning of natural behaviours.

Natural behaviour is learned through dopamine-mediated reinforcement

Many natural motor skills, such as speaking or locomotion, are acquired through a process of trial-and-error learning over the course of development. It has long been hypothesized, motivated by observations in artificial learning experiments, that dopamine has a crucial role in this process. Dopamine in the basal ganglia is thought to guide reward-based trial-and-error learning by encoding reward prediction errors1, decreasing after worse-than-predicted reward outcomes and increasing after better-than-predicted ones. Our previous work in adult zebra finches-in which we changed the perceived song quality with distorted auditory feedback-showed that dopamine in Area X, the singing-related basal ganglia, encodes performance prediction error: dopamine is suppressed after worse-than-predicted (distorted syllables) and activated after better-than-predicted (undistorted syllables) performance2. However, it remains unknown whether the learning of natural behaviours, such as developmental vocal learning, occurs through dopamine-based reinforcement. Here we tracked song learning trajectories in juvenile zebra finches and used fibre photometry3 to monitor concurrent dopamine activity in Area X. We found that dopamine was activated after syllable renditions that were closer to the eventual adult version of the song, compared with recent renditions, and suppressed after renditions that were further away. Furthermore, the relationship between dopamine and song fluctuations revealed that dopamine predicted the future evolution of song, suggesting that dopamine drives behaviour. Finally, dopamine activity was explained by the contrast between the quality of the current rendition and the recent history of renditions-consistent with dopamine's hypothesized role in encoding prediction errors in an actor-critic reinforcement-learning model4,5. Reinforcement-learning algorithms6 have emerged as a powerful class of model to explain learning in reward-based laboratory tasks, as well as for driving autonomous learning in artificial intelligence7. Our results suggest that complex natural behaviours in biological systems can also be acquired through dopamine-mediated reinforcement learning.

Foveal vision reduces neural resources in agent-based game learning

Efficient processing of information is crucial for the optimization of neural resources in both biological and artificial visual systems. In this paper, we study the efficiency that may be obtained via the use of a fovea. Using biologically-motivated agents, we study visual information processing, learning, and decision making in a controlled artificial environment, namely the Atari Pong video game. We compare the resources necessary to play Pong between agents with and without a fovea. Our study shows that a fovea can significantly reduce the neural resources, in the form of number of neurons, number of synapses, and number of computations, while at the same time maintaining performance at playing Pong. To our knowledge, this is the first study in which an agent must simultaneously optimize its visual system, along with its decision making and action generation capabilities. That is, the visual system is integral to a complete agent.

Humans forage for reward in reinforcement learning tasks

How do we make good decisions in uncertain environments? In psychology and neuroscience, the classic view is that we calculate the value of each option, compare them, and choose the most rewarding modulo exploratory noise. An ethologist, conversely, would argue that we commit to one option until its value drops below a threshold and then explore alternatives. Because the fields use incompatible methods, it remains unclear which view better describes human decision-making. Here, we found that humans use compare-to-threshold computations in classic compare-alternative tasks. Because compare-alternative computations are central to the reinforcement-learning (RL) models typically used in the cognitive and brain sciences, we developed a novel compare-to-threshold model ("foraging"). Compared to previous RL models, the foraging model better fit participant behavior, better predicted the tendency to repeat choices, and predicted held-out participants that were almost impossible under compare-alternative models. These results suggest that humans use compare-to-threshold computations in sequential decision-making.

Brain dopamine responses to ultra-processed milkshakes are highly variable and not significantly related to adiposity in humans

Ultra-processed foods high in fat and sugar have been theorized to be addictive due to their purported ability to induce an exaggerated post-ingestive brain dopamine response akin to drugs of abuse. Using [11C]raclopride positron emission tomography (PET) displacement methods used to measure brain dopamine responses to addictive drugs, we measured striatal dopamine responses beginning 30 min after ingesting an ultra-processed milkshake high in fat and sugar in 50 young, healthy adults over a wide body mass index (BMI) range (20-45 kg/m2). Surprisingly, milkshake consumption did not result in a significant post-ingestive dopamine response in the striatum (p = 0.62) nor in any striatal subregion (p > 0.33), and the highly variable interindividual responses were not significantly related to adiposity (BMI: r = 0.076, p = 0.51; % body fat: r = 0.16, p = 0.28). Thus, post-ingestive striatal dopamine responses to an ultra-processed milkshake were likely substantially smaller than for many addictive drugs and below the limits of detection using standard PET methods.

Opponent control of reinforcement by striatal dopamine and serotonin

The neuromodulators dopamine (DA) and serotonin (5-hydroxytryptamine; 5HT) powerfully regulate associative learning1-8. Similarities in the activity and connectivity of these neuromodulatory systems have inspired competing models of how DA and 5HT interact to drive the formation of new associations9-14. However, these hypotheses have not been tested directly because it has not been possible to interrogate and manipulate multiple neuromodulatory systems in a single subject. Here we establish a mouse model that enables simultaneous genetic access to the brain's DA and 5HT neurons. Anterograde tracing revealed the nucleus accumbens (NAc) to be a putative hotspot for the integration of convergent DA and 5HT signals. Simultaneous recording of DA and 5HT axon activity, together with genetically encoded DA and 5HT sensor recordings, revealed that rewards increase DA signalling and decrease 5HT signalling in the NAc. Optogenetically dampening DA or 5HT reward responses individually produced modest behavioural deficits in an appetitive conditioning task, while blunting both signals together profoundly disrupted learning and reinforcement. Optogenetically reproducing DA and 5HT reward responses together was sufficient to drive the acquisition of new associations and supported reinforcement more potently than either manipulation did alone. Together, these results demonstrate that striatal DA and 5HT signals shape learning by exerting opponent control of reinforcement.

Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning

Although the cerebellum is typically associated with supervised learning algorithms, it also exhibits extensive involvement in reward processing. In this study, we investigated the cerebellum's role in executing reinforcement learning algorithms, with a particular emphasis on essential reward-prediction errors. We employed the Q-learning model to accurately reproduce the licking responses of mice in a Go/No-go auditory-discrimination task. This method enabled the calculation of reinforcement learning variables, such as reward, predicted reward, and reward-prediction errors in each learning trial. Through tensor component analysis of two-photon Ca2+ imaging data from more than 6,000 Purkinje cells, we found that climbing fiber inputs of the two distinct components, which were specifically activated during Go and No-go cues in the learning process, showed an inverse relationship with predictive reward-prediction errors. Assuming bidirectional parallel-fiber Purkinje-cell synaptic plasticity, we constructed a cerebellar neural-network model with 5,000 spiking neurons of granule cells, Purkinje cells, cerebellar nuclei neurons, and inferior olive neurons. The network model qualitatively reproduced distinct changes in licking behaviors, climbing-fiber firing rates, and their synchronization during discrimination learning separately for Go/No-go conditions. We found that Purkinje cells in the two components could develop specific motor commands for their respective auditory cues, guided by the predictive reward-prediction errors from their climbing fiber inputs. These results indicate a possible role of context-specific actors in modular reinforcement learning, integrating with cerebellar supervised learning capabilities.

The neural basis of the insight memory advantage

Creative problem solving and memory are inherently intertwined: memory accesses existing knowledge while creativity enhances it. Recent studies show that insights often accompanying creative solutions enhance long-term memory. This insight memory advantage (IMA) is explained by the 'insight as prediction error (PE)' hypothesis which states that insights arise from PEs updating predictive solution models and thereby enhancing memory. Neurally, the hippocampus initially detects PEs and then, together with the medial prefrontal cortex (mPFC), integrates and updates these expectations facilitating efficient memory encoding and retrieval. Dopamine (DA) mediates reward PEs and long-term potentiation (LTP) in the hippocampus, while noradrenaline (NE) enhances arousal and attention impacting the amygdala, the salience network, and hippocampal plasticity. These neurobiological mechanisms likely underpin IMA and have significant implications for educational practices and problem-solving strategies.