A neural substrate of prediction and reward

Neuroimaging of the effects of drug exposure or self-administration in rodents: A systematic review

A systematic review of functional neuroimaging studies on drug (self-) administration in rodents is lacking. Here, we summarized effects of acute or chronic drug administration of various classes of drugs on brain function and determined consistency with human literature. We performed a systematic literature search and identified 125 studies on in vivo rodent resting-state functional magnetic resonance imaging (n = 84) or positron emission tomography (n = 41) spanning depressants (n = 27), opioids (n = 23), stimulants (n = 72), and cannabis (n = 3). Results primarily showed alterations in the striatum, consistent with the human literature. The anterior cingulate cortex and (nonspecific) prefrontal cortex were also frequently implicated. Upregulation was most often found after shorter administration and downregulation after long chronic administration, particularly in the striatum. Importantly, results were consistent across study design, administration models, imaging method, and animal states. Results provide evidence of altered resting-state brain function in rodents upon drug administration, implicating the brain's reward network analogous to human studies. However, alterations were more dynamic than previously known, with dynamic adaptation depending on the length of drug administration.

Needing: An Active Inference Process for Physiological Motivation

Need states are internal states that arise from deprivation of crucial biological stimuli. They direct motivation, independently of external learning. Despite their separate origin, they interact with reward processing systems that respond to external stimuli. This article aims to illuminate the functioning of the needing system through the lens of active inference, a framework for understanding brain and cognition. We propose that need states exert a pervasive influence on the organism, which in active inference terms translates to a "pervasive surprise"-a measure of the distance from the organism's preferred state. Crucially, we define needing as an active inference process that seeks to reduce this pervasive surprise. Through a series of simulations, we demonstrate that our proposal successfully captures key aspects of the phenomenology and neurobiology of needing. We show that as need states increase, the tendency to occupy preferred states strengthens, independently of external reward prediction. Furthermore, need states increase the precision of states (stimuli and actions) leading to preferred states, suggesting their ability to amplify the value of reward cues and rewards themselves. Collectively, our model and simulations provide valuable insights into the directional and underlying influence of need states, revealing how this influence amplifies the wanting or liking associated with relevant stimuli.

Exploring undiscovered public knowledge in neuroscience

In this essay, I argue that the combination of research synthesis and philosophical methods can fill an important methodological gap in neuroscience. While experimental research and formal modelling have seen their methods progressively increase in rigour and sophistication over the years, the task of analysing and synthesizing the vast literature reporting new results and models has lagged behind. The problem is aggravated because neuroscience has grown and expanded into a vast mosaic of related but partially independent subfields, each with their own literatures. This fragmentation not only makes it difficult to see the full picture emerging from neuroscience research but also limits progress in individual subfields. The current neuroscience literature has the perfect conditions to create what the information scientist Don Swanson called "undiscovered public knowledge"-knowledge that exists in the mutual implications of different published pieces of information but that is nonetheless undiscovered because those pieces have not been explicitly connected. Current methods for rigorous research synthesis, such as systematic reviews and meta-analyses, mostly focus on combining similar studies and are not suited for exploring undiscovered public knowledge. To that aim, they need to be adapted and supplemented. I argue that successful exploration of the hidden implications in the neuroscience literature will require the combination of these adapted research synthesis methods with philosophical methods for rigorous (and creative) analysis and synthesis.

Kappa opioid receptor availability predicts severity of anhedonia in schizophrenia

The kappa opioid receptor (KOR) and its endogenous agonist dynorphin have been implicated in multiple psychiatric conditions including psychotic disorders. We tested the hypotheses that kappa expression is elevated and associated with psychotic symptoms in schizophrenia. We measured kappa expression in unmedicated patients with schizophrenia (7 female, 6 male) and matched controls (7 female, 6 male) with positron emission tomography (PET). We also acquired a measurement of cumulative dopamine activity over the life span in the same subjects using neuromelanin sensitive MRI. We hypothesized that neuromelanin accumulation would be higher in patients than controls and that in patients there would be a positive association between KOR availability and neuromelanin accumulation. Fourteen patients and thirteen controls were enrolled. Whole brain dynamic PET imaging data using the KOR selective tracer [18F]LY245998 were acquired. Distribution volume (VT) was measured with region of interest analysis in 14 brain regions. Neuromelanin accumulation in midbrain dopaminergic nuclei was assessed in the same subjects. Positive and negative symptoms were measured by a clinical psychologist. We did not observe group level differences in KOR expression, neuromelanin accumulation or relationships of these to positive symptoms. Unexpectedly, we did observe strong positive associations between KOR expression and symptoms of anhedonia in the patients (Pearson r > 0.7, uncorrected p < 0.01 in 8 cortical brain regions). We also observed moderate associations between KOR expression and neuromelanin levels in patients. In conclusion, we did not observe a relationship between kappa and symptoms of psychosis but the observed relationship to the negative symptom of anhedonia is in line with recent work testing kappa antagonism as a therapy for anhedonia in depression.

Dopamine Release in the Nucleus Accumbens Core Encodes the General Excitatory Components of Learning

Dopamine release in the nucleus accumbens core (NAcC) is generally considered to be a proxy for phasic firing of the ventral tegmental area dopamine (VTADA) neurons. Thus, dopamine release in NAcC is hypothesized to reflect a unitary role in reward prediction error signaling. However, recent studies reveal more diverse roles of dopamine neurons, which support an emerging idea that dopamine regulates learning differently in distinct circuits. To understand whether the NAcC might regulate a unique component of learning, we recorded dopamine release in NAcC while male rats performed a backward conditioning task where a reward is followed by a neutral cue. We used this task because we can delineate different components of learning, which include sensory-specific inhibitory and general excitatory components. Furthermore, we have shown that VTADA neurons are necessary for both the specific and general components of backward associations. Here, we found that dopamine release in NAcC increased to the reward across learning while reducing to the cue that followed as it became more expected. This mirrors the dopamine prediction error signal seen during forward conditioning and cannot be accounted for temporal-difference reinforcement learning. Subsequent tests allowed us to dissociate these learning components and revealed that dopamine release in NAcC reflects the general excitatory component of backward associations, but not their sensory-specific component. These results emphasize the importance of examining distinct functions of different dopamine projections in reinforcement learning.

Conditioning- and reward-related dendritic and presynaptic plasticity of nucleus accumbens neurons in male and female sign-tracker rats

For a subset of individuals known as sign-trackers, discrete Pavlovian cues associated with rewarding stimuli can acquire incentive properties and exert control over behaviour. Because responsiveness to cues is a feature of various neuropsychiatric conditions, rodent models of sign-tracking may prove useful for exploring the neurobiology of individual variation in psychiatric vulnerabilities. Converging evidence points towards the involvement of dopaminergic neurotransmission in the nucleus accumbens core (NAc) in the development of sign-tracking, yet whether this phenotype is associated with specific accumbal postsynaptic properties is unknown. Here, we examined dendritic spine structural organisation, as well as presynaptic and postsynaptic markers of activity, in the NAc core of male and female rats following a Pavlovian-conditioned approach procedure. In contrast to our prediction that cue re-exposure would increase spine density, experiencing the discrete lever-cue without reward delivery resulted in lower spine density than control rats for which the lever was unpaired with reward during training; this effect was tempered in the most robust sign-trackers. Interestingly, this same behavioural test (lever presentation without reward) resulted in increased levels of a marker of presynaptic activity (synaptophysin), and this effect was greatest in female rats. Whilst some behavioural differences were observed in females during initial Pavlovian training, final conditioning scores did not differ from males and were unaffected by the oestrous cycle. This work provides novel insights into how conditioning impacts the neuronal plasticity of the NAc core, whilst highlighting the importance of studying the behaviour and neurobiology of both male and female rats.

Distinct dynamics and intrinsic properties in ventral tegmental area populations mediate reward association and motivation

Ventral tegmental area (VTA) dopamine neurons regulate reward-related associative learning and reward-driven motivated behaviors, but how these processes are coordinated by distinct VTA neuronal subpopulations remains unresolved. Here, we compare the contribution of two primarily dopaminergic and largely non-overlapping VTA subpopulations, all VTA dopamine neurons and VTA GABAergic neurons of the mouse midbrain, to these processes. We find that the dopamine subpopulation that projects to the nucleus accumbens (NAc) core preferentially encodes reward-predictive cues and prediction errors. In contrast, the subpopulation that projects to the NAc shell preferentially encodes goal-directed actions and relative reward anticipation. VTA GABA neuron activity strongly contrasts VTA dopamine population activity and preferentially encodes reward outcome and retrieval. Electrophysiology, targeted optogenetics, and whole-brain input mapping reveal multiple convergent sources that contribute to the heterogeneity among VTA dopamine subpopulations that likely underlies their distinct encoding of reward-related associations and motivation that defines their functions in these contexts.

Association of Unc-51-like Kinase 4 (ULK4) with the reactivity of the extended reward system in response to conditioned stimuli

OBJECTIVES

ULK4 is an established candidate gene for mental disorders and antipsychotic treatment response. We investigated the association of functional genetic variation at the ULK4 locus with the human extended dopaminergic reward system using fMRI during the performance of a well-established reward paradigm.

METHODS

Two hundred and thirty-four patients were included in this study. Association of genetic variation in the ULK4 gene with reward system functioning were determined using the Desire-Reason-Dilemma (DRD) paradigm which allows to assess brain activation in response to conditioned reward stimuli.

RESULTS

Variant prioritisation revealed the strongest functional signatures for the ULK4 variant rs17215589, coding for amino acid exchange Ala715Thr. For rs17215589 minor allele carriers, we detected increased activation responses to conditioned reward stimuli in the ventral tegmental area, nucleus accumbens and several cortical brain regions of the extended reward system.

CONCLUSIONS

Our findings provide further evidence in humans that genetic variation in ULK4 may increase the vulnerability to mental disorders, by modulating the extended reward system function. Future studies are needed to confirm the modulation of the extended reward system by ULK4 and to specify the role of this mechanism in the pathogenesis of psychiatric disorders.

Opto-seq reveals input-specific immediate-early gene induction in ventral tegmental area cell types

The ventral tegmental area (VTA) is a critical node in circuits governing motivated behavior and is home to diverse populations of neurons that release dopamine, gamma-aminobutyric acid (GABA), glutamate, or combinations of these neurotransmitters. The VTA receives inputs from many brain regions, but a comprehensive understanding of input-specific activation of VTA neuronal subpopulations is lacking. To address this, we combined optogenetic stimulation of select VTA inputs with single-nucleus RNA sequencing (snRNA-seq) and highly multiplexed in situ hybridization to identify distinct neuronal clusters and characterize their spatial distribution and activation patterns. Quantification of immediate-early gene (IEG) expression revealed that different inputs activated select VTA subpopulations, which demonstrated cell-type-specific transcriptional programs. Within dopaminergic subpopulations, IEG induction levels correlated with differential expression of ion channel genes. This new transcriptomics-guided circuit analysis reveals the diversity of VTA activation driven by distinct inputs and provides a resource for future analysis of VTA cell types.

Distributed midbrain responses signal the content of positive identity prediction errors

Recent work across species has shown that midbrain dopamine neurons signal not only errors in the prediction of reward value but also in the prediction of value-neutral sensory features. To support learning of associative structures in downstream areas, identity prediction errors (iPEs) should signal specific information about the mis-predicted outcome. Here, we used pattern-based analysis of functional magnetic resonance imaging (fMRI) data acquired during reversal learning to characterize the information content of iPE responses in the human midbrain. We find that fMRI responses to value-neutral identity errors contain information about the identity of the unexpectedly received reward (positive iPE+) but not about the identity of the omitted reward (negative iPE-). Exploratory analyses revealed representations of iPE- in the dorsomedial prefrontal cortex. These results demonstrate that ensemble midbrain responses to value-neutral identity errors convey information about the identity of unexpectedly received outcomes, which could shape the formation of novel stimulus-outcome associations that constitute cognitive maps.

A Bio-Inspired Dopamine Model for Robots with Autonomous Decision-Making

Decision-making systems allow artificial agents to adapt their behaviours, depending on the information they perceive from the environment and internal processes. Human beings possess unique decision-making capabilities, adapting to current situations and anticipating future challenges. Autonomous robots with adaptive and anticipatory decision-making emulating humans can bring robots with skills that users can understand more easily. Human decisions highly depend on dopamine, a brain substance that regulates motivation and reward, acknowledging positive and negative situations. Considering recent neuroscience studies about the dopamine role in the human brain and its influence on decision-making and motivated behaviour, this paper proposes a model based on how dopamine drives human motivation and decision-making. The model allows robots to behave autonomously in dynamic environments, learning the best action selection strategy and anticipating future rewards. The results show the model's performance in five scenarios, emphasising how dopamine levels vary depending on the robot's situation and stimuli perception. Moreover, we show the model's integration into the Mini social robot to provide insights into how dopamine levels drive motivated autonomous behaviour regulating biologically inspired internal processes emulated in the robot.

Spontaneous eye blinks in horses (Equus caballus) are modulated by attention

Spontaneous eye blinks are brief closures of both eyelids. The spontaneous eye blink rate (SEBR) exceeds physiological corneal needs and is modulated by emotions and cognitive states, including vigilance and attention, in humans. In several animal species, the SEBR is modulated by stress and antipredator vigilance, which may limit the loss of visual information due to spontaneous eye closing. Here, we investigated whether the SEBR is modulated by attention in the domestic horse (Equus caballus). Our data supported previous studies indicating a tonic SEBR specific to each individual. We also found that, superimposed on a tonic SEBR, phasic changes were induced by cognitive processing. Attention downmodulated the SEBR, with the magnitude of blink inhibition proportional to the degree of attentional selectivity. On the other hand, reward anticipation upregulated the SEBR. Our data also suggested that horses possess the cognitive property of object permanence: they understand that an object that is no longer in their visual field has not ceased to exist. In conclusion, our results suggested that spontaneous eye blinks in horses are modulated by attentional cognitive processing.

Temporal regularities shape perceptual decisions and striatal dopamine signals

Perceptual decisions should depend on sensory evidence. However, such decisions are also influenced by past choices and outcomes. These choice history biases may reflect advantageous strategies to exploit temporal regularities of natural environments. However, it is unclear whether and how observers can adapt their choice history biases to different temporal regularities, to exploit the multitude of temporal correlations that exist in nature. Here, we show that male mice adapt their perceptual choice history biases to different temporal regularities of visual stimuli. This adaptation was slow, evolving over hundreds of trials across several days. It occurred alongside a fast non-adaptive choice history bias, limited to a few trials. Both fast and slow trial history effects are well captured by a normative reinforcement learning algorithm with multi-trial belief states, comprising both current trial sensory and previous trial memory states. We demonstrate that dorsal striatal dopamine tracks predictions of the model and behavior, suggesting that striatal dopamine reports reward predictions associated with adaptive choice history biases. Our results reveal the adaptive nature of perceptual choice history biases and shed light on their underlying computational principles and neural correlates.

Neural response to monetary and social rewards in adolescent girls and their parents

Functional magnetic resonance imaging (fMRI) studies have indicated that the mesocorticolimbic dopamine system is heavily involved in all stages of reward processing. However, the majority of research has been conducted using monetary rewards and it is unclear to what extent other types of rewards, such as social rewards, evoke similar or different neural activation. There have also been few investigations into potential differences or similarities between reward processing in parents and offspring. The present study examined fMRI neural activation in response to monetary and social reward in a sample of 14-22-year-old adolescent girls (N = 145) and a biological parent (N = 124) and compared activation across adolescent-parent dyads (N = 82). Across all participants, both monetary and social reward elicited bilateral striatal activation, which did not differ between reward types or between adolescents and their parents. Neural activation in response to the different reward types were positively correlated in the striatum among adolescents and in the mPFC and OFC among parents. Overall, the present study suggests that both monetary and social reward elicit striatal activation regardless of age and provides evidence that neural mechanisms underlying reward processing may converge differentially among youth and adults.

Frontal Norepinephrine Represents a Threat Prediction Error Under Uncertainty

BACKGROUND

To adapt to threats in the environment, animals must predict them and engage in defensive behavior. While the representation of a prediction error signal for reward has been linked to dopamine, a neuromodulatory prediction error for aversive learning has not been identified.

METHODS

We measured and manipulated norepinephrine release during threat learning using optogenetics and a novel fluorescent norepinephrine sensor.

RESULTS

We found that norepinephrine response to conditioned stimuli reflects aversive memory strength. When delays between auditory stimuli and footshock are introduced, norepinephrine acts as a prediction error signal. However, temporal difference prediction errors do not fully explain norepinephrine dynamics. To explain noradrenergic signaling, we used an updated reinforcement learning model with uncertainty about time and found that it explained norepinephrine dynamics across learning and variations in temporal and auditory task structure.

CONCLUSIONS

Norepinephrine thus combines cognitive and affective information into a predictive signal and links time with the anticipation of danger.

Cortico-basal ganglia plasticity in motor learning

One key function of the brain is to control our body's movements, allowing us to interact with the world around us. Yet, many motor behaviors are not innate but require learning through repeated practice. Among the brain's motor regions, the cortico-basal ganglia circuit is particularly crucial for acquiring and executing motor skills, and neuronal activity in these regions is directly linked to movement parameters. Cell-type-specific adaptations of activity patterns and synaptic connectivity support the learning of new motor skills. Functionally, neuronal activity sequences become structured and associated with learned movements. On the synaptic level, specific connections become potentiated during learning through mechanisms such as long-term synaptic plasticity and dendritic spine dynamics, which are thought to mediate functional circuit plasticity. These synaptic and circuit adaptations within the cortico-basal ganglia circuitry are thus critical for motor skill acquisition, and disruptions in this plasticity can contribute to movement disorders.

Test-retest reliability of reinforcement learning parameters

It has recently been suggested that parameter estimates of computational models can be used to understand individual differences at the process level. One area of research in which this approach, called computational phenotyping, has taken hold is computational psychiatry. One requirement for successful computational phenotyping is that behavior and parameters are stable over time. Surprisingly, the test-retest reliability of behavior and model parameters remains unknown for most experimental tasks and models. The present study seeks to close this gap by investigating the test-retest reliability of canonical reinforcement learning models in the context of two often-used learning paradigms: a two-armed bandit and a reversal learning task. We tested independent cohorts for the two tasks (N = 69 and N = 47) via an online testing platform with a between-test interval of five weeks. Whereas reliability was high for personality and cognitive measures (with ICCs ranging from .67 to .93), it was generally poor for the parameter estimates of the reinforcement learning models (with ICCs ranging from .02 to .52 for the bandit task and from .01 to .71 for the reversal learning task). Given that simulations indicated that our procedures could detect high test-retest reliability, this suggests that a significant proportion of the variability must be ascribed to the participants themselves. In support of that hypothesis, we show that mood (stress and happiness) can partly explain within-participant variability. Taken together, these results are critical for current practices in computational phenotyping and suggest that individual variability should be taken into account in the future development of the field.

The Role of the Cerebellum in Learning to Predict Reward: Evidence from Cerebellar Ataxia

Recent findings in animals have challenged the traditional view of the cerebellum solely as the site of motor control, suggesting that the cerebellum may also be important for learning to predict reward from trial-and-error feedback. Yet, evidence for the role of the cerebellum in reward learning in humans is lacking. Moreover, open questions remain about which specific aspects of reward learning the cerebellum may contribute to. Here we address this gap through an investigation of multiple forms of reward learning in individuals with cerebellum dysfunction, represented by cerebellar ataxia cases. Nineteen participants with cerebellar ataxia and 57 age- and sex-matched healthy controls completed two separate tasks that required learning about reward contingencies from trial-and-error. To probe the selectivity of reward learning processes, the tasks differed in their underlying structure: while one task measured incremental reward learning ability alone, the other allowed participants to use an alternative learning strategy based on episodic memory alongside incremental reward learning. We found that individuals with cerebellar ataxia were profoundly impaired at reward learning from trial-and-error feedback on both tasks, but retained the ability to learn to predict reward based on episodic memory. These findings provide evidence from humans for a specific and necessary role for the cerebellum in incremental learning of reward associations based on reinforcement. More broadly, the findings suggest that alongside its role in motor learning, the cerebellum likely operates in concert with the basal ganglia to support reinforcement learning from reward.

Brain Networks, Neurotransmitters and Psychedelics: Towards a Neurochemistry of Self-Awareness

PURPOSE OF REVIEW

Self-awareness can be defined as the capacity of becoming the object of one's own awareness and, increasingly, it has been the target of scientific inquiry. Self-awareness has important clinical implications, and a better understanding of the neurochemical basis of self-awareness may help clarifying causes and developing interventions for different psychopathological conditions. The current article explores the relationship between neurochemistry and self-awareness, with special attention to the effects of psychedelics.

RECENT FINDINGS

The functioning of self-related networks, such as the default-mode network and the salience network, and how these are influenced by different neurotransmitters is discussed. The impact of psychedelics on self-awareness is reviewed in relation to specific processes, such as interoception, body ownership, agency, metacognition, emotional regulation and autobiographical memory, within a framework based on predictive coding. Improved outcomes in emotional regulation and autobiographical memory have been observed in association with the use of psychedelics, suggesting higher-order self-awareness changes, which can be modulated by relaxation of priors and improved coping mechanisms linked to cognitive flexibility. Alterations in bodily self-awareness are less consistent, being potentially impacted by doses employed, differences in acute/long-term effects and the presence of clinical conditions. Future studies investigating the effects of different molecules in rebalancing connectivity between resting-state networks may lead to novel therapeutic approaches and the refinement of existing treatments.

Predictive Processing: A Circuit Approach to Psychosis

Predictive processing is a computational framework that aims to explain how the brain processes sensory information by making predictions about the environment and minimizing prediction errors. It can also be used to explain some of the key symptoms of psychotic disorders such as schizophrenia. In recent years, substantial advances have been made in our understanding of the neuronal circuitry that underlies predictive processing in cortex. In this review, we summarize these findings and how they might relate to psychosis and to observed cell type-specific effects of antipsychotic drugs. We argue that quantifying the effects of antipsychotic drugs on specific neuronal circuit elements is a promising approach to understanding not only the mechanism of action of antipsychotic drugs but also psychosis. Finally, we outline some of the key experiments that should be done. The aims of this review are to provide an overview of the current circuit-based approaches to psychosis and to encourage further research in this direction.

The neuroscience of active learning and direct instruction

Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.

Fundamental processes in sensorimotor learning: Reasoning, refinement, and retrieval

Motor learning is often viewed as a unitary process that operates outside of conscious awareness. This perspective has led to the development of sophisticated models designed to elucidate the mechanisms of implicit sensorimotor learning. In this review, we argue for a broader perspective, emphasizing the contribution of explicit strategies to sensorimotor learning tasks. Furthermore, we propose a theoretical framework for motor learning that consists of three fundamental processes: reasoning, the process of understanding action-outcome relationships; refinement, the process of optimizing sensorimotor and cognitive parameters to achieve motor goals; and retrieval, the process of inferring the context and recalling a control policy. We anticipate that this '3R' framework for understanding how complex movements are learned will open exciting avenues for future research at the intersection between cognition and action.

Disruption of positive- and negative-feature morphine interoceptive occasion setters by dopamine receptor agonism and antagonism in male and female rats

RATIONALE

Internally perceived stimuli evoked by morphine administration can form Pavlovian associations such that they can function as occasion setters (OSs) for externally perceived reward cues in rats, coming to modulate reward-seeking behaviour. Though much research has investigated mechanisms underlying opioid-related reinforcement and analgesia, neurotransmitter systems involved in the functioning of opioids as Pavlovian interoceptive discriminative stimuli remain to be disentangled despite documented differences in the development of tolerance to analgesic versus discriminative stimulus effects.

OBJECTIVES

Dopamine has been implicated in many opioid-related behaviours, so we aimed to investigate the role of this neurotransmitter in expression of morphine occasion setting.

METHODS

Male and female rats were assigned to positive- (FP) or negative-feature (FN) groups and received an injection of morphine or saline before each training session. A 15-s white noise conditioned stimulus (CS) was presented 8 times during every training session; offset of this stimulus was followed by 4-s access to liquid sucrose on morphine, but not saline, sessions for FP rats. FN rats learned the reverse contingency. Following stable discrimination, rats began generalization testing for expression of morphine-guided sucrose seeking after systemic pretreatment with different doses of the non-selective dopamine receptor antagonist, flupenthixol, and the non-selective dopamine receptor agonist, apomorphine, combined with training doses of morphine or saline in a Latin-square design.

RESULTS

The morphine discrimination was acquired under both FP and FN contingencies by males and females. Neither flupenthixol nor apomorphine at any dose substituted for morphine, but both apomorphine and flupenthixol disrupted expression of the morphine OS. This inhibition was specific to sucrose seeking during CS presentations rather than during the period before CS onset and, in the case of apomorphine more so than flupenthixol, to trials on which access to sucrose was anticipated.

CONCLUSIONS

Our findings lend support to a mechanism of occasion setting involving gating of CS-induced dopamine release rather than by direct dopaminergic modulation by the morphine stimulus.

Learning and memory processes in behavioural addiction: A systematic review

Similar to addictive substances, addictive behaviours such as gambling and gaming are associated with maladaptive modulation of key brain areas and functional networks implicated in learning and memory. Therefore, this review sought to understand how different learning and memory processes relate to behavioural addictions and to unravel their underlying neural mechanisms. Adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we systematically searched four databases - PsycINFO, PubMed, Scopus, and Web of Science using the agreed-upon search string. Findings suggest altered executive function-dependent learning processes and enhanced habit learning in behavioural addiction. Whereas the relationship between working memory and behavioural addiction is influenced by addiction type, working memory aspect, and task nature. Additionally, long-term memory is incoherent in individuals with addictive behaviours. Consistently, neurophysiological evidence indicates alterations in brain areas and networks implicated in learning and memory processes in behavioural addictions. Overall, the present review argues that, like substance use disorders, alteration in learning and memory processes may underlie the development and maintenance of behavioural addictions.

When the interoceptive and conceptual clash: The case of oppositional phenomenal self-modelling in Tourette syndrome

Tourette syndrome (TS) has been associated with a rich set of symptoms that are said to be uncomfortable, unwilled, and effortful to manage. Furthermore, tics, the canonical characteristic of TS, are multifaceted, and their onset and maintenance is complex. A formal account that integrates these features of TS symptomatology within a plausible theoretical framework is currently absent from the field. In this paper, we assess the explanatory power of hierarchical generative modelling in accounting for TS symptomatology from the perspective of active inference. We propose a fourfold analysis of sensory, motor, and cognitive phenomena associated with TS. In Section 1, we characterise tics as a form of action aimed at sensory attenuation. In Section 2, we introduce the notion of epistemic ticcing and describe such behaviour as the search for evidence that there is an agent (i.e., self) at the heart of the generative hierarchy. In Section 3, we characterise both epistemic (sensation-free) and nonepistemic (sensational) tics as habitual behaviour. Finally, in Section 4, we propose that ticcing behaviour involves an inevitable conflict between distinguishable aspects of selfhood; namely, between the minimal phenomenal sense of self-which is putatively underwritten by interoceptive inference-and the explicit preferences that constitute the individual's conceptual sense of self. In sum, we aim to provide an empirically informed analysis of TS symptomatology under active inference, revealing a continuity between covert and overt features of the condition.

A feature-specific prediction error model explains dopaminergic heterogeneity

The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary 'feature-specific RPE' model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal's moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.

Regulation of reinforcement learning parameters captures long-term changes in rat behaviour

In uncertain environments in which resources fluctuate continuously, animals must permanently decide whether to stabilise learning and exploit what they currently believe to be their best option, or instead explore potential alternatives and learn fast from new observations. While such a trade-off has been extensively studied in pretrained animals facing non-stationary decision-making tasks, it is yet unknown how they progressively tune it while learning the task structure during pretraining. Here, we compared the ability of different computational models to account for long-term changes in the behaviour of 24 rats while they learned to choose a rewarded lever in a three-armed bandit task across 24 days of pretraining. We found that the day-by-day evolution of rat performance and win-shift tendency revealed a progressive stabilisation of the way they regulated reinforcement learning parameters. We successfully captured these behavioural adaptations using a meta-learning model in which either the learning rate or the inverse temperature was controlled by the average reward rate.

Striatal GABA levels correlate with risk sensitivity in monetary loss

Background

Decision-making under risk is a common challenge. It is known that risk-taking behavior varies between contexts of reward and punishment, yet the mechanisms underlying this asymmetry in risk sensitivity remain unclear.

Methods

This study used a monetary task to investigate neurochemical mechanisms and brain dynamics underpinning risk sensitivity. Twenty-eight participants engaged in a task requiring selection of visual stimuli to maximize monetary gains and minimize monetary losses. We modeled participant trial-and-error processes using reinforcement learning.

Results

Participants with higher subjective utility parameters showed risk preference in the gain domain (r = -0.59) and risk avoidance in the loss domain (r = -0.77). Magnetic resonance spectroscopy (MRS) revealed that risk avoidance in the loss domain was associated with γ-aminobutyric acid (GABA) levels in the ventral striatum (r = -0.42), but not in the insula (r = -0.15). Using functional magnetic resonance imaging (fMRI), we tested whether risk-sensitive brain dynamics contribute to participant risky choices. Energy landscape analyses demonstrated that higher switching rates between brain states, including the striatum and insula, were correlated with risk avoidance in the loss domain (r = -0.59), a relationship not observed in the gain domain (r = -0.02).

Conclusions

These findings from MRS and fMRI suggest that distinct mechanisms are involved in gain/loss decision making, mediated by subcortical neurometabolite levels and brain dynamic transitions.

Synapses tagged, memories kept: synaptic tagging and capture hypothesis in brain health and disease

The synaptic tagging and capture (STC) hypothesis lays the framework on the synapse-specific mechanism of protein synthesis-dependent long-term plasticity upon synaptic induction. Activated synapses will display a transient tag that will capture plasticity-related products (PRPs). These two events, tag setting and PRP synthesis, can be teased apart and have been studied extensively-from their electrophysiological and pharmacological properties to the molecular events involved. Consequently, the hypothesis also permits interactions of synaptic populations that encode different memories within the same neuronal population-hence, it gives rise to the associativity of plasticity. In this review, the recent advances and progress since the experimental debut of the STC hypothesis will be shared. This includes the role of neuromodulation in PRP synthesis and tag integrity, behavioural correlates of the hypothesis and modelling in silico. STC, as a more sensitive assay for synaptic health, can also assess neuronal aberrations. We will also expound how synaptic plasticity and associativity are altered in ageing-related decline and pathological conditions such as juvenile stress, cancer, sleep deprivation and Alzheimer's disease. This article is part of a discussion meeting issue 'Long-term potentiation: 50 years on'.

D2 dopamine receptor expression, reactivity to rewards, and reinforcement learning in a complex value-based decision-making task

Different dopamine (DA) subtypes have opposing dynamics at postsynaptic receptors, with the ratio of D1 to D2 receptors determining the relative sensitivity to gains and losses, respectively, during value-based learning. This effective sensitivity to different reward feedback interacts with phasic DA levels to determine the effectiveness of learning, particularly in dynamic feedback situations where the frequency and magnitude of rewards need to be integrated over time to make optimal decisions. We modeled this effect in simulations of the underlying basal ganglia pathways and then tested the predictions in individuals with a variant of the human dopamine receptor D2 (DRD2; -141C Ins/Del and Del/Del) gene that associates with lower levels of D2 receptor expression (N = 119) and compared their performance in the Iowa Gambling Task to noncarrier controls (N = 319). Ventral striatal (VS) reactivity to rewards was measured in the Cards task with fMRI. DRD2 variant carriers made less effective decisions than noncarriers, but this effect was not moderated by VS reward reactivity as is hypothesized by our model. These results suggest that the interaction between DA receptor subtypes and reactivity to rewards during learning may be more complex than originally thought.

Voluntary wheel running access produces opposite effects in male and female rats on both palatable diet consumption and associated ventral striatal opioid- and dopamine-related gene expression

The relationship between physical activity levels and feeding behaviors has been a focus of preclinical research for decades, yet this interaction has only recently been explored for potential sex differences. The aim of the present study was to isolate sex-dependent effects of voluntary wheel running (RUN) vs. sedentary locked wheel (SED) home cage conditions on palatability-driven feeding behavior using a 2-diet choice task between standard chow and a high-fat diet. The sex-dependent effects of physical activity on feeding behavior were examined following a within-subject novel reversal design of physical activity conditions (i.e., RUN > SED > RUN), to assess temporal sensitivity of the interaction. Following the final 2 weeks of reestablished and sustained RUN vs. SED conditions in separate groups of both males and females, reward-related opioid and dopamine gene expression within the nucleus accumbens (Acb) brain region were analyzed. Results demonstrated that the initial RUN > SED transition led to sex-dependent effects of SED condition, as males increased, and females decreased their high fat consumption, compared to their respective high fat consumption during previous RUN condition phase. Following reintroduction to the RUN condition, males decreased, and females increased their high fat consumption, compared to their separate SED control group. Last, sex-dependent shifts in ventral striatal opioid- and dopamine-related gene expression were observed to parallel the behavioral effects. The major findings of the study reveal that SED and RUN home cage conditions shift palatability-driven feeding in the opposite direction for males and females, these effects are sensitive to reversal, and these sex-dependent feeding behaviors track sex-dependent changes to critical reward-related gene expression patterns in the Acb. Considering the present high rates of sedentary behavior and obesity, furthering our understanding of the interaction between physical activity (or lack thereof) and feeding behavior should be a priority, especially in the context of these divergent sex-dependent outcomes.

Explaining dopamine through prediction errors and beyond

The most influential account of phasic dopamine holds that it reports reward prediction errors (RPEs). The RPE-based interpretation of dopamine signaling is, in its original form, probably too simple and fails to explain all the properties of phasic dopamine observed in behaving animals. This Perspective helps to resolve some of the conflicting interpretations of dopamine that currently exist in the literature. We focus on the following three empirical challenges to the RPE theory of dopamine: why does dopamine (1) ramp up as animals approach rewards, (2) respond to sensory and motor features and (3) influence action selection? We argue that the prediction error concept, once it has been suitably modified and generalized based on an analysis of each computational problem, answers each challenge. Nonetheless, there are a number of additional empirical findings that appear to demand fundamentally different theoretical explanations beyond encoding RPE. Therefore, looking forward, we discuss the prospects for a unifying theory that respects the diversity of dopamine signaling and function as well as the complex circuitry that both underlies and responds to dopaminergic transmission.

Insights into conscious cognitive information processing

For over a century, the neuro- and pathophysiological, behavioral, and cognitive correlates of consciousness have been an active field of theoretical considerations and empirical research in a wide range of modern disciplines. Conscious cognitive processing of information cannot be observed directly, but might be inferred from step-like discontinuities in learning performance or sudden insight-based improvements in problem solving behavior. It is assumed that a sudden step of knowledge associated with insight requires a creative reorganization of mental representations of task- or problem-relevant information and the restructuration of the task, respectively problem to overcome an cognitive dead-end or impasse. Discontinuities in learning performance or problem solving after an insight event can be used as time-tags to capture the time window in which conscious cognitive information processing must have taken place. According to the platform theory of conscious cognitive information processing, the reorganization and restructuration processes, require the maintenance of task- or problem-relevant information in working memory for the operation of executive functions on these mental representations. Electrophysiological evidence suggests that the reorganization and restructuration processes in working memory, that precede insight-based problem solutions are accompanied by an increase in the power of gamma oscillations in cortical areas including the prefrontal cortex. Empirical evidence and theoretical assumptions argue for an involvement of gap junction channels and connexin hemichannels in cortical gamma-oscillations and working memory processes. Discontinuities in learning or problem solving performance might be used as time-tags to investigate the implication of gap junction channels and hemichannels in conscious cognitive processing.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.

Transformation of valence signaling in a striatopallidal circuit

The ways in which sensory stimuli acquire motivational valence through association with other stimuli is one of the simplest forms of learning. Though we have identified many brain nuclei that play various roles in reward processing, a significant gap remains in understanding how valence encoding transforms through the layers of sensory processing. To address this gap, we carried out a comparative investigation of the anteromedial olfactory tubercle (OT), and the ventral pallidum (VP) - 2 connected nuclei of the basal ganglia which have both been implicated in reward processing. First, using anterograde and retrograde tracing, we show that both D1 and D2 neurons of the anteromedial OT project primarily to the VP and minimally elsewhere. Using 2-photon calcium imaging, we then investigated how the identity of the odor and reward contingency of the odor are differently encoded by neurons in either structure during a classical conditioning paradigm. We find that VP neurons robustly encode reward contingency, but not identity, in low-dimensional space. In contrast, the OT neurons primarily encode odor identity in high-dimensional space. Although D1 OT neurons showed larger responses to rewarded odors than other odors, consistent with prior findings, we interpret this as identity encoding with enhanced contrast. Finally, using a novel conditioning paradigm that decouples reward contingency and licking vigor, we show that both features are encoded by non-overlapping VP neurons. These results provide a novel framework for the striatopallidal circuit in which a high-dimensional encoding of stimulus identity is collapsed onto a low-dimensional encoding of motivational valence.

Spatially organized striatum-wide acetylcholine dynamics for the learning and extinction of Pavlovian cues and actions

Striatal acetylcholine (ACh) has been linked to behavioral flexibility. A key component of flexibility is down-regulating responding as valued cues and actions become decoupled from positive outcomes. We used array fiber photometry in mice to investigate how ACh release across the striatum evolves during learning and extinction of Pavlovian associations. Changes in multi-phasic release to cues and consummatory actions were bi-directional and region-specific. Following extinction, increases in cue-evoked ACh release emerged in the anterior dorsal striatum (aDS) which preceded a down-regulation of anticipatory behavior. Silencing ACh release from cholinergic interneurons in the aDS blocked behavioral extinction. Dopamine release dipped below baseline for down-shifted cues, but glutamate input onto cholinergic interneurons did not change, suggesting an intrastriatal mechanism for the emergence of ACh increases. Our large-scale mapping of striatal ACh dynamics during learning pinpoints region-specific elevations in ACh release positioned to down-regulate behavior during extinction, a central feature of flexible behavior.

Natural phasic inhibition of dopamine neurons signals cognitive rigidity

When animals unexpectedly fail, their dopamine neurons undergo phasic inhibition that canonically drives extinction learning-a cognitive-flexibility mechanism for discarding outdated strategies. However, the existing evidence equates natural and artificial phasic inhibition, despite their spatiotemporal differences. Addressing this gap, we targeted a GABAA-receptor antagonist precisely to dopamine neurons, yielding three unexpected findings. First, this intervention blocked natural phasic inhibition selectively, leaving tonic activity unaffected. Second, blocking natural phasic inhibition accelerated extinction learning-opposite to canonical mechanisms. Third, our approach selectively benefitted perseverative mice, restoring rapid extinction without affecting new reward learning. Our findings reveal that extinction learning is rapid by default and slowed by natural phasic inhibition-challenging foundational learning theories, while delineating a synaptic mechanism and therapeutic target for cognitive rigidity.

Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time

The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference learning (TD) learning, whereby certain units signal reward prediction errors (RPE). The TD algorithm has been traditionally mapped onto the dopaminergic system, as firing properties of dopamine neurons can resemble RPEs. However, certain predictions of TD learning are inconsistent with experimental results, and previous implementations of the algorithm have made unscalable assumptions regarding stimulus-specific fixed temporal bases. We propose an alternate framework to describe dopamine signaling in the brain, FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, dopamine release is similar, but not identical to RPE, leading to predictions that contrast to those of TD. While FLEX itself is a general theoretical framework, we describe a specific, biophysically plausible implementation, the results of which are consistent with a preponderance of both existing and reanalyzed experimental data.

Humans forage for reward in reinforcement learning tasks

How do we make good decisions in uncertain environments? In psychology and neuroscience, the classic answer is that we calculate the value of each option and then compare the values to choose the most rewarding, modulo some exploratory noise. An ethologist, conversely, would argue that we commit to one option until its value drops below a threshold, at which point we start exploring other options. In order to determine which view better describes human decision-making, we developed a novel, foraging-inspired sequential decision-making model and used it to ask whether humans compare to threshold ("Forage") or compare alternatives ("Reinforcement-Learn" [RL]). We found that the foraging model was a better fit for participant behavior, better predicted the participants' tendency to repeat choices, and predicted the existence of held-out participants with a pattern of choice that was almost impossible under RL. Together, these results suggest that humans use foraging computations, rather than RL, even in classic reinforcement learning tasks.

Methamphetamine-induced adaptation of learning rate dynamics depend on baseline performance

The ability to calibrate learning according to new information is a fundamental component of an organism's ability to adapt to changing conditions. Yet, the exact neural mechanisms guiding dynamic learning rate adjustments remain unclear. Catecholamines appear to play a critical role in adjusting the degree to which we use new information over time, but individuals vary widely in the manner in which they adjust to changes. Here, we studied the effects of a low dose of methamphetamine (MA), and individual differences in these effects, on probabilistic reversal learning dynamics in a within-subject, double-blind, randomized design. Participants first completed a reversal learning task during a drug-free baseline session to provide a measure of baseline performance. Then they completed the task during two sessions, one with MA (20 mg oral) and one with placebo (PL). First, we showed that, relative to PL, MA modulates the ability to dynamically adjust learning from prediction errors. Second, this effect was more pronounced in participants who performed poorly at baseline. These results present novel evidence for the involvement of catecholaminergic transmission on learning flexibility and highlights that baseline performance modulates the effect of the drug.

Heightened lateral habenula activity during stress produces brainwide and behavioral substrates of susceptibility

Some individuals are susceptible to the experience of chronic stress and others are more resilient. While many brain regions implicated in learning are dysregulated after stress, little is known about whether and how neural teaching signals during stress differ between susceptible and resilient individuals. Here, we seek to determine if activity in the lateral habenula (LHb), which encodes a negative teaching signal, differs between susceptible and resilient mice during stress to produce different outcomes. After, but not before, chronic social defeat stress (CSDS), the LHb is active when susceptible mice are in the proximity of the aggressor strain. During stress itself, LHb activity is higher in susceptible mice during aggressor proximity, and activation of the LHb during stress biases mice towards susceptibility. This manipulation generates a persistent and widespread increase in the balance of subcortical versus cortical activity in susceptible mice. Taken together, our results indicate that heightened activity in the LHb during stress produces lasting brainwide and behavioral substrates of susceptibility.

Effects of psychedelic, DOI, on nucleus accumbens dopamine signaling to predictable rewards and cues in rats

Psychedelics produce lasting therapeutic responses in neuropsychiatric diseases suggesting they may disrupt entrenched associations and catalyze learning. Here, we examine psychedelic 5-HT2A/2C agonist, DOI, effects on dopamine signaling in the nucleus accumbens (NAc) core, a region extensively linked to reward learning, motivation, and drug-seeking. We measure phasic dopamine transients following acute DOI administration in rats during well learned Pavlovian tasks in which sequential cues predict rewards. We find that DOI (0.0-1.2 mg/kg, i.p.) increases dopamine signals, photometrically measured using GRABDA optical sensor, to rewards and proximal reward cues, but not to the distal cues that predict these events. We determine that the elevated dopamine produced by DOI to reward cues occurs independently of DOI-induced changes in reward value. The increased dopamine associated with predictable reward cues and rewards supports DOI-induced increases in prediction error signaling. These findings lay a foundation for developing psychedelic strategies aimed at engaging error-driven learning mechanisms to disrupt entrenched associations or produce new associations.

Influence of reinforcement learning on the inhibitory control of Internet gaming disorder

Reward processing dysfunction and inhibition control deficiency have been observed in Internet gaming disorder (IGD). However, it is still unclear whether the previous reinforcement learning depends on reward/punishment feedback influences on the cognitive inhibitory control of IGD. This study compared the differences between an IGD group and healthy people without game experiences in the probability selection task and the subsequent stop signal task by the method of behavioral experiments, in order to explore whether the reward learning ability is impaired in the IGD group. We also discuss the influence of previous reward learning on subsequent inhibition control. The results showed that (1) during the reward learning phase, the IGD group's accuracy was significantly lower than that of the control group; (2) compared with the control group, the IGD group's reaction times were longer in the transfer phase; (3) for no-go trials of the inhibitory control phase after reward learning, the accuracy of the reward-related stimulation in the IGD group was lower than that of punishment-related or neutral stimulation, but there was no significant difference among the three conditions in the control group. These findings indicated that the reinforcement learning ability of the IGD group was impaired, which further caused the abnormal response to reinforcement stimuli.

Constraints on the subsecond modulation of striatal dynamics by physiological dopamine signaling

Dopaminergic neurons play a crucial role in associative learning, but their capacity to regulate behavior on subsecond timescales remains debated. It is thought that dopaminergic neurons drive certain behaviors by rapidly modulating striatal spiking activity; however, a view has emerged that only artificially high (that is, supra-physiological) dopamine signals alter behavior on fast timescales. This raises the possibility that moment-to-moment striatal spiking activity is not strongly shaped by dopamine signals in the physiological range. To test this, we transiently altered dopamine levels while monitoring spiking responses in the ventral striatum of behaving mice. These manipulations led to only weak changes in striatal activity, except when dopamine release exceeded reward-matched levels. These findings suggest that dopaminergic neurons normally play a minor role in the subsecond modulation of striatal dynamics in relation to other inputs and demonstrate the importance of discerning dopaminergic neuron contributions to brain function under physiological and potentially nonphysiological conditions.

A chemogenetic approach for dopamine imaging with tunable sensitivity

Genetically-encoded dopamine (DA) sensors enable high-resolution imaging of DA release, but their ability to detect a wide range of extracellular DA levels, especially tonic versus phasic DA release, is limited by their intrinsic affinity. Here we show that a human-selective dopamine receptor positive allosteric modulator (PAM) can be used to boost sensor affinity on-demand. The PAM enhances DA detection sensitivity across experimental preparations (in vitro, ex vivo and in vivo) via one-photon or two-photon imaging. In vivo photometry-based detection of optogenetically-evoked DA release revealed that DETQ administration produces a stable 31 minutes window of potentiation without effects on animal behavior. The use of the PAM revealed region-specific and metabolic state-dependent differences in tonic DA levels and enhanced single-trial detection of behavior-evoked phasic DA release in cortex and striatum. Our chemogenetic strategy can potently and flexibly tune DA imaging sensitivity and reveal multi-modal (tonic/phasic) DA signaling across preparations and imaging approaches.

In vivo structural connectivity of the reward system along the hippocampal long axis

Recent work has identified a critical role for the hippocampus in reward-sensitive behaviors, including motivated memory, reinforcement learning, and decision-making. Animal histology and human functional neuroimaging have shown that brain regions involved in reward processing and motivation are more interconnected with the ventral/anterior hippocampus. However, direct evidence examining gradients of structural connectivity between reward regions and the hippocampus in humans is lacking. The present study used diffusion MRI (dMRI) and probabilistic tractography to quantify the structural connectivity of the hippocampus with key reward processing regions in vivo. Using a large sample of subjects (N = 628) from the human connectome dMRI data release, we found that connectivity profiles with the hippocampus varied widely between different regions of the reward circuit. While the dopaminergic midbrain (ventral tegmental area) showed stronger connectivity with the anterior versus posterior hippocampus, the ventromedial prefrontal cortex showed stronger connectivity with the posterior hippocampus. The limbic (ventral) striatum demonstrated a more homogeneous connectivity profile along the hippocampal long axis. This is the first study to generate a probabilistic atlas of the hippocampal structural connectivity with reward-related networks, which is essential to investigating how these circuits contribute to normative adaptive behavior and maladaptive behaviors in psychiatric illness. These findings describe nuanced structural connectivity that sets the foundation to better understand how the hippocampus influences reward-guided behavior in humans.

Cognitive representations of intracranial self-stimulation of midbrain dopamine neurons depend on stimulation frequency

Dopamine neurons in the ventral tegmental area support intracranial self-stimulation (ICSS), yet the cognitive representations underlying this phenomenon remain unclear. Here, 20-Hz stimulation of dopamine neurons, which approximates a physiologically relevant prediction error, was not sufficient to support ICSS beyond a continuously reinforced schedule and did not endow cues with a general or specific value. However, 50-Hz stimulation of dopamine neurons was sufficient to drive robust ICSS and was represented as a specific reward to motivate behavior. The frequency dependence of this effect is due to the rate (not the number) of action potentials produced by dopamine neurons, which differently modulates dopamine release downstream.

The role of objective sleep in implicit and explicit affect regulation: A comprehensive review

Impairments in sleep and affect regulation are evident across a wide range of mental disorders. Understanding the sleep factors that relate to affect regulatory difficulties will inform mechanistic understanding and aid in treatment. Despite rising interest, some research challenges in this area include integrating across different clinical and non-clinical literatures investigating the role of sleep architecture (measured with polysomnography) and experimentally manipulated sleep, as well as integrating more explicit versus implicit affect regulation processes. In this comprehensive review, we use a unifying framework to examine sleep's relationship with implicit-automatic regulation and explicit-controlled regulation, both of which are relevant to mental health (e.g., PTSD and depression). Many studies of implicit-automatic regulation (e.g., fear extinction and safety learning) demonstrate the importance of sleep, and REM sleep specifically. Studies of explicit-controlled regulation (e.g., cognitive reappraisal and expressive suppression) are less consistent in their findings, with results differing depending on the type of affect regulation and/or way that sleep was measured or manipulated. There is a clear relationship between objective sleep and affect regulation processes. However, there is a need for 1) more studies focusing on sleep and explicit-controlled affect regulation; 2) replication with the same types of regulation strategies; 3) more studies experimentally manipulating sleep to examine its impact on affect regulation and vice versa in order to infer cause and effect; and 4) more studies looking at sleep's impact on next-day affect regulation (not just overnight change in affect reactivity).

Implication of regional selectivity of dopamine deficits in impaired suppressing of involuntary movements in Parkinson's disease

To improve the initiation and speed of intended action, one of the crucial mechanisms is suppressing unwanted movements that interfere with goal-directed behavior, which is observed relatively aberrant in Parkinson's disease patients. Recent research has highlighted that dopamine deficits in Parkinson's disease predominantly occur in the caudal lateral part of the substantia nigra pars compacta (SNc) in human patients. We previously found two parallel circuits within the basal ganglia, primarily divided into circuits mediated by the rostral medial part and caudal lateral part of the SNc dopamine neurons. We have further discovered that the indirect pathway in caudal basal ganglia circuits, facilitated by the caudal lateral part of the SNc dopamine neurons, plays a critical role in suppressing unnecessary involuntary movements when animals perform voluntary goal-directed actions. We thus explored recent research in humans and non-human primates focusing on the distinct functions and networks of the caudal lateral part of the SNc dopamine neurons to elucidate the mechanisms involved in the impairment of suppressing involuntary movements in Parkinson's disease patients.

Behavioral therapies targeting reward mechanisms in substance use disorders

Behavioral therapies are considered best practices in the treatment of substance use disorders (SUD) and used as first-line approaches for SUDs without FDA-approved pharmacotherapies. Decades of research on the neuroscience of drug reward and addiction have informed the development of current leading behavioral therapies that, while differing in focus and technique, have in common the overarching goal of shifting reward responding away from drug and toward natural non-drug rewards. This review begins by describing key neurobiological processes of reward in addiction, followed by a description of how various behavioral therapies address specific reward processes. Based on this review, a conceptual 'map' is crafted to pinpoint gaps and areas of overlap, serving as a guide for selecting and integrating behavioral therapies.