Predictive reward signal of dopamine neurons

Heterogeneity of mesencephalic dopaminergic neurons: From molecular classifications, electrophysiological properties to functional connectivity

The mesencephalic dopamine (DA) system is composed of neuronal subtypes that are molecularly and functionally distinct, are responsible for specific behaviors, and are closely associated with numerous brain disorders. Existing research has made significant advances in identifying the heterogeneity of mesencephalic DA neurons, which is necessary for understanding their diverse physiological functions and disease susceptibility. Moreover, there is a conflict regarding the electrophysiological properties of the distinct subsets of midbrain DA neurons. This review aimed to elucidate recent developments in the heterogeneity of midbrain DA neurons, including subpopulation categorization, electrophysiological characteristics, and functional connectivity to provide new strategies for accurately identifying distinct subtypes of midbrain DA neurons and investigating the underlying mechanisms of these neurons in various diseases.

Alterations in type 2 dopamine receptors across neuropsychiatric conditions: A large-scale PET cohort

PURPOSE

Aberrant dopaminergic function is linked with motor, psychotic, and affective symptoms, but studies have typically compared a single patient group with healthy controls.

METHODS

Here, we investigated the variation in striatal (caudate nucleus, nucleus accumbens, and putamen) and thalamic type 2 dopamine receptor (D2R) availability using [11C]raclopride positron emission tomography (PET) data from a large sample of 437 humans including healthy controls, and subjects with Parkinson's disease (PD), antipsychotic-naïve schizophrenia, severe violent behavior, pathological gambling, depression, and overweight. We analyzed regional group differences in D2R availability. We also analyzed the interregional correlation in D2R availability within each group.

RESULTS

Subjects with PD showed the clearest decline in D2R availability. Overall, the groups showed high interregional correlation in D2R availability, while this pattern was weaker in violent offenders. Subjects with schizophrenia, pathological gambling, depression, or overweight did not show clear changes in either the regional receptor availability or the interregional correlation.

CONCLUSION

We conclude that the dopaminergic changes in neuropsychiatric conditions might not only affect the overall receptor availability but also how coupled regions are across people. The region-specific receptor availability more profoundly links to the motor symptoms, while the between-region coupling might be disrupted in violence.

Dopamine D2 receptors in WFS1-neurons regulate food-seeking and avoidance behaviors

The selection and optimization of appropriate adaptive responses depends on interoceptive and exteroceptive stimuli as well as on the animal's ability to switch from one behavioral strategy to another. Although growing evidence indicate that dopamine D2R-mediated signaling events ensure the selection of the appropriate strategy for each specific situation, the underlying neural circuits through which they mediate these effects are poorly characterized. Here, we investigated the role of D2R signaling in a mesolimbic neuronal subpopulation expressing the Wolfram syndrome 1 (Wfs1) gene. This subpopulation is located within the nucleus accumbens, the central amygdala, the bed nucleus of the stria terminalis, and the tail of the striatum, all brain regions critical for the regulation of emotions and motivated behaviors. Using a mouse model carrying a temporally controlled deletion of D2R in WFS1-neurons, we demonstrate that intact D2R signaling in this neuronal population is necessary to regulate homeostasis-dependent food-seeking behaviors in both male and female mice. In addition, we found that reduced D2R signaling in WFS1-neurons impaired active avoidance learning and innate escape responses. Collectively, these findings identify a yet undocumented role for D2R signaling in WFS1-neurons as a novel effector through which dopamine optimizes appetitive behaviors and regulates defensive behaviors.

Medicalization of Neurodivergence and the Embodied Experience of ADHD

Medicalization shapes, and in some cases legitimizes, individuals' embodied experiences even as it molds the landscape of healthcare and treatment. In this essay I provide a layered account that moves between my experiences as a neurodivergent person and academic theorizing to explore how processes of medicalization inform public discourses and personal sensemaking. In the case of ADHD, medicalization has contributed to societal narratives that focus on symptoms of hyperactivity rather than the etiology of dopamine dysregulation. Such narratives fail to fully account for the lived experience of ADHD and inadvertently stigmatize neurodivergent individuals. I urge scholars and practitioners to direct more attention to the communicative dimensions of medicalization including both the rhetorical nature of the diagnostic process and how diagnoses, in turn, are rhetorically framed with varying consequences.

Fast adaptation to rule switching using neuronal surprise

In humans and animals, surprise is a physiological reaction to an unexpected event, but how surprise can be linked to plausible models of neuronal activity is an open problem. We propose a self-supervised spiking neural network model where a surprise signal is extracted from an increase in neural activity after an imbalance of excitation and inhibition. The surprise signal modulates synaptic plasticity via a three-factor learning rule which increases plasticity at moments of surprise. The surprise signal remains small when transitions between sensory events follow a previously learned rule but increases immediately after rule switching. In a spiking network with several modules, previously learned rules are protected against overwriting, as long as the number of modules is larger than the total number of rules-making a step towards solving the stability-plasticity dilemma in neuroscience. Our model relates the subjective notion of surprise to specific predictions on the circuit level.

A Systematic Review of Direct Outputs from the Cerebellum to the Brainstem and Diencephalon in Mammals

The cerebellum is involved in many motor, autonomic and cognitive functions, and new tasks that have a cerebellar contribution are discovered on a regular basis. Simultaneously, our insight into the functional compartmentalization of the cerebellum has markedly improved. Additionally, studies on cerebellar output pathways have seen a renaissance due to the development of viral tracing techniques. To create an overview of the current state of our understanding of cerebellar efferents, we undertook a systematic review of all studies on monosynaptic projections from the cerebellum to the brainstem and the diencephalon in mammals. This revealed that important projections from the cerebellum, to the motor nuclei, cerebral cortex, and basal ganglia, are predominantly di- or polysynaptic, rather than monosynaptic. Strikingly, most target areas receive cerebellar input from all three cerebellar nuclei, showing a convergence of cerebellar information at the output level. Overall, there appeared to be a large level of agreement between studies on different species as well as on the use of different types of neural tracers, making the emerging picture of the cerebellar output areas a solid one. Finally, we discuss how this cerebellar output network is affected by a range of diseases and syndromes, with also non-cerebellar diseases having impact on cerebellar output areas.

Dynamic behaviour restructuring mediates dopamine-dependent credit assignment

Animals exhibit a diverse behavioural repertoire when exploring new environments and can learn which actions or action sequences produce positive outcomes. Dopamine release after encountering a reward is critical for reinforcing reward-producing actions1-3. However, it has been challenging to understand how credit is assigned to the exact action that produced the dopamine release during continuous behaviour. Here we investigated this problem in mice using a self-stimulation paradigm in which specific spontaneous movements triggered optogenetic stimulation of dopaminergic neurons. Dopamine self-stimulation rapidly and dynamically changes the structure of the entire behavioural repertoire. Initial stimulations reinforced not only the stimulation-producing target action, but also actions similar to the target action and actions that occurred a few seconds before stimulation. Repeated pairings led to a gradual refinement of the behavioural repertoire to home in on the target action. Reinforcement of action sequences revealed further temporal dependencies of refinement. Action pairs spontaneously separated by long time intervals promoted a stepwise credit assignment, with early refinement of actions most proximal to stimulation and subsequent refinement of more distal actions. Thus, a retrospective reinforcement mechanism promotes not only reinforcement, but also gradual refinement of the entire behavioural repertoire to assign credit to specific actions and action sequences that lead to dopamine release.

Prediction error in dopamine neurons during associative learning

Dopamine neurons have long been thought to facilitate learning by broadcasting reward prediction error (RPE), a teaching signal used in machine learning, but more recent work has advanced alternative models of dopamine's computational role. Here, I revisit this critical issue and review new experimental evidences that tighten the link between dopamine activity and RPE. First, I introduce the recent observation of a gradual backward shift of dopamine activity that had eluded researchers for over a decade. I also discuss several other findings, such as dopamine ramping, that were initially interpreted to conflict but later found to be consistent with RPE. These findings improve our understanding of neural computation in dopamine neurons.

Amygdalo-nigral inputs target dopaminergic and GABAergic neurons in the primate: a view from dendrites and soma

The central nucleus (CeN) of the amygdala is an important afferent to the DA system that mediates motivated learning. We previously found that CeN terminals in nonhuman primates primarily overlap the elongated lateral VTA (parabrachial pigmented nucleus, PBP, A10), and retrorubral field(A8) subregion. Here, we examined CeN afferent contacts on cell somata and proximal dendrites of DA and GABA neurons, and distal dendrites of each, using confocal and electron microscopy (EM) methods, respectively. At the soma/proximal dendrites, the proportion of TH+ and GAD1+ cells receiving at least one CeN afferent contact was surprisingly similar (TH = 0.55: GAD1=0.55 in PBP; TH = 0.56; GAD1 =0.51 in A8), with the vast majority of contacted TH+ and GAD1+ soma/proximal dendrites received 1-2 contacts. Similar numbers of tracer-labeled terminals also contacted TH-positive and GAD1-positive small dendrites and/or spines (39% of all contacted dendrites were either TH- or GAD1-labeled). Overall, axon terminals had more symmetric (putative inhibitory) axonal contacts with no difference in the relative distribution in the PBP versus A8, or onto TH+ versus GAD1+ dendrites/spines in either region. The striking uniformity in the amygdalonigral projection across the PBP-A8 terminal field suggests that neither neurotransmitter phenotype nor midbrain location dictates likelihood of a terminal contact. We discuss how this afferent uniformity can play out in recently discovered differences in DA:GABA cell densities between the PBP and A8, and affect specific outputs.

Significance statement

The amygdala's central nucleus (CeN) channels salient cues to influence both appetitive and aversive responses via DA outputs. In higher species, the broad CeN terminal field overlaps the parabrachial pigmented nucleus ('lateral A10') and the retrorubral field (A8). We quantified terminal contacts in each region on DA and GABAergic soma/proximal dendrites and small distal dendrites. There was striking uniformity in contacts on DA and GABAergic cells, regardless of soma and dendritic compartment, in both regions. Most contacts were symmetric (putative inhibitory) with little change in the ratio of inhibitory to excitatory contacts by region.We conclude that post-synaptic shifts in DA-GABA ratios are key to understanding how these relatively uniform inputs can produce diverse effects on outputs.

Altered neural activity in the reward-related circuit associated with anhedonia in mild to moderate Major Depressive Disorder

BACKGROUND

Anhedonia is a significant predictor of disease progression and treatment outcomes in Major Depressive Disorder (MDD), linked to reward network dysfunctions. However, understanding of its underlying neural mechanisms remains limited. This study aimed to investigate the brain functional mechanisms underlying MDD with anhedonia using resting-state functional magnetic resonance imaging (rs-fMRI).

METHODS

The Snaith-Hamilton Pleasure Scale (SHAPS) was used to evaluation MDD with anhedonia (anMDD) and non-anhedonia MDD (non-anMDD). Forty-eight patients with anMDD, Forty-four patients with non-anMDD, and Fifty healthy controls (HCs) were enrolled for the fMRI scans. A seed-based functional connectivity (FC) method was employed to explore reward network abnormalities.

RESULTS

anMDD patients exhibited lower FC values in Ventral Striatum (VS), right lateral Ventral Tegmental Area (VTA_R), left Thalamus (THA_L), and higher FC values in Ventromedial Prefrontal Cortex (vmPFC), left Anterior Insula (AI_L), and Presupplementary Motor Area (Pre-SMA) compared to HCs. Comparing anMDD to non-anMDD, significant differences were observed in FC values of VS, vmPFC, Pre-SMA, and THA_L regions. Correlation analysis revealed positive correlations between FC values of VS_R and NAc_R, as well as THA_L and Cerebellum_Crus1_L, with SHAPS scores. Negative correlations were observed between FC values of Pre-SMA and the right caudate, and between vmPFC and Frontal_Mid_Orb_L, and SHAPS scores.

CONCLUSION

Both anMDD and non-anMDD groups demonstrated abnormal FCs in the reward network. These findings indicate distinct roles of reward-related circuits in the two subtypes, contributing to a refined understanding of depression phenotypes and potential directions for targeted interventions.

Neural Representations of Post-Decision Accuracy and Reward Expectation in the Caudate Nucleus and Frontal Eye Field

Performance monitoring that supports ongoing behavioral adjustments is often examined in the context of either choice confidence for perceptual decisions (i.e., "did I get it right?") or reward expectation for reward-based decisions (i.e., "what reward will I receive?"). However, our understanding of how the brain encodes these distinct evaluative signals remains limited because they are easily conflated, particularly in commonly used two-alternative tasks with symmetric rewards for correct choices. Previously we used a motion-discrimination task with asymmetric rewards to identify neural substrates of forming reward-biased perceptual decisions in the caudate nucleus (part of the striatum in the basal ganglia) and the frontal eye field (FEF, in prefrontal cortex). Here we leveraged this task design to partially decouple estimates of accuracy and reward expectation and examine their impacts on subsequent decisions and their representations in those two brain areas. We identified distinguishable representations of these two evaluative signals in individual caudate and FEF neurons, with regional differences in their distribution patterns and time courses. We observed that well-trained monkeys (both sexes) used both evaluative signals, infrequently but consistently, to adjust their subsequent decisions. We found further that these behavioral adjustments had reliable relationships with the neural representations of both evaluative signals in caudate, but not FEF. These results suggest that the cortico-striatal decision network may use diverse evaluative signals to monitor and adjust decision-making behaviors, adding to our understanding of the different roles that the FEF and caudate nucleus play in a diversity of decision-related computations.

Balancing risk-return decisions by manipulating the mesofrontal circuits in primates

Decision-making is always coupled with some level of risk, with more pathological forms of risk-taking decisions manifesting as gambling disorders. In macaque monkeys trained in a high risk-high return (HH) versus low risk-low return (LL) choice task, we found that the reversible pharmacological inactivation of ventral Brodmann area 6 (area 6V) impaired the risk dependency of decision-making. Selective optogenetic activation of the mesofrontal pathway from the ventral tegmental area (VTA) to the ventral aspect of 6V resulted in stronger preference for HH, whereas activation of the pathway from the VTA to the dorsal aspect of 6V led to LL preference. Finally, computational decoding captured the modulations of behavioral preference. Our results suggest that VTA inputs to area 6V determine the decision balance between HH and LL.

Separation of Channels Subserving Approach and Avoidance/Escape at the Level of the Basal Ganglia and Related Brainstem Structures

The basal ganglia have the key function of directing our behavior in the context of events from our environment and/or our internal state. This function relies on afferents targeting the main input structures of the basal ganglia, entering bids for action selection at the level of the striatum or signals for behavioral interruption at the level of the subthalamic nucleus, with behavioral reselection facilitated by dopamine signaling. Numerous experiments have studied action selection in relation to inputs from the cerebral cortex. However, less is known about the anatomical and functional link between the basal ganglia and the brainstem. In this review, we describe how brainstem structures also project to the main input structures of the basal ganglia, namely the striatum, the subthalamic nucleus and midbrain dopaminergic neurons, in the context of approach and avoidance (including escape from threat), two fundamental, mutually exclusive behavioral choices in an animal's repertoire in which the brainstem is strongly involved. We focus on three particularly well-described loci involved in approach and avoidance, namely the superior colliculus, the parabrachial nucleus and the periaqueductal grey nucleus. We consider what is known about how these structures are related to the basal ganglia, focusing on their projections toward the striatum, dopaminergic neurons and subthalamic nucleus, and explore the functional consequences of those interactions.

Motivational Vigor in Parkinson's Disease Requires the Short and Long Duration Response to Levodopa

BACKGROUND

Impaired movement vigor (bradykinesia) is a cardinal feature of Parkinson's disease (PD) and hypothesized to result from abnormal motivational processes-impaired motivation-vigor coupling. Dopamine replacement therapy (DRT) improves bradykinesia, but the response to DRT is multifaceted, comprising a short-duration response (SDR) and a long-duration response (LDR) only manifesting with chronic treatment. Prior experiments assessing motivation-vigor coupling in PD used chronically treated subjects, obscuring the roles of the SDR and LDR.

METHODS

To disambiguate the SDR and LDR, 11 de novo PD subjects (6 male [M]:5 female [F]; mean age, 67) were studied before treatment, after an acute levodopa (l-dopa) dose, and in both the practical "off" (LDR) and "on" (LDR + SDR) states after chronic stable treatment. At each visit, subjects were characterized with a standard battery including the Movement Disorder Society-Sponsored Revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS) and an incentivized joystick task to assess motor performance in response to varying rewards.

RESULTS

l-Dopa induced a robust SDR and LDR, with further improvement in the combined SDR + LDR state. At baseline, after acute treatment (SDR), and after LDR induction, subjects did not exhibit the normal increase in movement speed with increasing reward. Only in the combined SDR + LDR state was there restoration of motivation-vigor coupling.

CONCLUSIONS

Although consistent with prior results in chronically treated PD subjects, the significant improvement in motor performance observed with the SDR and LDR suggests that bradykinesia is not solely secondary to deficient modulation of motivational processes. © 2023 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

Sensory Reinforced Corticostriatal Plasticity

BACKGROUND

Regional changes in corticostriatal transmission induced by phasic dopaminergic signals are an essential feature of the neural network responsible for instrumental reinforcement during discovery of an action. However, the timing of signals that are thought to contribute to the induction of corticostriatal plasticity is difficult to reconcile within the framework of behavioural reinforcement learning, because the reinforcer is normally delayed relative to the selection and execution of causally-related actions.

OBJECTIVE

While recent studies have started to address the relevance of delayed reinforcement signals and their impact on corticostriatal processing, our objective was to establish a model in which a sensory reinforcer triggers appropriately delayed reinforcement signals relayed to the striatum via intact neuronal pathways and to investigate the effects on corticostriatal plasticity.

METHODS

We measured corticostriatal plasticity with electrophysiological recordings using a light flash as a natural sensory reinforcer, and pharmacological manipulations were applied in an in vivo anesthetized rat model preparation.

RESULTS

We demonstrate that the spiking of striatal neurons evoked by single-pulse stimulation of the motor cortex can be potentiated by a natural sensory reinforcer, operating through intact afferent pathways, with signal timing approximating that required for behavioural reinforcement. The pharmacological blockade of dopamine receptors attenuated the observed potentiation of corticostriatal neurotransmission.

CONCLUSION

This novel in vivo model of corticostriatal plasticity offers a behaviourally relevant framework to address the physiological, anatomical, cellular, and molecular bases of instrumental reinforcement learning.

Pathways from the Superior Colliculus to the Basal Ganglia

The present work aims to review the structural organization of the mammalian superior colliculus (SC), the putative pathways connecting the SC and the basal ganglia, and their role in organizing complex behavioral output. First, we review how the complex intrinsic connections between the SC's laminae projections allow for the construction of spatially aligned, visual-multisensory maps of the surrounding environment. Moreover, we present a summary of the sensory-motor inputs of the SC, including a description of the integration of multi-sensory inputs relevant to behavioral control. We further examine the major descending outputs toward the brainstem and spinal cord. As the central piece of this review, we provide a thorough analysis covering the putative interactions between the SC and the basal ganglia. To this end, we explore the diverse thalamic routes by which information from the SC may reach the striatum, including the pathways through the lateral posterior, parafascicular, and rostral intralaminar thalamic nuclei. We also examine the interactions between the SC and subthalamic nucleus, representing an additional pathway for the tectal modulation of the basal ganglia. Moreover, we discuss how information from the SC might also be relayed to the basal ganglia through midbrain tectonigral and tectotegmental projections directed at the substantia nigra compacta and ventrotegmental area, respectively, influencing the dopaminergic outflow to the dorsal and ventral striatum. We highlight the vast interplay between the SC and the basal ganglia and raise several missing points that warrant being addressed in future studies.

Activity-dependent constraints on catecholamine signaling

Catecholamine signaling is thought to modulate cognition in an inverted-U relationship, but the mechanisms are unclear. We measured norepinephrine and dopamine release, postsynaptic calcium responses, and interactions between tonic and phasic firing modes under various stimuli and conditions. High tonic activity in vivo depleted catecholamine stores, desensitized postsynaptic responses, and decreased phasic transmission. Together, these findings provide a more complete understanding of the inverted-U relationship, offering insights into psychiatric disorders and neurodegenerative diseases with impaired catecholamine signaling.

Physical Exercise Promotes Beneficial Changes on Neurotrophic Factors in Mesolimbic Brain Areas After AMPH Relapse: Involvement of the Endogenous Opioid System

Addiction is a serious public health problem, and the current pharmacotherapy is unable to prevent drug use reinstatement. Studies have focused on physical exercise as a promising coadjuvant treatment. Our research group recently showed beneficial neuroadaptations in the dopaminergic system related to amphetamine-relapse prevention involving physical exercise-induced endogenous opioid system activation (EXE-OS activation). In this context, additional mechanisms were explored to understand the exercise benefits on drug addiction. Male rats previously exposed to amphetamine (AMPH, 4.0 mg/kg) for 8 days were submitted to physical exercise for 5 weeks. EXE-OS activation was blocked by naloxone administration (0.3 mg/kg) 5 min before each physical exercise session. After the exercise protocol, the rats were re-exposed to AMPH for 3 days, and in sequence, euthanasia was performed and the VTA and NAc were dissected. In the VTA, our findings showed increased immunocontent of proBDNF, BDNF, and GDNF and decreased levels of AMPH-induced TrkB; therefore, EXE-OS activation increased all these markers and naloxone administration prevented this exercise-induced effect. In the NAc, the same molecular markers were also increased by AMPH and decreased by EXE-OS activation. In this study, we propose a close relation between EXE-OS activation beneficial influence and a consequent neuroadaptation on neurotrophins and dopaminergic system levels in the mesolimbic brain area, preventing the observed AMPH-relapse behavior. Our outcomes bring additional knowledge concerning addiction neurobiology understanding and show that EXE-OS activation may be a potential adjuvant tool in drug addiction therapy.

The neurocognitive role of working memory load when Pavlovian motivational control affects instrumental learning

Research suggests that a fast, capacity-limited working memory (WM) system and a slow, incremental reinforcement learning (RL) system jointly contribute to instrumental learning. Thus, situations that strain WM resources alter instrumental learning: under WM loads, learning becomes slow and incremental, the reliance on computationally efficient learning increases, and action selection becomes more random. It is also suggested that Pavlovian learning influences people's behavior during instrumental learning by providing hard-wired instinctive responses including approach to reward predictors and avoidance of punishment predictors. However, it remains unknown how constraints on WM resources affect instrumental learning under Pavlovian influence. Thus, we conducted a functional magnetic resonance imaging (fMRI) study (N = 49) in which participants completed an instrumental learning task with Pavlovian-instrumental conflict (the orthogonalized go/no-go task) both with and without extra WM load. Behavioral and computational modeling analyses revealed that WM load reduced the learning rate and increased random choice, without affecting Pavlovian bias. Model-based fMRI analysis revealed that WM load strengthened RPE signaling in the striatum. Moreover, under WM load, the striatum showed weakened connectivity with the ventromedial and dorsolateral prefrontal cortex when computing reward expectations. These results suggest that the limitation of cognitive resources by WM load promotes slow and incremental learning through the weakened cooperation between WM and RL; such limitation also makes action selection more random, but it does not directly affect the balance between instrumental and Pavlovian systems.

Characterization of striatal dopamine projections across striatal subregions in behavioral flexibility

Behavioural flexibility is key to survival in a dynamic environmentWhile flexible, goal-directed behaviours are initially dependent on dorsomedial striatum, they become dependent on lateral striatum as behaviours become inflexible. Similarly, lesions of dopamine terminals in lateral striatum disrupt the development of inflexible habits. This work suggests that dopamine release in lateral striatum may drive inflexible behaviours, though few studies have investigated a causative role of subpopulations of striatal dopamine terminals in reversal learning, a measure of flexibility. Here, we performed two optogenetic experiments to activate dopamine terminals in dorsomedial (DMS), dorsolateral (DLS) or ventral (nucleus accumbens [NAc]) striatum in DAT-Cre mice that expressed channelrhodopsin-2 via viral injection (Experiment I) or through transgenic breeding with an Ai32 reporter line (Experiment II) to determine how specific dopamine subpopulations impact reversal learning. Mice performed a reversal task in which they self-stimulated DMS, DLS, or NAc dopamine terminals by pressing one of two levers before action-outcome lever contingencies were reversed. Largely consistent with presumed ventromedial/lateral striatal function, we found that mice self-stimulating medial dopamine terminals reversed lever preference following contingency reversal, while mice self-stimulating NAc showed parial flexibility, and DLS self-stimulation resulted in impaired reversal. Impairments in DLS mice were characterized by more regressive errors and reliance on lose-stay strategies following reversal, as well as reduced within-session learning, suggesting reward insensitivity and overreliance on previously learned actions. This study supports a model of striatal function in which DMS and ventral dopamine facilitate goal-directed responding, and DLS dopamine supports more inflexible responding.

Social cognition in hyperkinetic movement disorders: a systematic review

Numerous lines of research indicate that our social brain involves a network of cortical and subcortical brain regions that are responsible for sensing and controlling body movements. However, it remains unclear whether movement disorders have a systematic impact on social cognition. To address this question, we conducted a systematic review examining the influence of hyperkinetic movement disorders (including Huntington disease, Tourette syndrome, dystonia, and essential tremor) on social cognition. Following the PRISMA guidelines and registering the protocol in the PROSPERO database (CRD42022327459), we analyzed 50 published studies focusing on theory of mind (ToM), social perception, and empathy. The results from these studies provide evidence of impairments in ToM and social perception in all hyperkinetic movement disorders, particularly during the recognition of negative emotions. Additionally, individuals with Huntington's Disease and Tourette syndrome exhibit empathy disorders. These findings support the functional role of subcortical structures (such as the basal ganglia and cerebellum), which are primarily responsible for movement disorders, in deficits related to social cognition.

Top-down circuitry from the anterior insular cortex to VTA dopamine neurons modulates reward-related memory

The insular cortex (IC) has been linked to the processing of interoceptive and exteroceptive signals associated with addictive behavior. However, whether the IC modulates the acquisition of drug-related affective states by direct top-down connectivity with ventral tegmental area (VTA) dopamine neurons is unknown. We found that photostimulation of VTA terminals of the anterior insular cortex (aIC) induces rewarding contextual memory, modulates VTA activity, and triggers dopamine release within the VTA. Employing neuronal recordings and neurochemical and transsynaptic tagging techniques, we disclose the functional top-down organization tagging the aIC pre-synaptic neuronal bodies and identifying VTA recipient neurons. Furthermore, systemic administration of amphetamine altered the VTA excitability of neurons modulated by the aIC projection, where photoactivation enhances, whereas photoinhibition impairs, a contextual rewarding behavior. Our study reveals a key circuit involved in developing and retaining drug reward-related contextual memory, providing insight into the neurobiological basis of addictive behavior and helping develop therapeutic addiction strategies.

Premovement activity in the mesocortical system links peak force but not initiation of force generation under incentive motivation

Motivation facilitates motor performance; however, the neural substrates of the psychological effects on motor performance remain unclear. We conducted a functional magnetic resonance imaging experiment while human subjects performed a ready-set-go task with monetary incentives. Although subjects were only motivated to respond quickly, increasing the incentives improved not only reaction time but also peak grip force. However, the trial-by-trial correlation between reaction time and peak grip force was weak. Extensive areas in the mesocortical system, including the ventral midbrain (VM) and cortical motor-related areas, exhibited motivation-dependent activity in the premovement "Ready" period when the anticipated monetary reward was displayed. This premovement activity in the mesocortical system correlated only with subsequent peak grip force, whereas the activity in motor-related areas alone was associated with subsequent reaction time and peak grip force. These findings suggest that the mesocortical system linking the VM and motor-related regions plays a role in controlling the peak of force generation indirectly associated with incentives but not the initiation of force generation.

Monosynaptic inputs to ventral tegmental area glutamate and GABA co-transmitting neurons

A unique population of ventral tegmental area (VTA) neurons co-transmits glutamate and GABA as well as functionally signals rewarding and aversive outcomes. However, the circuit inputs to VTA VGluT2+VGaT+ neurons are unknown, limiting our understanding of the functional capabilities of these neurons. To identify the inputs to VTA VGluT2+VGaT+ neurons, we coupled monosynaptic rabies tracing with intersectional genetic targeting of VTA VGluT2+VGaT+ neurons in mice. We found that VTA VGluT2+VGaT+ neurons received diverse brain-wide inputs. The largest numbers of monosynaptic inputs to VTA VGluT2+VGaT+ neurons were from superior colliculus, lateral hypothalamus, midbrain reticular nucleus, and periaqueductal gray, whereas the densest inputs relative to brain region volume were from dorsal raphe nucleus, lateral habenula, and ventral tegmental area. Based on these and prior data, we hypothesized that lateral hypothalamus and superior colliculus inputs were glutamatergic neurons. Optical activation of glutamatergic lateral hypothalamus neurons robustly activated VTA VGluT2+VGaT+ neurons regardless of stimulation frequency and resulted in flee-like ambulatory behavior. In contrast, optical activation of glutamatergic superior colliculus neurons activated VTA VGluT2+VGaT+ neurons for a brief period of time at high stimulation frequency and resulted in head rotation and arrested ambulatory behavior (freezing). For both pathways, behaviors induced by stimulation were uncorrelated with VTA VGluT2+VGaT+ neuron activity. However, stimulation of glutamatergic lateral hypothalamus neurons, but not glutamatergic superior colliculus neurons, was associated with VTA VGluT2+VGaT+ footshock-induced activity. We interpret these results such that inputs to VTA VGluT2+VGaT+ neurons may integrate diverse signals related to the detection and processing of motivationally-salient outcomes. Further, VTA VGluT2+VGaT+ neurons may signal threat-related outcomes, possibly via input from lateral hypothalamus glutamate neurons, but not threat-induced behavioral kinematics.

Mobilization of endocannabinoids by midbrain dopamine neurons is required for the encoding of reward prediction

Brain levels of the endocannabinoid 2-arachidonoylglycerol (2-AG) shape motivated behavior and nucleus accumbens (NAc) dopamine release. However, it is not clear whether mobilization of 2-AG specifically from midbrain dopamine neurons is necessary for dopaminergic responses to external stimuli predicting forthcoming reward. Here, we use a viral-genetic strategy to prevent the expression of the 2-AG-synthesizing enzyme diacylglycerol lipase α (DGLα) from ventral tegmental area (VTA) dopamine cells in adult mice. We find that DGLα deletion from VTA dopamine neurons prevents depolarization-induced suppression of excitation (DSE), a form of 2-AG-mediated synaptic plasticity, in dopamine neurons. DGLα deletion also decreases effortful, cue-driven reward-seeking but has no effect on non-cued or low-effort operant tasks and other behaviors. Moreover, dopamine recording in the NAc reveals that deletion of DGLα impairs the transfer of accumbal dopamine signaling from a reward to its earliest predictors. These results demonstrate that 2-AG mobilization from VTA dopamine neurons is a necessary step for the generation of dopamine-based predictive associations that are required to direct and energize reward-oriented behavior.

Only cortical prediction error signals are involved in visual learning, despite availability of subcortical prediction error signals

Both the midbrain systems, encompassing the ventral striatum (VS), and the cortical systems, including the dorsal anterior cingulate cortex (dACC), play roles in reinforcing and enhancing learning. However, the specific contributions of signals from these regions in learning remains unclear. To investigate this, we examined how VS and dACC are involved in visual perceptual learning (VPL) through an orientation discrimination task. In the primary experiment, subjects fasted for 5 hours before each of 14 days of training sessions and 3 days of test sessions. Subjects were rewarded with water for accurate trial responses. During the test sessions, BOLD signals were recorded from regions including VS and dACC. Although BOLD signals in both areas were associated with positive and negative RPEs, only those in dACC associated with negative RPE showed a significant correlation with performance improvement. Additionally, no significant correlation was observed between BOLD signals associated with RPEs in VS and dACC. These results suggest that although signals associated with positive and negative RPEs from both midbrain and cortical systems are readily accessible, only RPE signals in the prefrontal system, generated without linking to RPE signals in VS, are utilized for the enhancement of VPL.

Sex-Specific and Traumatic Brain Injury Effects on Dopamine Receptor Expression in the Hippocampus

Traumatic brain injury (TBI) is a major health concern. Each year, over 50 million individuals worldwide suffer from TBI, and this leads to a number of acute and chronic health issues. These include affective and cognitive impairment, as well as an increased risk of alcohol and drug use. The dopaminergic system, a key component of reward circuitry, has been linked to alcohol and other substance use disorders, and previous research indicates that TBI can induce plasticity within this system. Understanding how TBI modifies the dopaminergic system may offer insights into the heightened substance use and reward-seeking behavior following TBI. The hippocampus, a critical component of the reward circuit, is responsible for encoding and integrating the spatial and salient aspects of rewarding stimuli. This study explored TBI-related changes in neuronal D2 receptor expression within the hippocampus, examining the hypothesis that sex differences exist in both baseline hippocampal D2 receptor expression and its response to TBI. Utilizing D2-expressing tdTomato transgenic male and female mice, we implemented either a sham injury or the lateral fluid percussion injury (FPI) model of TBI and subsequently performed a region-specific quantification of D2 expression in the hippocampus. The results show that male mice exhibit higher baseline hippocampal D2 expression compared to female mice. Additionally, there was a significant interaction effect between sex and injury on the expression of D2 in the hippocampus, particularly in regions of the dentate gyrus. Furthermore, TBI led to significant reductions in hippocampal D2 expression in male mice, while female mice remained mostly unaffected. These results suggest that hippocampal D2 expression varies between male and female mice, with the female dopaminergic system demonstrating less susceptibility to TBI-induced plasticity.

Trace Amine-Associated Receptors and Monoamine-Mediated Regulation of Insulin Secretion in Pancreatic Islets

Currently, metabolic syndrome treatment includes predominantly pharmacological symptom relief and complex lifestyle changes. Trace amines and their receptor systems modulate signaling pathways of dopamine, norepinephrine, and serotonin, which are involved in the pathogenesis of this disorder. Trace amine-associated receptor 1 (TAAR1) is expressed in endocrine organs, and it was revealed that TAAR1 may regulate insulin secretion in pancreatic islet β-cells. For instance, accumulating data demonstrate the positive effect of TAAR1 agonists on the dynamics of metabolic syndrome progression and MetS-associated disease development. The role of other TAARs (TAAR2, TAAR5, TAAR6, TAAR8, and TAAR9) in the islet's function is much less studied. In this review, we summarize the evidence of TAARs' contribution to the metabolic syndrome pathogenesis and regulation of insulin secretion in pancreatic islets. Additionally, by the analysis of public transcriptomic data, we demonstrate that TAAR1 and other TAAR receptors are expressed in the pancreatic islets. We also explore associations between the expression of TAARs mRNA and other genes in studied samples and demonstrate the deregulation of TAARs' functional associations in patients with metabolic diseases compared to healthy donors.

Contributions of transient and sustained reward to memory formation

Reward benefits to memory formation have been robustly linked to dopaminergic activity. Despite the established characterization of dopaminergic mechanisms as operating at multiple timescales, potentially supporting distinct functional outcomes, the temporal dynamics by which reward might modulate memory encoding are just beginning to be investigated. In the present study, we leveraged a mixed block/event experimental design to disentangle transient and sustained reward influences on task engagement and subsequent recognition memory in an adapted monetary-incentive-encoding (MIE) paradigm. Across three behavioral experiments, transient and sustained reward modulation of item and context memory was probed, at both 24-h and ~ 15-min retention intervals, to investigate the importance of overnight consolidation. In general, we observed that transient reward was associated with enhanced item memory encoding, while sustained reward modulated response speed but did not appear to benefit subsequent recognition accuracy. Notably, reward effects on item memory performance and response speed were somewhat inconsistent across the three experiments, with suggestions that RT speeding might also be related to time on task, and we did not observe reward modulation of context memory performance or amplification of reward benefits to memory by overnight consolidation. Taken together, the observed pattern of behavior is consistent with potentially distinct roles for transient and sustained reward in memory encoding and cognitive performance and suggests that further investigation of the temporal dynamics of dopaminergic contributions to memory formation will advance the understanding of motivated memory.

How trial-to-trial learning shapes mappings in the mental lexicon: Modelling lexical decision with linear discriminative learning

Trial-to-trial effects have been found in a number of studies, indicating that processing a stimulus influences responses in subsequent trials. A special case are priming effects which have been modelled successfully with error-driven learning (Marsolek, 2008), implying that participants are continuously learning during experiments. This study investigates whether trial-to-trial learning can be detected in an unprimed lexical decision experiment. We used the Discriminative Lexicon Model (DLM; Baayen et al., 2019), a model of the mental lexicon with meaning representations from distributional semantics, which models error-driven incremental learning with the Widrow-Hoff rule. We used data from the British Lexicon Project (BLP; Keuleers et al., 2012) and simulated the lexical decision experiment with the DLM on a trial-by-trial basis for each subject individually. Then, reaction times were predicted with Generalized Additive Models (GAMs), using measures derived from the DLM simulations as predictors. We extracted measures from two simulations per subject (one with learning updates between trials and one without), and used them as input to two GAMs. Learning-based models showed better model fit than the non-learning ones for the majority of subjects. Our measures also provide insights into lexical processing and individual differences. This demonstrates the potential of the DLM to model behavioural data and leads to the conclusion that trial-to-trial learning can indeed be detected in unprimed lexical decision. Our results support the possibility that our lexical knowledge is subject to continuous changes.

Neurocircuits for motivation

The nervous system coordinates various motivated behaviors such as feeding, drinking, and escape to promote survival and evolutionary fitness. Although the precise behavioral repertoires required for distinct motivated behaviors are diverse, common features such as approach or avoidance suggest that common brain substrates are required for a wide range of motivated behaviors. In this Review, I describe a framework by which neural circuits specified for some innate drives regulate the activity of ventral tegmental area (VTA) dopamine neurons to reinforce ongoing or planned actions to fulfill motivational demands. This framework may explain why signaling from VTA dopamine neurons is ubiquitously involved in many types of diverse volitional motivated actions, as well as how sensory and interoceptive cues can initiate specific goal-directed actions.

Comment on Albantakis et al. Computing the Integrated Information of a Quantum Mechanism. Entropy 2023, 25, 449

Integrated information theory (IIT) is a powerful tool that provides a framework for evaluating consciousness, whether in the human brain or in other systems. In Computing the Integrated Information of a Quantum Mechanism, the authors extend IIT from digital gates to a quantum CNOT logic gate, and while they explicitly distinguish the analysis from quantum theories of consciousness, they nonetheless provide an analytical road map for extending IIT not only to other quantum mechanisms but also to hybrid computing structures like the brain. This comment provides additional information relating to an adiabatic quantum mechanical energy routing mechanism that is part of a hybrid biological computer that provides an action selection mechanism, which has been hypothesized to exist in the human brain and for which predicted evidence has been subsequently observed, and it hopes to motivate the further evaluation and extension of IIT not only to that hypothesized mechanism but also to other hybrid biological computers.

The association between lifelong personality and clinical phenotype in the FTD-ALS spectrum

Introduction

Frontotemporal dementia (FTD) and amyotrophic lateral sclerosis (ALS) are two phenotypes of the same neurodegenerative disease, the FTD-ALS spectrum. What determines the development of one rather than the other phenotype is still unknown. Based on the clinical observation that patients' personality seems to differ between the two phenotypes, i.e., ALS patients tend to display kind, prosocial behaviors whereas FTD patients tend to present anti-social behaviors, and that these traits are often reported as pre-existing the disease onset by caregivers, we set up to study experimentally patients' personality in their premorbid life.

Methods

We first tested for differences between groups, then tested the association between premorbid personality and current functional organization of the brain. Premorbid personality of a cohort of forty patients, 27 FTD and 13 ALS, was explored through the NEO Personality Inventory 3 (NEO-PI-3), which analyses the five main personality factors, completed by the caregiver with reference to patient's personality 20 years before symptoms onset (premorbid). A subgroup of patients underwent a brain MRI including structural and resting-state functional MRI (rsfMRI).

Results

A significant difference between FTD and ALS in premorbid personality emerged in the Openness (133.92 FTD vs. 149.84 ALS, p = 0.01) and Extraversion (136.55 FTD vs. 150.53 ALS, p = 0.04) factors. This suggests that ALS patients had been, in their premorbid life, more open to new experiences, more sociable and optimistic than FTD patients. They also showed greater functional connectivity than both FTD and a control group in the Salience resting state network, over and above differences in gray matter atrophy. Finally, there was a positive correlation between premorbid Openness and functional connectivity in the Salience network across all patients, suggesting a possible association between premorbid personality and current functional organization of the brain, irrespective of the degree of atrophy.

Discussion

Our proof-of-concept results suggest that premorbid personality may eventually predispose to the development of one, rather than the other, phenotype in the FTD-ALS spectrum.

Unique functional responses differentially map onto genetic subtypes of dopamine neurons

Dopamine neurons are characterized by their response to unexpected rewards, but they also fire during movement and aversive stimuli. Dopamine neuron diversity has been observed based on molecular expression profiles; however, whether different functions map onto such genetic subtypes remains unclear. In this study, we established that three genetic dopamine neuron subtypes within the substantia nigra pars compacta, characterized by the expression of Slc17a6 (Vglut2), Calb1 and Anxa1, each have a unique set of responses to rewards, aversive stimuli and accelerations and decelerations, and these signaling patterns are highly correlated between somas and axons within subtypes. Remarkably, reward responses were almost entirely absent in the Anxa1+ subtype, which instead displayed acceleration-correlated signaling. Our findings establish a connection between functional and genetic dopamine neuron subtypes and demonstrate that molecular expression patterns can serve as a common framework to dissect dopaminergic functions.

Characterization of Behavioral Phenotypes in Heterozygous DAT Rat Based on Pedigree

Dopamine is an essential neurotransmitter whose key roles include movement control, pleasure and reward, attentional and cognitive skills, and regulation of the sleep/wake cycle. Reuptake is carried out by the dopamine transporter (DAT; DAT1 SLC6A3 gene). In order to study the effects of hyper-dopaminergia syndrome, the gene was silenced in rats. DAT-KO rats show stereotypical behavior, hyperactivity, a deficit in working memory, and an altered circadian cycle. In addition to KO rats, heterozygous (DAT-HET) rats show relative hypofunction of DAT; exact phenotypic effects are still unknown and may depend on whether the sire or the dam was KO. Our goal was to elucidate the potential importance of the parental origin of the healthy or silenced allele and its impact across generations, along with the potential variations in maternal care. We thus generated specular lines to study the effects of (grand) parental roles in inheriting the wild or mutated allele. MAT-HETs are the progeny of a KO sire and a WT dam; by breeding MAT-HET males and KO females, we obtained subjects with a DAT -/- epigenotype, named QULL, to reflect additional epigenetic DAT modulation when embryos develop within a hyper-dopaminergic KO uterus. We aimed to verify if any behavioral anomaly was introduced by a QULL (instead of KO) rat acting as a direct father or indirect maternal grandfather (or both). We thus followed epigenotypes obtained after three generations and observed actual effects on impaired maternal care of the offspring (based on pedigree). In particular, offspring of MAT-HET-dam × QULL-sire breeding showed a compulsive and obsessive phenotype. Although the experimental groups were all heterozygous, the impact of having a sire of epigenotype QULL (who developed in the uterus of a KO grand-dam) has emerged clearly. Along the generations, the effects of the DAT epigenotype on the obsessive/compulsive phenotype do vary as a function of the uterine impact on either allele in one's genealogical line.

Contributions of the Basal Ganglia to Visual Perceptual Decisions

The basal ganglia (BG) make up a prominent nexus between visual and motor-related brain regions. In contrast to the BG's well-established roles in movement control and value-based decision making, their contributions to the transformation of visual input into an action remain unclear, especially in the context of perceptual decisions based on uncertain visual evidence. This article reviews recent progress in our understanding of the BG's contributions to the formation, evaluation, and adjustment of such decisions. From theoretical and experimental perspectives, the review focuses on four key stations in the BG network, namely, the striatum, pallidum, subthalamic nucleus, and midbrain dopamine neurons, which can have different roles and together support the decision process.

Support from a Best Friend Makes People Eat Less under Stress: Evidence from Two Experiments

When experiencing acute stress, individuals often turn to eating for comfort, as it provides a sense of satiety and satisfaction that can temporarily alleviate the stressful condition. However, this may increase the risk of obesity, diabetes, cardiovascular disease. In this study, we conducted two behavioral experiments to investigate the effectiveness of social support in reducing stress-induced overeating and the mediative role of negative affect and self-efficacy (Experiment 1), as well as the role of reward sensitivity (Experiment 2). Acute stress was induced using a speech preparation task and then participants were asked to regulate their emotions and cognition, either alone or with the help of pictures and supportive sentences provided by a best friend or stranger. Participants in Experiment 1 then completed the food choice task, and participants in Experiment 2 completed the food incentive delay task and the bogus tasting task. The results of both experiments consistently showed that participants who received support from their friends reported lower levels of perceived stress, chose fewer food portions, and consumed fewer snacks during acute stress, compared to the other three groups. Further mediation analysis using the process macro revealed that the differential influence of social support on the choice of high-calorie foods was due to decreased negative affect and increased self-efficacy. This study provides valuable insights for the development of therapeutic interventions for clinical eating disorders.

Reward expectation enhances action-related activity of nigral dopaminergic and two striatal output pathways

Neurons comprising nigrostriatal system play important roles in action selection. However, it remains unclear how this system integrates recent outcome information with current action (movement) and outcome (reward or no reward) information to achieve appropriate subsequent action. We examined how neuronal activity of substantia nigra pars compacta (SNc) and dorsal striatum reflects the level of reward expectation from recent outcomes in rats performing a reward-based choice task. Movement-related activity of direct and indirect pathway striatal projection neurons (dSPNs and iSPNs, respectively) were enhanced by reward expectation, similarly to the SNc dopaminergic neurons, in both medial and lateral nigrostriatal projections. Given the classical basal ganglia model wherein dopamine stimulates dSPNs and suppresses iSPNs through distinct dopamine receptors, dopamine might not be the primary driver of iSPN activity increasing following higher reward expectation. In contrast, outcome-related activity was affected by reward expectation in line with the classical model and reinforcement learning theory, suggesting purposive effects of reward expectation.

Recording neural reward signals in the real-world using mobile-EEG and augmented reality

The electrophysiological response to rewards recorded during laboratory-based tasks has been well documented over the past two decades, yet little is known about the neural response patterns in 'real-world' settings. To address this issue, we combined a mobile-EEG system with an augmented reality headset (which blends high definition "holograms" within the real-world) to record event-related brain potentials (ERP) while participants navigated an operant chamber to find rewards. 25 participants (age = 18-43, Male=6, Female=19) were asked to choose between two floating holograms marking a west or east goal-location in a large room, and once participants reached the goal location, the hologram would turn into a reward (5 cents) or no-reward (0 cents) cue. Following the feedback cue, participants were required to return to a hologram marking the start location, and once standing in it, a 3 second counter hologram would initiate the next trial. This sequence was repeated until participants completed 200 trials. Consistent with previous research, reward feedback evoked the reward positivity, an ERP component believed to index the sensitivity of the anterior cingulate cortex to reward prediction error signals. The reward positivity peaked around 235ms post-feedback with a maximal at channel FCz (M=-2.60μV, SD=1.73μV) and was significantly different than zero (p < 0.01). At a behavioral level, participants took approximately 3.38 seconds to reach the goal-location and exhibited a general lose-shift (68.3% ± 3.5) response strategy and were slightly slower to return to the start location following negative feedback (2.43 sec) compared to positive feedback (2.38 sec), evidence of post-error slowing. Overall, these findings provide the first evidence that combining mobile-EEG with augmented reality technology is a feasible solution to enhance the ecological validity of human electrophysiological studies of goal-directed behavior and a step towards a new era of human cognitive neuroscience research that blurs the line between laboratory and reality.

Mixtures of strategies underlie rodent behavior during reversal learning

In reversal learning tasks, the behavior of humans and animals is often assumed to be uniform within single experimental sessions to facilitate data analysis and model fitting. However, behavior of agents can display substantial variability in single experimental sessions, as they execute different blocks of trials with different transition dynamics. Here, we observed that in a deterministic reversal learning task, mice display noisy and sub-optimal choice transitions even at the expert stages of learning. We investigated two sources of the sub-optimality in the behavior. First, we found that mice exhibit a high lapse rate during task execution, as they reverted to unrewarded directions after choice transitions. Second, we unexpectedly found that a majority of mice did not execute a uniform strategy, but rather mixed between several behavioral modes with different transition dynamics. We quantified the use of such mixtures with a state-space model, block Hidden Markov Model (block HMM), to dissociate the mixtures of dynamic choice transitions in individual blocks of trials. Additionally, we found that blockHMM transition modes in rodent behavior can be accounted for by two different types of behavioral algorithms, model-free or inference-based learning, that might be used to solve the task. Combining these approaches, we found that mice used a mixture of both exploratory, model-free strategies and deterministic, inference-based behavior in the task, explaining their overall noisy choice sequences. Together, our combined computational approach highlights intrinsic sources of noise in rodent reversal learning behavior and provides a richer description of behavior than conventional techniques, while uncovering the hidden states that underlie the block-by-block transitions.

Impulse control disorder in Parkinson's disease is associated with abnormal frontal value signalling

Dopaminergic medication is well established to boost reward- versus punishment-based learning in Parkinson's disease. However, there is tremendous variability in dopaminergic medication effects across different individuals, with some patients exhibiting much greater cognitive sensitivity to medication than others. We aimed to unravel the mechanisms underlying this individual variability in a large heterogeneous sample of early-stage patients with Parkinson's disease as a function of comorbid neuropsychiatric symptomatology, in particular impulse control disorders and depression. One hundred and ninety-nine patients with Parkinson's disease (138 ON medication and 61 OFF medication) and 59 healthy controls were scanned with functional MRI while they performed an established probabilistic instrumental learning task. Reinforcement learning model-based analyses revealed medication group differences in learning from gains versus losses, but only in patients with impulse control disorders. Furthermore, expected-value related brain signalling in the ventromedial prefrontal cortex was increased in patients with impulse control disorders ON medication compared with those OFF medication, while striatal reward prediction error signalling remained unaltered. These data substantiate the hypothesis that dopamine's effects on reinforcement learning in Parkinson's disease vary with individual differences in comorbid impulse control disorder and suggest they reflect deficient computation of value in medial frontal cortex, rather than deficient reward prediction error signalling in striatum. See Michael Browning (https://doi.org/10.1093/brain/awad248) for a scientific commentary on this article.

Dopamine and glutamate regulate striatal acetylcholine in decision-making

Striatal dopamine and acetylcholine are essential for the selection and reinforcement of motor actions and decision-making1. In vitro studies have revealed an intrastriatal circuit in which acetylcholine, released by cholinergic interneurons (CINs), drives the release of dopamine, and dopamine, in turn, inhibits the activity of CINs through dopamine D2 receptors (D2Rs). Whether and how this circuit contributes to striatal function in vivo is largely unknown. Here, to define the role of this circuit in a living system, we monitored acetylcholine and dopamine signals in the ventrolateral striatum of mice performing a reward-based decision-making task. We establish that dopamine and acetylcholine exhibit multiphasic and anticorrelated transients that are modulated by decision history and reward outcome. Dopamine dynamics and reward encoding do not require the release of acetylcholine by CINs. However, dopamine inhibits acetylcholine transients in a D2R-dependent manner, and loss of this regulation impairs decision-making. To determine how other striatal inputs shape acetylcholine signals, we assessed the contribution of cortical and thalamic projections, and found that glutamate release from both sources is required for acetylcholine release. Altogether, we uncover a dynamic relationship between dopamine and acetylcholine during decision-making, and reveal multiple modes of CIN regulation. These findings deepen our understanding of the neurochemical basis of decision-making and behaviour.

Hippocampal, Whole Midbrain, Red Nucleus, and Ventral Tegmental Area Volumes Are Increased by Selective Breeding for High Voluntary Wheel-Running Behavior

Uncovering relationships between neuroanatomy, behavior, and evolution are important for understanding the factors that control brain function. Voluntary exercise is one key behavior that both affects, and may be affected by, neuroanatomical variation. Moreover, recent studies suggest an important role for physical activity in brain evolution. We used a unique and ongoing artificial selection model in which mice are bred for high voluntary wheel-running behavior, yielding four replicate lines of high runner (HR) mice that run ∼3-fold more revolutions per day than four replicate nonselected control (C) lines. Previous studies reported that, with body mass as a covariate, HR mice had heavier whole brains, non-cerebellar brains, and larger midbrains than C mice. We sampled mice from generation 66 and used high-resolution microscopy to test the hypothesis that HR mice have greater volumes and/or cell densities in nine key regions from either the midbrain or limbic system. In addition, half of the mice were given 10 weeks of wheel access from weaning, and we predicted that chronic exercise would increase the volumes of the examined brain regions via phenotypic plasticity. We replicated findings that both selective breeding and wheel access increased total brain mass, with no significant interaction between the two factors. In HR compared to C mice, adjusting for body mass, both the red nucleus (RN) of the midbrain and the hippocampus (HPC) were significantly larger, and the whole midbrain tended to be larger, with no effect of wheel access nor any interactions. Linetype and wheel access had an interactive effect on the volume of the periaqueductal gray (PAG), such that wheel access increased PAG volume in C mice but decreased volume in HR mice. Neither linetype nor wheel access affected volumes of the substantia nigra, ventral tegmental area, nucleus accumbens, ventral pallidum (VP), or basolateral amygdala. We found no main effect of either linetype or wheel access on neuronal densities (numbers of cells per unit area) for any of the regions examined. Taken together, our results suggest that the increased exercise phenotype of HR mice is related to increased RN and hippocampal volumes, but that chronic exercise alone does not produce such phenotypes.

Reward prediction error in learning-related behaviors

Learning is a complex process, during which our opinions and decisions are easily changed due to unexpected information. But the neural mechanism underlying revision and correction during the learning process remains unclear. For decades, prediction error has been regarded as the core of changes to perception in learning, even driving the learning progress. In this article, we reviewed the concept of reward prediction error, and the encoding mechanism of dopaminergic neurons and the related neural circuities. We also discussed the relationship between reward prediction error and learning-related behaviors, including reversal learning. We then demonstrated the evidence of reward prediction error signals in several neurological diseases, including Parkinson's disease and addiction. These observations may help to better understand the regulatory mechanism of reward prediction error in learning-related behaviors.

Non-invasive stimulation reveals ventromedial prefrontal cortex function in reward prediction and reward processing

Introduction

Studies suggest an involvement of the ventromedial prefrontal cortex (vmPFC) in reward prediction and processing, with reward-based learning relying on neural activity in response to unpredicted rewards or non-rewards (reward prediction error, RPE). Here, we investigated the causal role of the vmPFC in reward prediction, processing, and RPE signaling by transiently modulating vmPFC excitability using transcranial Direct Current Stimulation (tDCS).

Methods

Participants received excitatory or inhibitory tDCS of the vmPFC before completing a gambling task, in which cues signaled varying reward probabilities and symbols provided feedback on monetary gain or loss. We collected self-reported and evaluative data on reward prediction and processing. In addition, cue-locked and feedback-locked neural activity via magnetoencephalography (MEG) and pupil diameter using eye-tracking were recorded.

Results

Regarding reward prediction (cue-locked analysis), vmPFC excitation (versus inhibition) resulted in increased prefrontal activation preceding loss predictions, increased pupil dilations, and tentatively more optimistic reward predictions. Regarding reward processing (feedback-locked analysis), vmPFC excitation (versus inhibition) resulted in increased pleasantness, increased vmPFC activation, especially for unpredicted gains (i.e., gain RPEs), decreased perseveration in choice behavior after negative feedback, and increased pupil dilations.

Discussion

Our results support the pivotal role of the vmPFC in reward prediction and processing. Furthermore, they suggest that transient vmPFC excitation via tDCS induces a positive bias into the reward system that leads to enhanced anticipation and appraisal of positive outcomes and improves reward-based learning, as indicated by greater behavioral flexibility after losses and unpredicted outcomes, which can be seen as an improved reaction to the received feedback.

Norepinephrine release in the cerebellum contributes to aversive learning

The modulation of dopamine release from midbrain projections to the striatum has long been demonstrated in reward-based learning, but the synaptic basis of aversive learning is far less characterized. The cerebellum receives axonal projections from the locus coeruleus, and norepinephrine release is implicated in states of arousal and stress, but whether aversive learning relies on plastic changes in norepinephrine release in the cerebellum is unknown. Here we report that in mice, norepinephrine is released in the cerebellum following an unpredicted noxious event (a foot-shock) and that this norepinephrine release is potentiated powerfully with fear acquisition as animals learn that a previously neutral stimulus (tone) predicts the aversive event. Importantly, both chemogenetic and optogenetic inhibition of the locus coeruleus-cerebellum pathway block fear memory without impairing motor function. Thus, norepinephrine release in the cerebellum is modulated by experience and underlies aversive learning.

Modeling intentionality in the human brain

This paper is focusing on a rather neglected issue that concerns both aspects of philosophy and neurobiology in relation to the concept of intentionality. Intentionality is concerned with the 'directedness' or 'aboutness' of mental phenomena towards an object. Despite the fact that in philosophy both concepts of aboutness and directedness are conceptually identical with intentionality, a careful neuroscientific approach can demonstrate that these two phenomena represent two distinct conceptual and neurobiological aspects of intentionality with complementary functions. We described the interaction between a series of intentionality and pathogenetic psychobiological factors, the corresponding brain topography, and the resulting clinical manifestation and psychopathology. A permanent failure of intentionality dominates in psychosis, which includes an inappropriateness of the intentional object or connection, from the outset, or even from the prodromal phase of the disorder. Affective disorders may result from imprecise interoceptive prediction error signals, due to a confused identification of the intentional object. In suicidal patients there is an emotional intentionality failure, characterized by an absence of intentional object or a loss of conscious access to normal intentional objects. We may model an 'intentional system' as a higher order system, with a monitoring and regulatory role attributed to the brain and behavior. Also, we may consider mental disorders as the result of a radical disruption of intentionality, due to an inappropriateness or lack of the intentional object or due to an inappropriate connection in some points of the suggested brain pathways of intentionality.

A reward effect on memory retention, consolidation, and generalization?

Reward improves memory through both encoding and consolidation processes. In this preregistered study, we tested whether reward effects on memory generalize from high-rewarded items to low-rewarded but episodically related items. Fifty-nine human volunteers incidentally encoded associations between unique objects and repeated scenes. Some scenes typically yielded high reward, whereas others typically yielded low reward. Memory was tested immediately after encoding (n = 29) or the next day (n = 30). Overall, reward had only a limited influence on memory. It did not enhance consolidation and its effect did not generalize to episodically related stimuli. We thus contribute to understanding the boundary conditions of reward effects on memory.

Disentangling the role of NAc D1 and D2 cells in hedonic eating

Overeating is driven by both the hedonic component ('liking') of food, and the motivation ('wanting') to eat it. The nucleus accumbens (NAc) is a key brain center implicated in these processes, but how distinct NAc cell populations encode 'liking' and 'wanting' to shape overconsumption remains unclear. Here, we probed the roles of NAc D1 and D2 cells in these processes using cell-specific recording and optogenetic manipulation in diverse behavioral paradigms that disentangle reward traits of 'liking' and 'wanting' related to food choice and overeating in healthy mice. Medial NAc shell D2 cells encoded experience-dependent development of 'liking', while D1 cells encoded innate 'liking' during the first food taste. Optogenetic control confirmed causal links of D1 and D2 cells to these aspects of 'liking'. In relation to 'wanting', D1 and D2 cells encoded and promoted distinct aspects of food approach: D1 cells interpreted food cues while D2 cells also sustained food-visit-length that facilitates consumption. Finally, at the level of food choice, D1, but not D2, cell activity was sufficient to switch food preference, programming subsequent long-lasting overconsumption. By revealing complementary roles of D1 and D2 cells in consumption, these findings assign neural bases to 'liking' and 'wanting' in a unifying framework of D1 and D2 cell activity.