Predictive reward signal of dopamine neurons

Novel multi-modal methodology to investigate placebo response in major depressive disorder

The neurobiological mechanisms underlying the placebo phenomenon in patients with major depressive disorder (MDD) remain largely unknown. The progressive rise in rates of placebo responses within clinical trials over the past two decades may impede the detection of a true signal and thus present a major obstacle in new treatment development. Understanding the mechanisms would have several important implications, including (1) identifying biomarkers of placebo responders (thereby identifying those individuals who could benefit therapeutically from such interventions), (2) opening new avenues for manipulating such mechanisms to maximize symptom reduction, and (3) refining treatments with approaches that decrease (in clinical trials) or increase (in clinical practice) the placebo response. Here we investigated the research question: is the dopaminergic system one of the neurobiological underpinnings of the placebo response within MDD? Inspired by preclinical and clinical findings that have implicated dopamine in the occurrence, prediction, and expectation of reward, we hypothesized that dopaminergic activity in the mesolimbic system is a critical mediator of placebo response in MDD. To test this hypothesis, we designed a double-blind, placebo-controlled, sequential parallel comparison design clinical trial aimed at maximizing placebo antidepressant response. We integrated behavioral, imaging, and hemodynamic probes of mesocorticolimbic dopaminergic pathways within the context of manipulations of psychological constructs previously linked to placebo responses (e.g., expectation of improvement). The aim of this manuscript is to present the rationale of the study design and to demonstrate how a cross-modal methodology may be utilized to investigate the role of reward circuitry in placebo response in MDD.

Attention to food stimuli in binge eating disorder: Electrophysiological evidence

Attentional biases towards food play an important role in the pathology of binge eating disorder (BED). Later stage electrophysiological potentials (P300, late positive potential) present promising markers of motivated attention with high temporal, albeit low spatial resolution. Complementing this, the N2pc is an earlier-latency component providing the possibility of more directly analyzing visuospatial attention. Therefore, we tested a group with BED (N = 60), as well as an overweight (OW; N = 28) and normal weight (NW; N = 30) group without BED in a Go/No-Go paradigm using food and nonfood distractor images. Only the OW group in exclusively the Go trials displayed a stronger spatial attention allocation towards nonfood distractors as evidenced by an increased N2pc amplitude. In the P300's time window, the OW group displayed no attentional bias towards food and the NW group only did so in the absence of a target. Solely the BED group allocated more motivated attention towards food distractors both in Go and No-Go trials. In the following late positive potential (LPP), the OW group exhibited a general attentional bias towards food distractors, while the BED group only did so in the absence of a target. These results are discussed in light of the incentive sensitization theory and a potential early attentional suppression of potent distractors.

Kappa opioid receptor availability predicts severity of anhedonia in schizophrenia

The kappa opioid receptor (KOR) and its endogenous agonist dynorphin have been implicated in multiple psychiatric conditions including psychotic disorders. We tested the hypotheses that kappa expression is elevated and associated with psychotic symptoms in schizophrenia. We measured kappa expression in unmedicated patients with schizophrenia (7 female, 6 male) and matched controls (7 female, 6 male) with positron emission tomography (PET). We also acquired a measurement of cumulative dopamine activity over the life span in the same subjects using neuromelanin sensitive MRI. We hypothesized that neuromelanin accumulation would be higher in patients than controls and that in patients there would be a positive association between KOR availability and neuromelanin accumulation. Fourteen patients and thirteen controls were enrolled. Whole brain dynamic PET imaging data using the KOR selective tracer [18F]LY245998 were acquired. Distribution volume (VT) was measured with region of interest analysis in 14 brain regions. Neuromelanin accumulation in midbrain dopaminergic nuclei was assessed in the same subjects. Positive and negative symptoms were measured by a clinical psychologist. We did not observe group level differences in KOR expression, neuromelanin accumulation or relationships of these to positive symptoms. Unexpectedly, we did observe strong positive associations between KOR expression and symptoms of anhedonia in the patients (Pearson r > 0.7, uncorrected p < 0.01 in 8 cortical brain regions). We also observed moderate associations between KOR expression and neuromelanin levels in patients. In conclusion, we did not observe a relationship between kappa and symptoms of psychosis but the observed relationship to the negative symptom of anhedonia is in line with recent work testing kappa antagonism as a therapy for anhedonia in depression.

Anhedonia is associated with higher functional connectivity between the nucleus accumbens and paraventricular nucleus of thalamus

BACKGROUND

Anhedonia stands as a life-threatening transdiagnostic feature of many mental illnesses, most notably major depression and involves neural circuits for processing reward information. The paraventricular nucleus of the thalamus (PVT) is associated with reward-seeking behavior, however, links between the PVT circuit and anhedonia have not been investigated in humans.

METHODS

In a sample of adults with and without psychiatric symptoms (n = 75, 18-41 years, 55 female), we generated an anhedonia factor score for each participant using a latent factor analysis, utilizing data from depression and anxiety assessments. Functional connectivity between the PVT and the nucleus accumbens (NAc) was calculated from high-resolution (1.5 mm) resting state fMRI.

RESULTS

Anhedonia factor scores showed a positive relationship with functional connectivity between the PVT and the NAc, principally in males and in those with psychiatric symptoms. In males, connectivity between other midline thalamic nuclei and the NAc did not show these relationships, suggesting that this link may be specific to PVT.

LIMITATIONS

This cohort was originally recruited to study depression and not anhedonia per se. The distribution of male and female participants in our cohort was not equal. Partial acquisition in high-resolution fMRI scans restricted regions of interest outside of the thalamus and reward networks.

CONCLUSIONS

We report evidence that anhedonia is associated with enhanced functional connectivity between the PVT and the NAc, regions that are relevant to reward processing. These results offer clues as to the potential prevention and prevention and treatment of anhedonia.

Representational spaces in orbitofrontal and ventromedial prefrontal cortex: task states, values, and beyond

The orbitofrontal cortex (OFC) and ventromedial-prefrontal cortex (vmPFC) play a key role in decision-making and encode task states in addition to expected value. We review evidence suggesting a connection between value and state representations and argue that OFC / vmPFC integrate stimulus, context, and outcome information. Comparable encoding principles emerge in late layers of deep reinforcement learning (RL) models, where single nodes exhibit similar forms of mixed-selectivity, which enables flexible readout of relevant variables by downstream neurons. Based on these lines of evidence, we suggest that outcome-maximization leads to complex representational spaces that are insufficiently characterized by linear value signals that have been the focus of most prior research on the topic. Major outstanding questions concern the role of OFC/ vmPFC in learning across tasks, in encoding of task-irrelevant aspects, and the role of hippocampus-PFC interactions.

Cannabidiol Plays a Modulatory Function on the Methamphetamine-Induced Reward Through Hippocampal D2-Like Dopamine Receptors

Methamphetamine (METH), a stimulant that is extremely addictive, directly affects the central nervous system. METH's abuse and consumption are directly linked to mental illnesses, psychosis, and behavioral and cognitive impairments. It may disrupt the reward system and dopaminergic transmission. METH's rewarding qualities are associated with a rise in dopamine. Additionally, cannabidiol (CBD), one of the primary cannabinoid components of the cannabis plant, significantly affects dopaminergic transmission and may aid in reward- and addiction-related behaviors. To shed light on the role of the D2-like dopamine receptor (D2R) in the hippocampal dentate gyrus (DG), the present study examined the effects of CBD on the acquisition and expression of the conditioned place preference (CPP) induced by METH. The function of D2R was ascertained by delivering Sulpiride microinjections, as a D2R antagonist Sulpiride (0.25, 1, and 4 μg/0.5 μL DMSO12%) into the DG. Moreover, an intracerebroventricular injection of CBD at a dose of 10 μg/5 μL for CPP acquisition and 50 μg/5 μL for CPP expression was given to rats. According to the current research, CBD dramatically reduced the acquisition and expression of CPP resulting from METH. However, Sulpiride suppressed the effect of CBD on METH-induced CPP acquisition and expression, with a greater impact on expression experiments. Ultimately, this study proposed that the expression experiment of METH-induced CPP appears to be heavily dependent on D2R in the DG.

Monosynaptic Inputs to Ventral Tegmental Area Glutamate and GABA Co-transmitting Neurons

A unique population of ventral tegmental area (VTA) neurons co-transmits glutamate and GABA. However, the circuit inputs to VTA VGluT2+VGaT+ neurons are unknown, limiting our understanding of their functional capabilities. By coupling monosynaptic rabies tracing with intersectional genetic targeting in male and female mice, we found that VTA VGluT2+VGaT+ neurons received diverse brainwide inputs. The largest numbers of monosynaptic inputs to VTA VGluT2+VGaT+ neurons were from superior colliculus (SC), lateral hypothalamus (LH), midbrain reticular nucleus, and periaqueductal gray, whereas the densest inputs relative to brain region volume were from the dorsal raphe nucleus, lateral habenula, and VTA. Based on these and prior data, we hypothesized that LH and SC inputs were from glutamatergic neurons. Optical activation of glutamatergic LH neurons activated VTA VGluT2+VGaT+ neurons regardless of stimulation frequency and resulted in flee-like ambulatory behavior. In contrast, optical activation of glutamatergic SC neurons activated VTA VGluT2+VGaT+ neurons for a brief period of time at high frequency and resulted in head rotation and arrested ambulatory behavior (freezing). Stimulation of glutamatergic LH neurons, but not glutamatergic SC neurons, was associated with VTA VGluT2+VGaT+ footshock-induced activity and inhibition of LH glutamatergic neurons disrupted VTA VGluT2+VGaT+ tailshock-induced activity. We interpret these results such that inputs to VTA VGluT2+VGaT+ neurons may integrate diverse signals related to the detection and processing of motivationally salient outcomes.

Parabrachial Calca neurons mediate second-order conditioning

Learning to associate cues, both directly and indirectly, with biologically significant events is essential for survival. Second-order conditioning (SOC) involves forming an association between a previously reinforced conditioned stimulus (CS1) and a new conditioned stimulus (CS2) without the presence of an unconditioned stimulus (US). The neural substrates mediating SOC, however, remain unclear. Parabrachial Calca neurons, which react to the noxious US, also respond to a CS after pairing with a US, suggesting that Calca neurons mediate SOC. We established an aversive SOC behavioral paradigm in mice and monitored Calca neuron activity via single-cell calcium imaging during conditioning and subsequent recall phases. These neurons were activated by both CS1 and CS2 after SOC. Chemogenetically inhibiting Calca neurons during CS1-CS2 pairing attenuated SOC. Thus, reactivation of the US pathway by a learned CS plays an important role in forming the association between the old and a new CS, promoting the formation of second-order memories.

Neural Circuitries between the Brain and Peripheral Solid Tumors

The recent discovery of the pivotal role of the central nervous system in controlling tumor initiation and progression has opened a new field of research. Increasing evidence suggests a bidirectional interaction between the brain and tumors. The brain influences the biological behavior of tumor cells through complex neural networks involving the peripheral nervous system, the endocrine system, and the immune system, whereas tumors can establish local autonomic and sensory neural networks to transmit signals into the central nervous system, thereby affecting brain activity. This review aims to summarize the latest research in brain-tumor cross-talk, exploring neural circuitries between the brain and various peripheral solid tumors, analyzing the roles in tumor development and the related molecular mediators and pathologic mechanisms, and highlighting the critical impact on the understanding of cancer biology. Enhanced understanding of reciprocal communication between the brain and tumors will establish a solid theoretical basis for further research and could open avenues for repurposing psychiatric interventions in cancer treatment.

Medicalization of Neurodivergence and the Embodied Experience of ADHD

Medicalization shapes, and in some cases legitimizes, individuals' embodied experiences even as it molds the landscape of healthcare and treatment. In this essay I provide a layered account that moves between my experiences as a neurodivergent person and academic theorizing to explore how processes of medicalization inform public discourses and personal sensemaking. In the case of ADHD, medicalization has contributed to societal narratives that focus on symptoms of hyperactivity rather than the etiology of dopamine dysregulation. Such narratives fail to fully account for the lived experience of ADHD and inadvertently stigmatize neurodivergent individuals. I urge scholars and practitioners to direct more attention to the communicative dimensions of medicalization including both the rhetorical nature of the diagnostic process and how diagnoses, in turn, are rhetorically framed with varying consequences.

Valence-dependent dopaminergic modulation during reversal learning in Parkinson's disease: A neurocomputational approach

Reinforcement learning, crucial for behavior in dynamic environments, is driven by rewards and punishments, modulated by dopamine (DA) changes. This study explores the dopaminergic system's influence on learning, particularly in Parkinson's disease (PD), where medication leads to impaired adaptability. Highlighting the role of tonic DA in signaling the valence of actions, this research investigates how DA affects response vigor and decision-making in PD. DA not only influences reward and punishment learning but also indicates the cognitive effort level and risk propensity in actions, which are essential for understanding and managing PD symptoms. In this work, we adapt our existing neurocomputational model of basal ganglia (BG) to simulate two reversal learning tasks proposed by Cools et al. We first optimized a Hebb rule for both probabilistic and deterministic reversal learning, conducted a sensitivity analysis (SA) on parameters related to DA effect, and compared performances between three groups: PD-ON, PD-OFF, and control subjects. In our deterministic task simulation, we explored switch error rates after unexpected task switches and found a U-shaped relationship between tonic DA levels and switch error frequency. Through SA, we classify these three groups. Then, assuming that the valence of the stimulus affects the tonic levels of DA, we were able to reproduce the results by Cools et al. As for the probabilistic task simulation, our results are in line with clinical data, showing similar trends with PD-ON, characterized by higher tonic DA levels that are correlated with increased difficulty in both acquisition and reversal tasks. Our study proposes a new hypothesis: valence, signaled by tonic DA levels, influences learning in PD, confirming the uncorrelation between phasic and tonic DA changes. This hypothesis challenges existing paradigms and opens new avenues for understanding cognitive processes in PD, particularly in reversal learning tasks.

Optogenetic activation of mesencephalic projections to the nucleus accumbens shell impairs probabilistic reversal learning by disrupting learning from negative reinforcement

Cognitive flexibility, the capacity to adapt behaviour to changes in the environment, is impaired in a range of brain disorders, including schizophrenia and Parkinson's disease. Putative neural substrates of cognitive flexibility include mesencephalic pathways to the ventral striatum (VS) and dorsomedial striatum (DMS), hypothesized to encode learning signals needed to maximize rewarded outcomes during decision-making. However, it is unclear whether mesencephalic projections to the ventral and dorsal striatum are distinct in their contribution to flexible reward-related learning. Here, rats acquired a two-choice spatial probabilistic reversal learning (PRL) task, reinforced on an 80%|20% basis (correct|incorrect responses) that assessed the flexibility of behaviour to repeated reversals of response-outcome contingencies. We report that optogenetic stimulation of projections from the ventral tegmental area (VTA) to the nucleus accumbens shell (NAcS) in the VS significantly impaired reversal learning when optical stimulation was temporally aligned with negative feedback (i.e., reward omission). VTA → NAcS stimulation during other phases of the behavioural task was without significant effect. Optogenetic stimulation of projection neurons from the substantia nigra (SN) to the DMS, aligned either with reward receipt or omission or prior to making a choice, had no significant effect on reversal learning. These findings are consistent with the notion that increased activity in the VTA → NAcS pathway disrupts behavioural flexibility by impairing learning from negative reinforcement.

Dopamine and Norepinephrine Differentially Mediate the Exploration-Exploitation Tradeoff

Dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision-making processes. Although two neuromodulators share a synthesis pathway and are coactivated under states of arousal, they engage in distinct circuits and modulatory roles. However, the specific role of each neuromodulator in decision-making, in particular the exploration-exploitation tradeoff, remains unclear. Revealing how each neuromodulator contributes to exploration-exploitation tradeoff is important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration, a direct comparison using the same dynamic decision-making task is needed. Here, we ran male and female mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA antagonist (flupenthixol), a nonselective DA agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol) and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine on exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. The modulatory effect of beta-noradrenergic receptor activity on exploration was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via sensitivity to outcome. Together, these findings suggested that the mechanisms that govern the exploration-exploitation transition are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

New Drug Treatments for Schizophrenia: A Review of Approaches to Target Circuit Dysfunction

Schizophrenia is a leading cause of global disease burden. Current drug treatments are associated with significant side effects and have limited efficacy for many patients, highlighting the need to develop new approaches that target other aspects of the neurobiology of schizophrenia. Preclinical, in vivo imaging, postmortem, genetic, and pharmacological studies have highlighted the key role of cortical GABAergic (gamma-aminobutyric acidergic)-glutamatergic microcircuits and their projections to subcortical dopaminergic circuits in the pathoetiology of negative, cognitive, and psychotic symptoms. Antipsychotics primarily act downstream of the dopaminergic component of this circuit. However, multiple drugs are currently in development that could target other elements of this circuit to treat schizophrenia. These include drugs for GABAergic or glutamatergic targets, including glycine transporters, D-amino acid oxidase, sodium channels, or potassium channels. Other drugs in development are likely to primarily act on pathways that regulate the dopaminergic system, such as muscarinic or trace amine receptors or 5-HT2A receptors, while PDE10A inhibitors are being developed to modulate the downstream consequences of dopaminergic dysfunction. Our review considers where new drugs may act on this circuit and their latest clinical trial evidence in terms of indication, efficacy, and side effects. Limitations of the circuit model, including whether there are neurobiologically distinct subgroups of patients, and future directions are also considered. Several drugs based on the mechanisms reviewed have promising clinical data, with the muscarinic agonist KarXT most advanced. If these drugs are approved for clinical use, they have the potential to revolutionize understanding of the pathophysiology and treatment of schizophrenia.

Neuronal activity in the ventral tegmental area during goal-directed navigation recorded by low-curvature microelectrode arrays

Navigating toward destinations with rewards is a common behavior among animals. The ventral tegmental area (VTA) has been shown to be responsible for reward coding and reward cue learning, and its response to other variables, such as kinematics, has also been increasingly studied. These findings suggest a potential relationship between animal navigation behavior and VTA activity. However, the deep location and small volume of the VTA pose significant challenges to the precision of electrode implantation, increasing the uncertainty of measurement results during animal navigation and thus limiting research on the role of the VTA in goal-directed navigation. To address this gap, we innovatively designed and fabricated low-curvature microelectrode arrays (MEAs) via a novel backside dry etching technique to release residual stress. Histological verification confirmed that low-curvature MEAs indeed improved electrode implantation precision. These low-curvature MEAs were subsequently implanted into the VTA of the rats to observe their electrophysiological activity in a freely chosen modified T-maze. The results of the behavioral experiments revealed that the rats could quickly learn the reward probability corresponding to the left and right paths and that VTA neurons were deeply involved in goal-directed navigation. Compared with those in no-reward trials, VTA neurons in reward trials presented a significantly greater firing rate and larger local field potential (LFP) amplitude during the reward-consuming period. Notably, we discovered place fields mapped by VTA neurons, which disappeared or were reconstructed with changes in the path-outcome relationship. These results provide new insights into the VTA and its role in goal-directed navigation. Our designed and fabricated low-curvature microelectrode arrays can serve as a new device for precise deep brain implantation in the future.

Dopamine release plateau and outcome signals in dorsal striatum contrast with classic reinforcement learning formulations

We recorded dopamine release signals in centromedial and centrolateral sectors of the striatum as mice learned consecutive versions of visual cue-outcome conditioning tasks. Dopamine release responses differed for the centromedial and centrolateral sites. In neither sector could these be accounted for by classic reinforcement learning alone as classically applied to the activity of nigral dopamine-containing neurons. Medially, cue responses ranged from initial sharp peaks to modulated plateau responses; outcome (reward) responses during cue conditioning were minimal or, initially, negative. At centrolateral sites, by contrast, strong, transient dopamine release responses occurred at both cue and outcome. Prolonged, plateau release responses to cues emerged in both regions when discriminative behavioral responses became required. At most sites, we found no evidence for a transition from outcome signaling to cue signaling, a hallmark of temporal difference reinforcement learning as applied to midbrain dopaminergic neuronal activity. These findings delineate a reshaping of striatal dopamine release activity during learning and suggest that current views of reward prediction error encoding need review to accommodate distinct learning-related spatial and temporal patterns of striatal dopamine release in the dorsal striatum.

Pre-choice midbrain fluctuations affect self-control in food choice: A functional magnetic resonance imaging (fMRI) study

Recent studies have shown that spontaneous pre-stimulus fluctuations in brain activity affect higher-order cognitive processes, including risky decision-making, cognitive flexibility, and aesthetic judgments. However, there is currently no direct evidence to suggest that pre-choice activity influences value-based decisions that require self-control. We examined the impact of fluctuations in pre-choice activity in key regions of the reward system on self-control in food choice. In the functional magnetic resonance imaging (fMRI) scanner, 49 participants made 120 food choices that required self-control in high and low working memory load conditions. The task was designed to ensure that participants were cognitively engaged and not thinking about upcoming choices. We defined self-control success as choosing a food item that was healthier over one that was tastier. The brain regions of interest (ROIs) were the ventral tegmental area (VTA), putamen, nucleus accumbens (NAc), and caudate nucleus. For each participant and condition, we calculated the mean activity in the 3-s interval preceding the presentation of food stimuli in successful and failed self-control trials. These activities were then used as predictors of self-control success in a fixed-effects logistic regression model. The results indicate that increased pre-choice VTA activity was linked to a higher probability of self-control success in a subsequent food-choice task within the low-load condition, but not in the high-load condition. We posit that pre-choice fluctuations in VTA activity change the reference point for immediate (taste) reward evaluation, which may explain our finding. This suggests that the neural context of decisions may be a key factor influencing human behavior.

A mismatch between striatal cholinergic pauses and dopaminergic reward prediction errors

Striatal acetylcholine and dopamine critically regulate movement, motivation, and reward-related learning. Pauses in cholinergic interneuron (CIN) firing are thought to coincide with dopamine pulses encoding reward prediction errors (RPE) to jointly enable synaptic plasticity. Here, we examine the firing of identified CINs during reward-guided decision-making in freely moving rats and compare this firing to dopamine release. Relationships between CINs, dopamine, and behavior varied strongly by subregion. In the dorsal-lateral striatum, a Go! cue evoked burst-pause CIN spiking, followed by a brief dopamine pulse that was unrelated to RPE. In the dorsal-medial striatum, this cue evoked only a CIN pause, that was curtailed by a movement-selective rebound in firing. Finally, in the ventral striatum, a reward cue evoked RPE-coding increases in both dopamine and CIN firing, without a consistent pause. Our results demonstrate a spatial and temporal dissociation between CIN pauses and dopamine RPE signals and will inform future models of striatal information processing under both normal and pathological conditions.

Distinct dopaminergic spike-timing-dependent plasticity rules are suited to different functional roles

Various mathematical models have been formulated to describe the changes in synaptic strengths resulting from spike-timing-dependent plasticity (STDP). A subset of these models include a third factor, dopamine, which interacts with spike timing to contribute to plasticity at specific synapses, notably those from cortex to striatum at the input layer of the basal ganglia. Theoretical work to analyze these plasticity models has largely focused on abstract issues, such as the conditions under which they may promote synchronization and the weight distributions induced by inputs with simple correlation structures, rather than on scenarios associated with specific tasks, and has generally not considered dopamine-dependent forms of STDP. In this paper we introduce three forms of dopamine-modulated STDP adapted from previously proposed plasticity rules. We then analyze, mathematically and with simulations, their performance in three biologically relevant scenarios. We test the ability of each of the three models to maintain its weights in the face of noise and to complete simple reward prediction and action selection tasks, studying the learned weight distributions and corresponding task performance in each setting. Interestingly, we find that each plasticity rule is well suited to a subset of the scenarios studied but falls short in others. Different tasks may therefore require different forms of synaptic plasticity, yielding the prediction that the precise form of the STDP mechanism present may vary across regions of the striatum, and other brain areas impacted by dopamine, that are involved in distinct computational functions.

Investigating Transfer Learning in Noisy Environments: A Study of Predecessor and Successor Features in Spatial Learning Using a T-Maze

In this study, we investigate the adaptability of artificial agents within a noisy T-maze that use Markov decision processes (MDPs) and successor feature (SF) and predecessor feature (PF) learning algorithms. Our focus is on quantifying how varying the hyperparameters, specifically the reward learning rate (αr) and the eligibility trace decay rate (λ), can enhance their adaptability. Adaptation is evaluated by analyzing the hyperparameters of cumulative reward, step length, adaptation rate, and adaptation step length and the relationships between them using Spearman's correlation tests and linear regression. Our findings reveal that an αr of 0.9 consistently yields superior adaptation across all metrics at a noise level of 0.05. However, the optimal setting for λ varies by metric and context. In discussing these results, we emphasize the critical role of hyperparameter optimization in refining the performance and transfer learning efficacy of learning algorithms. This research advances our understanding of the functionality of PF and SF algorithms, particularly in navigating the inherent uncertainty of transfer learning tasks. By offering insights into the optimal hyperparameter configurations, this study contributes to the development of more adaptive and robust learning algorithms, paving the way for future explorations in artificial intelligence and neuroscience.

Consensus Paper: Cerebellum and Reward

Cerebellum is a key-structure for the modulation of motor, cognitive, social and affective functions, contributing to automatic behaviours through interactions with the cerebral cortex, basal ganglia and spinal cord. The predictive mechanisms used by the cerebellum cover not only sensorimotor functions but also reward-related tasks. Cerebellar circuits appear to encode temporal difference error and reward prediction error. From a chemical standpoint, cerebellar catecholamines modulate the rate of cerebellar-based cognitive learning, and mediate cerebellar contributions during complex behaviours. Reward processing and its associated emotions are tuned by the cerebellum which operates as a controller of adaptive homeostatic processes based on interoceptive and exteroceptive inputs. Lobules VI-VII/areas of the vermis are candidate regions for the cortico-subcortical signaling pathways associated with loss aversion and reward sensitivity, together with other nodes of the limbic circuitry. There is growing evidence that the cerebellum works as a hub of regional dysconnectivity across all mood states and that mental disorders involve the cerebellar circuitry, including mood and addiction disorders, and impaired eating behaviors where the cerebellum might be involved in longer time scales of prediction as compared to motor operations. Cerebellar patients exhibit aberrant social behaviour, showing aberrant impulsivity/compulsivity. The cerebellum is a master-piece of reward mechanisms, together with the striatum, ventral tegmental area (VTA) and prefrontal cortex (PFC). Critically, studies on reward processing reinforce our view that a fundamental role of the cerebellum is to construct internal models, perform predictions on the impact of future behaviour and compare what is predicted and what actually occurs.

Serial engagement of distinct motor learning mechanisms to alter walking after stroke

This study asked if combining different motor learning mechanisms-adaptation and reinforcement-could produce immediate improvements in over ground walking after stroke. Fifteen adults with stroke engaged in three conditions: (1) reinforcement following adaptation, (2) reinforcement alone, and (3) adaptation alone. Adaptation involved split-belt treadmill walking to produce after-effects that reduce step asymmetry. Reinforcement involved the use of real-time auditory feedback about step length asymmetry. Auditory feedback was binary, signaling whether steps were asymmetric or equal, but not whether to shorten or lengthen either step. Change in step length asymmetry was the outcome assessed during over ground walking. Reinforcement following adaptation led to reductions in step length asymmetry that persisted into an immediate retention period. Importantly, it led to the desired pattern of lengthening the shorter step in a majority of participants. Reinforcement alone led to no significant change in step length asymmetry, and sometimes produced a non-optimal pattern of shortening the longer step. Our control condition of adaptation alone led to more transient reductions in step length asymmetry. These findings reveal the potential for utilizing serial delivery of adaptation and reinforcement to influence a complex movement in the real-world context of over ground walking, in people with stroke.

Distinct dynamics and intrinsic properties in ventral tegmental area populations mediate reward association and motivation

Ventral tegmental area (VTA) dopamine neurons regulate reward-related associative learning and reward-driven motivated behaviors, but how these processes are coordinated by distinct VTA neuronal subpopulations remains unresolved. Here, we compare the contribution of two primarily dopaminergic and largely non-overlapping VTA subpopulations, all VTA dopamine neurons and VTA GABAergic neurons of the mouse midbrain, to these processes. We find that the dopamine subpopulation that projects to the nucleus accumbens (NAc) core preferentially encodes reward-predictive cues and prediction errors. In contrast, the subpopulation that projects to the NAc shell preferentially encodes goal-directed actions and relative reward anticipation. VTA GABA neuron activity strongly contrasts VTA dopamine population activity and preferentially encodes reward outcome and retrieval. Electrophysiology, targeted optogenetics, and whole-brain input mapping reveal multiple convergent sources that contribute to the heterogeneity among VTA dopamine subpopulations that likely underlies their distinct encoding of reward-related associations and motivation that defines their functions in these contexts.

State of the Art in Sub-Phenotyping Midbrain Dopamine Neurons

Dopaminergic neurons in the ventral tegmental area (VTA) and the substantia nigra pars compacta (SNpc) comprise around 75% of all dopaminergic neurons in the human brain. While both groups of dopaminergic neurons are in close proximity in the midbrain and partially overlap, development, function, and impairments in these two classes of neurons are highly diverse. The molecular and cellular mechanisms underlying these differences are not yet fully understood, but research over the past decade has highlighted the need to differentiate between these two classes of dopaminergic neurons during their development and in the mature brain. This differentiation is crucial not only for understanding fundamental circuitry formation in the brain but also for developing therapies targeted to specific dopaminergic neuron classes without affecting others. In this review, we summarize the state of the art in our understanding of the differences between the dopaminergic neurons of the VTA and the SNpc, such as anatomy, structure, morphology, output and input, electrophysiology, development, and disorders, and discuss the current technologies and methods available for studying these two classes of dopaminergic neurons, highlighting their advantages, limitations, and the necessary improvements required to achieve more-precise therapeutic interventions.

Stimulus valence moderates self-learning

Self-relevance has been demonstrated to impair instrumental learning. Compared to unfamiliar symbols associated with a friend, analogous stimuli linked with the self are learned more slowly. What is not yet understood, however, is whether this effect extends beyond arbitrary stimuli to material with intrinsically meaningful properties. Take, for example, stimulus valence an established moderator of self-bias. Does the desirability of to-be-learned material influence self-learning? Here, in conjunction with computational modelling (i.e. Reinforcement Learning Drift Diffusion Model analysis), a probabilistic selection task was used to establish if and how stimulus valence (i.e. desirable/undesirable posters) impacts the acquisition of knowledge relating to object-ownership (i.e. owned-by-self vs. owned-by-friend). Several interesting results were observed. First, undesirable posters were learned more rapidly for self compared to friend, an effect that was reversed for desirable posters. Second, learning rates were accompanied by associated differences in reward sensitivity toward desirable and undesirable choice selections as a function of ownership. Third, decisional caution was greater for self-relevant (vs. friend relevant) responses. Collectively, these findings inform understanding of self-function and how valence and stimulus relevance mutually influence probabilistic learning.

Neural processing of sweet taste in reward regions is reduced following bariatric surgery

OBJECTIVE

Bariatric surgery reduces sweet-liking, but mechanisms remain unclear. We examined related brain responses.

METHODS

A total of 24 nondiabetic bariatric surgery and 21 control participants with normal weight to overweight were recruited for an observational controlled cohort study. They underwent sucrose taste testing outside the scanner followed by stimulation with 0.40M and 0.10M sucrose compared with water during functional magnetic resonance imaging. A total of 21 bariatric participants repeated these procedures after surgery.

RESULTS

Perceived sweet intensity was not different among the control, presurgery, or postsurgery groups. Bariatric participants' preferred sweet concentration decreased after surgery (0.52M to 0.29M; p = 0.008). Brain reward system (ventral tegmental area, ventral striatum, and orbitofrontal cortex) region of interest analysis showed that 0.40M sucrose activation  (but not 0.10M) decreased after surgery. Sensory region (primary somatosensory and primary taste cortex) 0.40M sucrose activation was unchanged by surgery and did not differ between control and bariatric participants. Primary taste cortex activation to 0.10M sucrose solution was greater in postsurgical bariatric participants compared with control participants.

CONCLUSIONS

Bariatric surgery reduces the reward system response to sweet taste in women with obesity without affecting activity in sensory regions, which is consistent with reduced drive to consume sweet foods.

Aberrant type 2 dopamine receptor availability in violent offenders with psychopathy

Psychopathy is characterized by antisocial behavior, poor behavioral control and lacking empathy, and structural alterations in the corresponding neural circuits. Molecular brain basis of psychopathy remains poorly characterized. Here we studied type 2 dopamine receptor (D2R) and mu-opioid receptor (MOR) availability in convicted violent offenders with high psychopathic traits (n = 11) and healthy matched controls (n = 17) using positron emission tomography (PET). D2R were measured with radioligand [11C]raclopride and MORs with radioligand [11C]carfentanil. Psychopathic subjects had lowered D2R availability in caudate and putamen, and striatal D2R availability was also associated with degree of psychopathic traits in this prisoner sample. No group differences were found in MOR availability, although in the prisoner sample, psychopathic traits were negatively correlated with MOR availability in the amygdala and nucleus accumbens. We conclude that D2R signaling could be the putative neuromolecular pathway for psychopathy, whereas evidence for alterations in the MOR system is more limited.

The nucleus accumbens in reward and aversion processing: insights and implications

The nucleus accumbens (NAc), a central component of the brain's reward circuitry, has been implicated in a wide range of behaviors and emotional states. Emerging evidence, primarily drawing from recent rodent studies, suggests that the function of the NAc in reward and aversion processing is multifaceted. Prolonged stress or drug use induces maladaptive neuronal function in the NAc circuitry, which results in pathological conditions. This review aims to provide comprehensive and up-to-date insights on the role of the NAc in motivated behavior regulation and highlights areas that demand further in-depth analysis. It synthesizes the latest findings on how distinct NAc neuronal populations and pathways contribute to the processing of opposite valences. The review examines how a range of neuromodulators, especially monoamines, influence the NAc's control over various motivational states. Furthermore, it delves into the complex underlying mechanisms of psychiatric disorders such as addiction and depression and evaluates prospective interventions to restore NAc functionality.

Dysregulation of the dopaminergic system secondary to traumatic brain injury: implications for mood and anxiety disorders

Traumatic brain injury (TBI) represents a public health issue with a high mortality rate and severe neurological and psychiatric consequences. Mood and anxiety disorders are some of the most frequently reported. Primary and secondary damage can cause a loss of neurons and glial cells, leading to dysfunction of neuronal circuits, which can induce imbalances in many neurotransmitter systems. Monoaminergic systems, especially the dopaminergic system, are some of the most involved in the pathogenesis of neuropsychiatric and cognitive symptoms after TBI. In this work, we summarize the studies carried out in patients who have suffered TBI and describe alterations in the dopaminergic system, highlighting (1) dysfunction of the dopaminergic neuronal circuits caused by TBI, where modifications are shown in the dopamine transporter (DAT) and alterations in the expression of dopamine receptor 2 (D2R) in brain areas with dopaminergic innervation, thus establishing a hypodopaminergic state and (2) variations in the concentration of dopamine and its metabolites in biological fluids of post-TBI patients, such as elevated dopamine (DA) and alterations in homovanillic acid (HVA). On the other hand, we show a large number of reports of alterations in the dopaminergic system after a TBI in animal models, in which modifications in the levels of DA, DAT, and HVA have been reported, as well as alterations in the expression of tyrosine hydroxylase (TH). We also describe the biological pathways, neuronal circuits, and molecular mechanisms potentially involved in mood and anxiety disorders that occur after TBI and are associated with alterations of the dopaminergic system in clinical studies and animal models. We describe the changes that occur in the clinical picture of post-TBI patients, such as alterations in mood and anxiety associated with DAT activity in the striatum, the relationship between post-TBI major depressive disorders (MDD) with lower availability of the DA receptors D2R and D3R in the caudate and thalamus, as well as a decrease in the volume of the substantia nigra (SN) associated with anxiety symptoms. With these findings, we discuss the possible relationship between the disorders caused by alterations in the dopaminergic system in patients with TBI.

CBGTPy: An extensible cortico-basal ganglia-thalamic framework for modeling biological decision making

Here we introduce CBGTPy, a virtual environment for designing and testing goal-directed agents with internal dynamics that are modeled on the cortico-basal-ganglia-thalamic (CBGT) pathways in the mammalian brain. CBGTPy enables researchers to investigate the internal dynamics of the CBGT system during a variety of tasks, allowing for the formation of testable predictions about animal behavior and neural activity. The framework has been designed around the principle of flexibility, such that many experimental parameters in a decision making paradigm can be easily defined and modified. Here we demonstrate the capabilities of CBGTPy across a range of single and multi-choice tasks, highlighting the ease of set up and the biologically realistic behavior that it produces. We show that CBGTPy is extensible enough to apply to a range of experimental protocols and to allow for the implementation of model extensions with minimal developmental effort.

A biophysical model for dopamine modulating working memory through reward system in obsessive-compulsive disorder

Dopamine modulates working memory in the prefrontal cortex (PFC) and is crucial for obsessive-compulsive disorder (OCD). However, the mechanism is unclear. Here we establish a biophysical model of the effect of dopamine (DA) in PFC to explain the mechanism of how high dopamine concentrations induce persistent neuronal activities with the network plunging into a deep, stable attractor state. The state develops a defect in working memory and tends to obsession and compulsion. Weakening the reuptake of dopamine acts on synaptic plasticity according to Hebbian learning rules and reward learning, which in turn affects the strength of neuronal synaptic connections, resulting in the tendency of compulsion and learned obsession. In addition, we elucidate the potential mechanisms of dopamine antagonists in OCD, indicating that dopaminergic drugs might be available for treatment, even if the abnormality is a consequence of glutamate hypermetabolism rather than dopamine. The theory highlights the significance of early intervention and behavioural therapies for obsessive-compulsive disorder. It potentially offers new approaches to dopaminergic pharmacotherapy and psychotherapy for OCD patients.

The DNA repair protein DNA-PKcs modulates synaptic plasticity via PSD-95 phosphorylation and stability

The key DNA repair enzyme DNA-PKcs has several and important cellular functions. Loss of DNA-PKcs activity in mice has revealed essential roles in immune and nervous systems. In humans, DNA-PKcs is a critical factor for brain development and function since mutation of the prkdc gene causes severe neurological deficits such as microcephaly and seizures, predicting yet unknown roles of DNA-PKcs in neurons. Here we show that DNA-PKcs modulates synaptic plasticity. We demonstrate that DNA-PKcs localizes at synapses and phosphorylates PSD-95 at newly identified residues controlling PSD-95 protein stability. DNA-PKcs -/- mice are characterized by impaired Long-Term Potentiation (LTP), changes in neuronal morphology, and reduced levels of postsynaptic proteins. A PSD-95 mutant that is constitutively phosphorylated rescues LTP impairment when over-expressed in DNA-PKcs -/- mice. Our study identifies an emergent physiological function of DNA-PKcs in regulating neuronal plasticity, beyond genome stability.

A feature-specific prediction error model explains dopaminergic heterogeneity

The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary 'feature-specific RPE' model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal's moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.

D2 dopamine receptor expression, reactivity to rewards, and reinforcement learning in a complex value-based decision-making task

Different dopamine (DA) subtypes have opposing dynamics at postsynaptic receptors, with the ratio of D1 to D2 receptors determining the relative sensitivity to gains and losses, respectively, during value-based learning. This effective sensitivity to different reward feedback interacts with phasic DA levels to determine the effectiveness of learning, particularly in dynamic feedback situations where the frequency and magnitude of rewards need to be integrated over time to make optimal decisions. We modeled this effect in simulations of the underlying basal ganglia pathways and then tested the predictions in individuals with a variant of the human dopamine receptor D2 (DRD2; -141C Ins/Del and Del/Del) gene that associates with lower levels of D2 receptor expression (N = 119) and compared their performance in the Iowa Gambling Task to noncarrier controls (N = 319). Ventral striatal (VS) reactivity to rewards was measured in the Cards task with fMRI. DRD2 variant carriers made less effective decisions than noncarriers, but this effect was not moderated by VS reward reactivity as is hypothesized by our model. These results suggest that the interaction between DA receptor subtypes and reactivity to rewards during learning may be more complex than originally thought.

Dendritic spines and their role in the pathogenesis of neurodevelopmental and neurological disorders

Since Cajal introduced dendritic spines in the 19th century, they have attained considerable attention, especially in neuropsychiatric and neurologic disorders. Multiple roles of dendritic spine malfunction and pathology in the progression of various diseases have been reported. Thus, it is inevitable to consider these structures as new therapeutic targets for treating neuropsychiatric and neurologic disorders such as autism spectrum disorders, schizophrenia, dementia, Down syndrome, etc. Therefore, we attempted to prepare a narrative review of the literature regarding the role of dendritic spines in the pathogenesis of aforementioned diseases and to shed new light on their pathophysiology.

Phasic Stimulation of Dopaminergic Neurons of the Lateral Substantia Nigra Increases Open Field Exploratory Behaviour and Reduces Habituation Over Time

Transient nigrostriatal dopaminergic signalling is well known for its role in reinforcement learning and increasingly so for its role in the initiation of voluntary movement. However, how transient bursts of dopamine modulate voluntary movement remains unclear, likely due to the heterogeneity of the nigrostriatal system, the focus of optogenetic studies on locomotion at sub-sec time intervals, and the overlapping roles of phasic dopamine in behaviour and novelty signalling. In this study we investigated how phasic activity in the lateral substantia nigra pars compacta (lateral SNc) over time affects voluntary behaviours during exploration. Using a transgenic mouse model of both sexes expressing channelrhodopsin (ChR2) in dopamine transporter-expressing cells, we stimulated the lateral SNc while mice explored an open field over two consecutive days. We found that phasic activation of the lateral SNc induced an increase in exploratory behaviours including horizontal movement activity, locomotion initiation, and rearing specifically on the first open field exposure, but not on the second day. In addition, stimulated animals did not habituate to the same extent as their ChR2-negative counterparts, as indicated by a lack of decrease in baseline activity. These findings suggest that rather than prompting voluntary movement in general, phasic nigrostriatal dopamine prompts context-appropriate behaviours. In addition, dopamine signalling that modulates movement acts over longer timescales than the transient signal, affecting behaviour even after the signal has ended.

Emerging evidence for pregnane steroid therapeutics for alcohol use disorders

Many lines of research have suggested that the neuroactive pregnane steroids, including pregnenolone, progesterone, and allopregnanolone ([3α,5α]-3-hydroxypregnan-20-one, 3α,5α-THP), have therapeutic potential for treatment of alcohol use disorders (AUDs). In this chapter, we systematically address the preclinical and clinical evidence that supports this approach for AUD treatment, describe the underlying neurobiology of AUDs that are targeted by these treatments, and delineate how pregnane steroids may address various components of the disease. This review updates the theoretical framework for understanding how endogenous steroids that modulate the effects of alcohol, stress, excitatory/inhibitory and dopamine transmission, and the innate immune system appear to play a key role in the prevention and mitigation of AUDs. We further discuss newly discovered limitations of pregnane steroid therapies as well as the challenges that are inherent to development of endogenous compounds for therapeutics. We argue that overcoming these challenges presents the opportunity to help millions who suffer from AUDs across the world.

Lesions of the lateral habenula excite dopamine neurons in the ventral tegmental area and serotonin neurons in the dorsal raphe nuclei in hemiparkinsonian rats

The lateral habenula (LHb) projects to the ventral tegmental area (VTA) and dorsal raphe nuclei (DRN) that deliver dopamine (DA) and serotonin (5-HT) to cortical and limbic regions such as the medial prefrontal cortex (mPFC), hippocampus and basolateral amygdala (BLA). Dysfunctions of VTA-related mesocorticolimbic dopaminergic and DRN-related serotonergic systems contribute to non-motor symptoms in Parkinson's disease (PD). However, how the LHb affects the VTA and DRN in PD remains unclear. Here, we used electrophysiological and neurochemical approaches to explore the effects of LHb lesions on the firing activity of VTA and DRN neurons, as well as the levels of DA and 5-HT in related brain regions in unilateral 6-hydroxydopamie (6-OHDA)-induced PD rats. We found that compared to sham lesions, lesions of the LHb increased the firing rate of DA neurons in the VTA and 5-HT neurons in the DRN, but decreased the firing rate of GABAergic neurons in the same nucleus. In addition, lesions of the LHb increased the levels of DA and 5-HT in the mPFC, ventral hippocampus and BLA compared to sham lesions. These findings suggest that lesions of the LHb enhance the activity of mesocorticolimbic dopaminergic and serotonergic systems in PD.

Natural phasic inhibition of dopamine neurons signals cognitive rigidity

When animals unexpectedly fail, their dopamine neurons undergo phasic inhibition that canonically drives extinction learning-a cognitive-flexibility mechanism for discarding outdated strategies. However, the existing evidence equates natural and artificial phasic inhibition, despite their spatiotemporal differences. Addressing this gap, we targeted a GABAA-receptor antagonist precisely to dopamine neurons, yielding three unexpected findings. First, this intervention blocked natural phasic inhibition selectively, leaving tonic activity unaffected. Second, blocking natural phasic inhibition accelerated extinction learning-opposite to canonical mechanisms. Third, our approach selectively benefitted perseverative mice, restoring rapid extinction without affecting new reward learning. Our findings reveal that extinction learning is rapid by default and slowed by natural phasic inhibition-challenging foundational learning theories, while delineating a synaptic mechanism and therapeutic target for cognitive rigidity.

Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time

The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference learning (TD) learning, whereby certain units signal reward prediction errors (RPE). The TD algorithm has been traditionally mapped onto the dopaminergic system, as firing properties of dopamine neurons can resemble RPEs. However, certain predictions of TD learning are inconsistent with experimental results, and previous implementations of the algorithm have made unscalable assumptions regarding stimulus-specific fixed temporal bases. We propose an alternate framework to describe dopamine signaling in the brain, FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, dopamine release is similar, but not identical to RPE, leading to predictions that contrast to those of TD. While FLEX itself is a general theoretical framework, we describe a specific, biophysically plausible implementation, the results of which are consistent with a preponderance of both existing and reanalyzed experimental data.

Gradient-free training of recurrent neural networks using random perturbations

Recurrent neural networks (RNNs) hold immense potential for computations due to their Turing completeness and sequential processing capabilities, yet existing methods for their training encounter efficiency challenges. Backpropagation through time (BPTT), the prevailing method, extends the backpropagation (BP) algorithm by unrolling the RNN over time. However, this approach suffers from significant drawbacks, including the need to interleave forward and backward phases and store exact gradient information. Furthermore, BPTT has been shown to struggle to propagate gradient information for long sequences, leading to vanishing gradients. An alternative strategy to using gradient-based methods like BPTT involves stochastically approximating gradients through perturbation-based methods. This learning approach is exceptionally simple, necessitating only forward passes in the network and a global reinforcement signal as feedback. Despite its simplicity, the random nature of its updates typically leads to inefficient optimization, limiting its effectiveness in training neural networks. In this study, we present a new approach to perturbation-based learning in RNNs whose performance is competitive with BPTT, while maintaining the inherent advantages over gradient-based learning. To this end, we extend the recently introduced activity-based node perturbation (ANP) method to operate in the time domain, leading to more efficient learning and generalization. We subsequently conduct a range of experiments to validate our approach. Our results show similar performance, convergence time and scalability when compared to BPTT, strongly outperforming standard node perturbation and weight perturbation methods. These findings suggest that perturbation-based learning methods offer a versatile alternative to gradient-based methods for training RNNs which can be ideally suited for neuromorphic computing applications.

Age-dependent predictors of effective reinforcement motor learning across childhood

Across development, children must learn motor skills such as eating with a spoon and drawing with a crayon. Reinforcement learning, driven by success and failure, is fundamental to such sensorimotor learning. It typically requires a child to explore movement options along a continuum (grip location on a crayon) and learn from probabilistic rewards (whether the crayon draws or breaks). Here, we studied the development of reinforcement motor learning using online motor tasks to engage children aged 3 to 17 and adults (cross-sectional sample, N=385). Participants moved a cartoon penguin across a scene and were rewarded (animated cartoon clip) based on their final movement position. Learning followed a clear developmental trajectory when participants could choose to move anywhere along a continuum and the reward probability depended on final movement position. Learning was incomplete or absent in 3 to 8-year-olds and gradually improved to adult-like levels by adolescence. A reinforcement learning model fit to each participant identified three age-dependent factors underlying improvement: amount of exploration after a failed movement, learning rate, and level of motor noise. We predicted, and confirmed, that switching to discrete targets and deterministic reward would improve 3 to 8-year-olds' learning to adult-like levels by increasing exploration after failed movements. Overall, we show a robust developmental trajectory of reinforcement motor learning abilities under ecologically relevant conditions i.e., continuous movement options mapped to probabilistic reward. This learning appears to be limited by immature spatial processing and probabilistic reasoning abilities in young children and can be rescued by reducing the demands in these domains.

PET/CT study of dopamine transporter (DAT) binding with the triple reuptake inhibitor toludesvenlafaxine in rats and humans

PURPOSE

Toludesvenlafaxine is a recently developed antidepressant that belongs to the triple reuptake inhibitor class. Despite the in vitro evidence that toludesvenlafaxine inhibits the reuptake of serotonin (5-HT), norepinephrine (NE) and dopamine (DA), there is no in vivo evidence that toludesvenlafaxine binds to DAT and increases DA level, a mechanism thought to contribute to its favorable clinical performance.

METHODS

Positron emission tomography/computed tomography (PET/CT) was used to examine the DAT binding capacity in healthy rats and human subjects and microdialysis was used to examine the striatal DA level in rats. [18F]FECNT and [11C]CFT were used as PET/CT radioactive tracer for rat and human studies, respectively.

RESULTS

In rats, 9 mg/kg of toludesvenlafaxine hydrochloride (i.v.) followed by an infusion of 3 mg/kg via minipump led to the binding rate to striatum DAT at 3.7 - 32.41% and to hypothalamus DAT at 5.91 - 17.52% during the 45 min scanning period. 32 mg/kg oral administration with toludesvenlafaxine hydrochloride significantly increased the striatal DA level with the AUC0 - 180 min increased by 63.9%. In healthy volunteers, 160 mg daily toludesvenlafaxine hydrochloride sustained-release tablets for 4 days led to an average occupancy rates of DAT at 8.04% ± 7.75% and 8.09% ± 7.00%, respectively, in basal ganglion 6 h and 10 h postdose.

CONCLUSION

These results represent the first to confirm the binding of toludesvenlafaxine to DAT in both rats and humans using PET/CT, and its elevation of brain DA level, which may help understand the unique pharmacological and functional effects of triple reuptake inhibitors such as toludesvenlafaxine.

CLINICALTRIALS

GOV IDENTIFIERS

NCT05905120. Registered 14 June 2023. (retrospectively registered).

Reinforcement Learning May Demystify the Limited Human Motor Learning Efficacy Due to Visual-Proprioceptive Mismatch

Vision and proprioception have fundamental sensory mismatches in delivering locational information, and such mismatches are critical factors limiting the efficacy of motor learning. However, it is still not clear how and to what extent this mismatch limits motor learning outcomes. To further the understanding of the effect of sensory mismatch on motor learning outcomes, a reinforcement learning algorithm and the simplified biomechanical elbow joint model were employed to mimic the motor learning process in a computational environment. By applying a reinforcement learning algorithm to the motor learning of elbow joint flexion task, simulation results successfully explained how visual-proprioceptive mismatch limits motor learning outcomes in terms of motor control accuracy and task completion speed. The larger the perceived angular offset between the two sensory modalities, the lower the motor control accuracy. Also, the more similar the peak reward amplitude of the two sensory modalities, the lower the motor control accuracy. In addition, simulation results suggest that insufficient exploration rate limits task completion speed, and excessive exploration rate limits motor control accuracy. Such a speed-accuracy trade-off shows that a moderate exploration rate could serve as another important factor in motor learning.

The distribution and evolutionary dynamics of dopaminergic neurons in molluscs

Dopamine is one of the most versatile neurotransmitters in invertebrates. It's distribution and plethora of functions is likely coupled to feeding ecology, especially in Euthyneura (the largest clade of molluscs), which presents the broadest spectrum of environmental adaptations. Still, the analyses of dopamine-mediated signaling were dominated by studies of grazers. Here, we characterize the distribution of dopaminergic neurons in representatives of two distinct ecological groups: the sea angel - obligate predatory pelagic mollusc Clione limacina (Pteropoda, Gymnosomata) and its prey - the sea devil Limacina helicina (Pteropoda, Thecosomata) as well as the plankton eater Melibe leonina (Nudipleura, Nudibranchia). By using tyrosine hydroxylase-immunoreactivity (TH-ir) as a reporter, we showed that the dopaminergic system is moderately conservative among euthyneurans. Across all studied species, small numbers of dopaminergic neurons in the central ganglia contrast to significant diversification of TH-ir neurons in the peripheral nervous system, primarily representing sensory-like cells, which predominantly concentrated in the chemotactic areas and projecting afferent axons to the central nervous system. Combined with α-tubulin immunoreactivity, this study illuminates the unprecedented complexity of peripheral neural systems in gastropod molluscs, with lineage-specific diversification of sensory and modulatory functions.

Convergence of oxytocin and dopamine signalling in neuronal circuits: Insights into the neurobiology of social interactions across species

Social behaviour is essential for animal survival, and the hypothalamic neuropeptide oxytocin (OXT) critically impacts bonding, parenting, and decision-making. Dopamine (DA), is released by ventral tegmental area (VTA) dopaminergic neurons, regulating social cues in the mesolimbic system. Despite extensive exploration of OXT and DA roles in social behaviour independently, limited studies investigate their interplay. This narrative review integrates insights from human and animal studies, particularly rodents, emphasising recent research on pharmacological manipulations of OXT or DA systems in social behaviour. Additionally, we review studies correlating social behaviour with blood/cerebral OXT and DA levels. Behavioural facets include sociability, cooperation, pair bonding and parental care. In addition, we provide insights into OXT-DA interplay in animal models of social stress, autism, and schizophrenia. Emphasis is placed on the complex relationship between the OXT and DA systems and their collective influence on social behaviour across physiological and pathological conditions. Understanding OXT and DA imbalance is fundamental for unravelling the neurobiological underpinnings of social interaction and reward processing deficits observed in psychiatric conditions.

Evolution of reciprocity with limited payoff memory

Direct reciprocity is a mechanism for the evolution of cooperation in repeated social interactions. According to the literature, individuals naturally learn to adopt conditionally cooperative strategies if they have multiple encounters with their partner. Corresponding models have greatly facilitated our understanding of cooperation, yet they often make strong assumptions on how individuals remember and process payoff information. For example, when strategies are updated through social learning, it is commonly assumed that individuals compare their average payoffs. This would require them to compute (or remember) their payoffs against everyone else in the population. To understand how more realistic constraints influence direct reciprocity, we consider the evolution of conditional behaviours when individuals learn based on more recent experiences. Even in the most extreme case that they only take into account their very last interaction, we find that cooperation can still evolve. However, such individuals adopt less generous strategies, and they cooperate less often than in the classical setup with average payoffs. Interestingly, once individuals remember the payoffs of two or three recent interactions, cooperation rates quickly approach the classical limit. These findings contribute to a literature that explores which kind of cognitive capabilities are required for reciprocal cooperation. While our results suggest that some rudimentary form of payoff memory is necessary, it suffices to remember a few interactions.

Impaired striatal glutamate/GABA regulation in violent offenders with antisocial personality disorder and psychopathy

Men with antisocial personality disorder (ASPD) with or without psychopathy (+/-P) are responsible for most violent crime in society. Development of effective treatments is hindered by poor understanding of the neurochemical underpinnings of the condition. Men with ASPD with and without psychopathy demonstrate impulsive decision-making, associated with striatal abnormalities in functional neuroimaging studies. However, to date, no study has directly examined the potential neurochemical underpinnings of such abnormalities. We therefore investigated striatal glutamate: GABA ratio using Magnetic Resonance Spectroscopy in 30 violent offenders (16 ASPD-P, 14 ASPD + P) and 21 healthy non-offenders. Men with ASPD +/- P had a significant reduction in striatal glutamate : GABA ratio compared to non-offenders. We report, for the first time, striatal Glutamate/GABA dysregulation in ASPD +/- P, and discuss how this may be related to core behavioral abnormalities in the disorders.

The Mesocortical System Encodes the Strength of Subsequent Force Generation

Our minds impact motor outputs. Such mind-motor interactions are critical for understanding motor control mechanisms and optimizing motor performance. In particular, incentive motivation strongly enhances motor performance. Dopaminergic neurons located in the ventral midbrain (VM) are believed to be the center of incentive motivation. Direct projections from the VM to the primary motor cortex constitute a mesocortical pathway. However, the functional role of this pathway in humans remains unclear. Recently, we demonstrated the functional role of the mesocortical pathway in human motor control in the context of incentive motivation by using functional magnetic resonance imaging (fMRI). Incentive motivation remarkably improved not only reaction times but also the peak grip force in subsequent grip responses. Although the reaction time has been used as a proxy for incentive motivation mediated by dopaminergic midbrain activity, the premovement activity of the mesocortical pathway is involved in controlling the force strength rather than the initiation of subsequent force generation. In this commentary, we review our recent findings and discuss remaining questions regarding the functional role of the mesocortical pathway in mind-motor interactions.

Endogenous opioids in the olfactory tubercle and their roles in olfaction and quality of life

Olfactory dysfunctions decrease daily quality of life (QOL) in part by reducing the pleasure of eating. Olfaction plays an essential role in flavor sensation and palatability. The decreased QOL due to olfactory dysfunction is speculated to result from abnormal neural activities in the olfactory and limbic areas of the brain, as well as peripheral odorant receptor dysfunctions. However, the specific underlying neurobiological mechanisms remain unclear. As the olfactory tubercle (OT) is one of the brain's regions with high expression of endogenous opioids, we hypothesize that the mechanism underlying the decrease in QOL due to olfactory dysfunction involves the reduction of neural activity in the OT and subsequent endogenous opioid release in specialized subregions. In this review, we provide an overview and recent updates on the OT, the endogenous opioid system, and the pleasure systems in the brain and then discuss our hypothesis. To facilitate the effective treatment of olfactory dysfunctions and decreased QOL, elucidation of the neurobiological mechanisms underlying the pleasure of eating through flavor sensation is crucial.