Opponent interactions between serotonin and dopamine

Dynamics of human serotonin synthesis differentially link to reward anticipation and feedback

Serotonin (5-HT) plays an essential role in reward processing, however, the possibilities to investigate 5-HT action in humans during emotional stimulation are particularly limited. Here we demonstrate the feasibility of assessing reward-specific dynamics in 5-HT synthesis using functional PET (fPET), combining its molecular specificity with the high temporal resolution of blood oxygen level dependent (BOLD) fMRI. Sixteen healthy volunteers underwent simultaneous fPET/fMRI with the radioligand [11C]AMT, a substrate for tryptophan hydroxylase. During the scan, participants completed the monetary incentive delay task and arterial blood samples were acquired for quantifying 5-HT synthesis rates. BOLD fMRI was recorded as a proxy of neuronal activation, allowing differentiation of reward anticipation and feedback. Monetary gain and loss resulted in substantial increases in 5-HT synthesis in the ventral striatum (VStr, +21% from baseline) and the anterior insula (+41%). In the VStr, task-specific 5-HT synthesis was further correlated with BOLD signal changes during reward feedback (ρ = -0.65), but not anticipation. Conversely, 5-HT synthesis in the anterior insula correlated with BOLD reward anticipation (ρ = -0.61), but not feedback. In sum, we provide a robust tool to identify task-induced changes in 5-HT action in humans, linking the dynamics of 5-HT synthesis to distinct phases of reward processing in a regionally specific manner. Given the relevance of altered reward processing in psychiatric disorders such as addiction, depression and schizophrenia, our approach offers a tailored assessment of impaired 5-HT signaling during cognitive and emotional processing.

Understanding the neurobiology of social behavior through exploring brain-wide dynamics of neural activity

Social behavior is highly complex and adaptable. It can be divided into multiple temporal stages: detection, approach, and consummatory behavior. Each stage can be further divided into several cognitive and behavioral processes, such as perceiving social cues, evaluating the social and non-social contexts, and recognizing the internal/emotional state of others. Recent studies have identified numerous brain-wide circuits implicated in social behavior and suggested the existence of partially overlapping functional brain networks underlying various types of social and non-social behavior. However, understanding the brain-wide dynamics underlying social behavior remains challenging, and several brain-scale dynamics (macro-, meso-, and micro-scale levels) need to be integrated. Here, we suggest leveraging new tools and concepts to explore social brain networks and integrate those different levels. These include studying the expression of immediate-early genes throughout the entire brain to impartially define the structure of the neuronal networks involved in a given social behavior. Then, network dynamics could be investigated using electrode arrays or multi-channel fiber photometry. Finally, tools like high-density silicon probes and miniscopes can probe neural activity in specific areas and across neuronal populations at the single-cell level.

Inhibitory mechanisms in the prefrontal-cortex differentially mediate Putamen activity during valence-based learning

Learning from appetitive and aversive stimuli is important for survival. It involves interactions between the prefrontal cortex and subcortical structures, with inhibition playing a crucial role. However, direct evidence for this in humans is limited. Here, we overcome the difficulty of measuring inhibition in the human brain and find that GABA, the main inhibitory neurotransmitter, affects how the dACC interacts with subcortical structures during appetitive and aversive learning differently. We used 7T magnetic resonance spectroscopy (MRS) to track GABA levels in the dACC alongside whole-brain fMRI scans while participants engaged in appetitive and aversive learning tasks. During appetitive learning, dACC GABA levels were negatively correlated with learning performance and BOLD activity measured from the dACC and the Putamen. While under aversive learning, dACC GABA concentration negatively correlated with the functional connectivity between the dACC and the Putamen. Our results show that inhibition in the dACC mediates appetitive and aversive learning in humans through distinct mechanisms.

Influence of surprise on reinforcement learning in younger and older adults

Surprise is a key component of many learning experiences, and yet its precise computational role, and how it changes with age, remain debated. One major challenge is that surprise often occurs jointly with other variables, such as uncertainty and outcome probability. To assess how humans learn from surprising events, and whether aging affects this process, we studied choices while participants learned from bandits with either Gaussian or bi-modal outcome distributions, which decoupled outcome probability, uncertainty, and surprise. A total of 102 participants (51 older, aged 50-73; 51 younger, 19-30 years) chose between three bandits, one of which had a bimodal outcome distribution. Behavioral analyses showed that both age-groups learned the average of the bimodal bandit less well. A trial-by-trial analysis indicated that participants performed choice reversals immediately following large absolute prediction errors, consistent with heightened sensitivity to surprise. This effect was stronger in older adults. Computational models indicated that learning rates in younger as well as older adults were influenced by surprise, rather than uncertainty, but also suggested large interindividual variability in the process underlying learning in our task. Our work bridges between behavioral economics research that has focused on how outcomes with low probability affect choice in older adults, and reinforcement learning work that has investigated age differences in the effects of uncertainty and suggests that older adults overly adapt to surprising events, even when accounting for probability and uncertainty effects.

A feature-specific prediction error model explains dopaminergic heterogeneity

The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary 'feature-specific RPE' model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal's moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Understanding Why Homeopathic Medicines are Used for Menopause: Searching for Insights into Neuroendocrine Features

BACKGROUND

 Menopause is a physiological event that marks the end of a woman's reproductive stage in life. Vasomotor symptoms and changes in mood are among its most important effects. Homeopathy has been used for many years in treating menopausal complaints, though clinical and pre-clinical research in this field is limited. Homeopathy often bases its prescription on neuropsychiatric symptoms, but it is unknown if homeopathic medicines (HMs) exert a neuroendocrine effect that causes an improvement in vasomotor symptoms and mood during menopause.

OBJECTIVES

 The study's objectives were to address the pathophysiological changes of menopause that could help in the understanding of the possible effect of HMs at a neuroendocrine level, to review the current evidence for two of the most frequently prescribed HMs for menopause (Lachesis mutus and Sepia officinalis), and to discuss the future directions of research in this field.

METHODS

 An extensive literature search for the pathophysiologic events of menopause and depression, as well as for the current evidence for HMs in menopause and depression, was performed.

RESULTS

 Neuroendocrine changes are involved in the pathophysiology of vasomotor symptoms and changes in mood during menopause. Gonadal hormones modulate neurotransmitter systems. Both play a role in mood disorders and temperature regulation. It has been demonstrated that Gelsemium sempervirens, Ignatia amara and Chamomilla matricaria exert anxiolytic effects in rodent models. Lachesis mutus and Sepia officinalis are frequently prescribed for important neuropsychiatric and vasomotor symptoms. Dopamine, a neurotransmitter involved in mood, is among the constituents of the ink of the common cuttlefish, Sepia officinalis.

CONCLUSION

 Based on all the pathophysiologic events of menopause and the improvement in menopausal complaints that certain HMs show in daily practice, these medicines might have a direct or indirect neuroendocrine effect in the body, possibly triggered via an as-yet unidentified biological mechanism. Many unanswered questions in this field require further pre-clinical and clinical research.

Addressing Combative Behaviour in Spanish Bulls by Measuring Hormonal Indicators

The fighting bull is characterised by its natural aggressiveness, but the physiological mechanisms that underlie its aggressive behaviour are poorly studied. This study determines the hormonal component of aggressiveness in fighting bulls by analysing their behaviour during a fight and correlating it to their serotonin, dopamine and testosterone levels. We also determine whether aggressive behaviour can be estimated in calves. Using 195 animals, samples were obtained when the animals were calves and after 5 years. Aggressiveness scores were obtained by an observational method during bullfights, and serotonin, dopamine and testosterone levels were determined in all animals using validated enzyme immunoassay kits. The results revealed a strong correlation of serotonin and dopamine levels with aggressiveness scores in bulls during fights, but no correlation was found with respect to testosterone. These correlations led to established cut-off point and linear regression curves to obtain expected aggressiveness scores for calves at shoeing. There were no significant differences between the expected scores obtained in calves and the observed scores in bulls. Therefore, this study demonstrates that hormone determination in calves may be a great indicator of combativeness in bulls and can reliably be used in the selection of fighting bulls.

Impaired value-based decision-making in Parkinson's disease apathy

Apathy is a common and disabling complication of Parkinson's disease characterized by reduced goal-directed behaviour. Several studies have reported dysfunction within prefrontal cortical regions and projections from brainstem nuclei whose neuromodulators include dopamine, serotonin and noradrenaline. Work in animal and human neuroscience have confirmed contributions of these neuromodulators on aspects of motivated decision-making. Specifically, these neuromodulators have overlapping contributions to encoding the value of decisions, and influence whether to explore alternative courses of action or persist in an existing strategy to achieve a rewarding goal. Building upon this work, we hypothesized that apathy in Parkinson's disease should be associated with an impairment in value-based learning. Using a four-armed restless bandit reinforcement learning task, we studied decision-making in 75 volunteers; 53 patients with Parkinson's disease, with and without clinical apathy, and 22 age-matched healthy control subjects. Patients with apathy exhibited impaired ability to choose the highest value bandit. Task performance predicted an individual patient's apathy severity measured using the Lille Apathy Rating Scale (R = -0.46, P < 0.001). Computational modelling of the patient's choices confirmed the apathy group made decisions that were indifferent to the learnt value of the options, consistent with previous reports of reward insensitivity. Further analysis demonstrated a shift away from exploiting the highest value option and a reduction in perseveration, which also correlated with apathy scores (R = -0.5, P < 0.001). We went on to acquire functional MRI in 59 volunteers; a group of 19 patients with and 20 without apathy and 20 age-matched controls performing the Restless Bandit Task. Analysis of the functional MRI signal at the point of reward feedback confirmed diminished signal within ventromedial prefrontal cortex in Parkinson's disease, which was more marked in apathy, but not predictive of their individual apathy severity. Using a model-based categorization of choice type, decisions to explore lower value bandits in the apathy group activated prefrontal cortex to a similar degree to the age-matched controls. In contrast, Parkinson's patients without apathy demonstrated significantly increased activation across a distributed thalamo-cortical network. Enhanced activity in the thalamus predicted individual apathy severity across both patient groups and exhibited functional connectivity with dorsal anterior cingulate cortex and anterior insula. Given that task performance in patients without apathy was no different to the age-matched control subjects, we interpret the recruitment of this network as a possible compensatory mechanism, which compensates against symptomatic manifestation of apathy in Parkinson's disease.

Distinct spatiotemporal brainstem pathways of outcome valence during reward- and punishment-based learning

Learning to seek rewards and avoid punishments, based on positive and negative choice outcomes, is essential for human survival. Yet, the neural underpinnings of outcome valence in the human brainstem and the extent to which they differ in reward and punishment learning contexts remain largely elusive. Here, using simultaneously acquired electroencephalography and functional magnetic resonance imaging data, we show that during reward learning the substantia nigra (SN)/ventral tegmental area (VTA) and locus coeruleus are initially activated following negative outcomes, while the VTA subsequently re-engages exhibiting greater responses for positive than negative outcomes, consistent with an early arousal/avoidance response and a later value-updating process, respectively. During punishment learning, we show that distinct raphe nucleus and SN subregions are activated only by negative outcomes with a sustained post-outcome activity across time, supporting the involvement of these brainstem subregions in avoidance behavior. Finally, we demonstrate that the coupling of these brainstem structures with other subcortical and cortical areas helps to shape participants' serial choice behavior in each context.

Silences, spikes and bursts: Three-part knot of the neural code

When a neuron breaks silence, it can emit action potentials in a number of patterns. Some responses are so sudden and intense that electrophysiologists felt the need to single them out, labelling action potentials emitted at a particularly high frequency with a metonym - bursts. Is there more to bursts than a figure of speech? After all, sudden bouts of high-frequency firing are expected to occur whenever inputs surge. The burst coding hypothesis advances that the neural code has three syllables: silences, spikes and bursts. We review evidence supporting this ternary code in terms of devoted mechanisms for burst generation, synaptic transmission and synaptic plasticity. We also review the learning and attention theories for which such a triad is beneficial.

Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans

In the mammalian brain, midbrain dopamine neuron activity is hypothesized to encode reward prediction errors that promote learning and guide behavior by causing rapid changes in dopamine levels in target brain regions. This hypothesis (and alternatives regarding dopamine's role in punishment-learning) has limited direct evidence in humans. We report intracranial, subsecond measurements of dopamine release in human striatum measured, while volunteers (i.e., patients undergoing deep brain stimulation surgery) performed a probabilistic reward and punishment learning choice task designed to test whether dopamine release encodes only reward prediction errors or whether dopamine release may also encode adaptive punishment learning signals. Results demonstrate that extracellular dopamine levels can encode both reward and punishment prediction errors within distinct time intervals via independent valence-specific pathways in the human brain.

Differential effects of reward and punishment on reinforcement-based motor learning and generalization

Reward and punishment have long been recognized as potent modulators of human behavior. Although reinforcement learning is a significant motor learning process, the exact mechanisms underlying how the brain learns movements through reward and punishment are not yet fully understood. Beyond the memory of specific examples, investigating the ability to generalize to new situations offers a better understanding of motor learning. This study hypothesizes that reward and punishment engage qualitatively different motivational systems with different neurochemical and neuroanatomical substrates, which would have differential effects on reinforcement-based motor learning and generalization. To test this hypothesis, two groups of participants learn a motor task in one direction and then relearn the same task in a new direction, receiving only performance-based reward or punishment score feedback. Our findings support our hypothesis, showing that reward led to slower learning but promoted generalization. On the other hand, punishment led to faster learning but impaired generalization. These behavioral differences may be due to different tendencies of movement variability in each group. The punishment group tended to explore more actively than the reward group during the initial learning phase, possibly due to loss aversion. In contrast, the reward group tended to explore more actively than the initial learning phase during the generalization test phase, seemingly recalling the strategy that led to the reward. These results suggest that reward and punishment may engage different neural mechanisms during reinforcement-based motor learning and generalization, with important implications for practical applications such as sports training and motor rehabilitation.NEW & NOTEWORTHY Although reinforcement learning is a significant motor learning process, the mechanisms underlying how the brain learns movements through reward and punishment are not fully understood. We modified a well-established motor adaptation task and used savings (faster relearning) to measure generalization. We found reward led to slower learning but promoted generalization, whereas punishment led to faster learning but impaired generalization, suggesting that reward and punishment may engage different neural mechanisms during reinforcement-based motor learning and generalization.

Alterations in Serotonin Neurotransmission in Hyperdopaminergic Rats Lacking the Dopamine Transporter

Biogenic amines dopamine (DA) and serotonin (5-HT) are among the most significant monoaminergic neurotransmitters in the central nervous system (CNS). Separately, the physiological roles of DA and 5-HT have been studied in detail, and progress has been made in understanding their roles in normal and various pathological conditions (Parkinson's disease, schizophrenia, addiction, depression, etc.). In this article we showed that knockout of the gene encoding DAT leads not only to a profound dysregulation of dopamine neurotransmission in the striatum but also in the midbrain, prefrontal cortex, hippocampus, medulla oblongata and spinal cord. Furthermore, significant changes were observed in the production of mRNA of enzymes of monoamine metabolism, as well as to a notable alteration in the tissue level of serotonin, most clearly manifested in the cerebellum and the spinal cord. The observed region-specific changes in the tissue levels of serotonin and in the expression of dopamine and serotonergic metabolism enzymes in rats with an excess of dopamine can indicate important consequences for the pharmacotherapy of drugs that modulate the dopaminergic system. The drugs that affect the dopaminergic system could potently affect the serotonergic system, and this fact is important to consider when predicting their possible therapeutic or side effects.

Reframing dopamine: A controlled controller at the limbic-motor interface

Pavlovian influences notoriously interfere with operant behaviour. Evidence suggests this interference sometimes coincides with the release of the neuromodulator dopamine in the nucleus accumbens. Suppressing such interference is one of the targets of cognitive control. Here, using the examples of active avoidance and omission behaviour, we examine the possibility that direct manipulation of the dopamine signal is an instrument of control itself. In particular, when instrumental and Pavlovian influences come into conflict, dopamine levels might be affected by the controlled deployment of a reframing mechanism that recasts the prospect of possible punishment as an opportunity to approach safety, and the prospect of future reward in terms of a possible loss of that reward. We operationalize this reframing mechanism and fit the resulting model to rodent behaviour from two paradigmatic experiments in which accumbens dopamine release was also measured. We show that in addition to matching animals' behaviour, the model predicts dopamine transients that capture some key features of observed dopamine release at the time of discriminative cues, supporting the idea that modulation of this neuromodulator is amongst the repertoire of cognitive control strategies.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Boosting Serotonin Increases Information Gathering by Reducing Subjective Cognitive Costs

Serotonin is implicated in the valuation of aversive costs, such as delay or physical effort. However, its role in governing sensitivity to cognitive effort, for example, deliberation costs during information gathering, is unclear. We show that treatment with a serotonergic antidepressant in healthy human individuals of either sex enhances a willingness to gather information when trying to maximize reward. Using computational modeling, we show this arises from a diminished sensitivity to subjective deliberation costs during the sampling process. This result is consistent with the notion that serotonin alleviates sensitivity to aversive costs in a domain-general fashion, with implications for its potential contribution to a positive impact on motivational deficits in psychiatric disorders.SIGNIFICANCE STATEMENT Gathering information about the world is essential for successfully navigating it. However, sampling information is costly, and we need to balance between gathering too little and too much information. The neurocomputational mechanisms underlying this arbitration between a putative gain, such as reward, and the associated costs, such as allocation of cognitive resources, remain unclear. In this study, we show that week-long daily treatment with a serotonergic antidepressant enhances a willingness to gather information when trying to maximize reward. Computational modeling indicates this arises from a reduced perception of aversive costs, rendering information gathering less cognitively effortful. This finding points to a candidate mechanism by which serotonergic treatment might help alleviate motivational deficits in a range of mental illnesses.

Explaining the description-experience gap in risky decision-making: learning and memory retention during experience as causal mechanisms

When making decisions based on probabilistic outcomes, people guide their behavior using knowledge gathered through both indirect descriptions and direct experience. Paradoxically, how people obtain information significantly impacts apparent preferences. A ubiquitous example is the description-experience gap: individuals seemingly overweight low probability events when probabilities are described yet underweight them when probabilities must be experienced firsthand. A leading explanation for this fundamental gap in decision-making is that probabilities are weighted differently when learned through description relative to experience, yet a formal theoretical account of the mechanism responsible for such weighting differences remains elusive. We demonstrate how various learning and memory retention models incorporating neuroscientifically motivated learning mechanisms can explain why probability weighting and valuation parameters often are found to vary across description and experience. In a simulation study, we show how learning through experience can lead to systematically biased estimates of probability weighting when using a traditional cumulative prospect theory model. We then use hierarchical Bayesian modeling and Bayesian model comparison to show how various learning and memory retention models capture participants' behavior over and above changes in outcome valuation and probability weighting, accounting for description and experience-based decisions in a within-subject experiment. We conclude with a discussion of how substantive models of psychological processes can lead to insights that heuristic statistical models fail to capture.

The effects of acute tryptophan depletion on instrumental reward learning in anorexia nervosa - an fMRI study

BACKGROUND

The serotonin (5-HT) hypothesis of anorexia nervosa (AN) posits that individuals predisposed toward or recovered from AN (recAN) have a central nervous hyperserotonergic state and therefore restrict food intake as a means to reduce 5-HT availability (via diminished tryptophan-derived precursor supply) and alleviate associated negative mood states. Importantly, the 5-HT system has also been generally implicated in reward processing, which has also been shown to be altered in AN.

METHODS

In this double-blind crossover study, 22 individuals recAN and 25 healthy control participants (HC) underwent functional magnetic resonance imaging (fMRI) while performing an established instrumental reward learning paradigm during acute tryptophan depletion (ATD; a dietary intervention that lowers central nervous 5-HT availability) as well as a sham depletion.

RESULTS

On a behavioral level, the main effects of reward and ATD were evident, but no group differences were found. fMRI analyses revealed a group × ATD × reward level interaction in the ventral anterior insula during reward anticipation as well as in the medial orbitofrontal cortex during reward consumption.

DISCUSSION

The precise pattern of results is suggestive of a 'normalization' of reward-related neural responses during ATD in recAN compared to HC. Our results lend further evidence to the 5-HT hypothesis of AN. Decreasing central nervous 5-HT synthesis and availability during ATD and possibly also by dieting may be a means to normalize 5-HT availability and associated brain processes.

Striatal adenosine A2A receptor involvement in normal and large nest building deer mice: perspectives on compulsivity and anxiety

Obsessive-compulsive disorder (OCD) is characterized by recurring obsessive thoughts and repetitive behaviors that are often associated with anxiety and perturbations in cortico-striatal signaling. Given the suboptimal response of OCD to current serotonergic interventions, there is a need to better understand the psychobiological mechanisms that may underlie the disorder. In this regard, investigations into adenosinergic processes might be fruitful. Indeed, adenosine modulates both anxiety- and motor behavioral output. Thus, we aimed to explore the potential associations between compulsive-like large nest building (LNB) behavior in deer mice, anxiety and adenosinergic processes. From an initial pool of 120 adult deer mice, 34 normal nest building (NNB)- and 32 LNB-expressing mice of both sexes were selected and exposed to either a normal water (wCTRL) or vehicle control (vCTRL), lorazepam (LOR) or istradefylline (ISTRA) for 7- (LOR) or 28 days after which nesting assessment was repeated and animals screened for anxiety-like behavior in an anxiogenic open field. Mice were then euthanized, the striatal tissue removed on ice and the adenosine A_2A receptor expression quantified. Our findings indicate that NNB and LNB behavior are not distinctly associated with measures of generalized anxiety and that ISTRA-induced changes in nesting expression are dissociated from changes in anxiety scores. Further, data from this investigation show that nesting in deer mice is directly related to striatal adenosine signaling, and that LNB is founded upon a lower degree of adenosinergic A_2A stimulation.

Psychopharmacological treatment of disruptive behavior in youths: systematic review and network meta-analysis

To conduct a systematic review of the comparative efficacy of various psychotropic medications for the treatment of disruptive behavior (DBs) in youths. To this aim, we systematically reviewed randomized clinical trials (RCTs) of various psychotropic medications targeting symptoms of DBs and applied network meta-analysis to investigate their relative efficacy. Fifty-five RCTs meeting the inclusion criteria were selected. To predict and interpret relative treatment efficacy, we compared the efficacy of various psychotropic medications prescribed for DB symptoms based on their mechanism of action. Network meta-analysis revealed that for reducing DBs, second-generation antipsychotics, stimulants, and non-stimulant ADHD medications were more efficacious than placebo, and second-generation antipsychotics were the most efficacious. The dopaminergic modulation of top-down inhibitory process by these medications is discussed in this review. This study offers information on the relative efficacy of various psychotropic medications for the treatment of DB, and insight into a potential neurobiological underpinning for those symptoms. It also illustrates the potential utility of these neurobiological mechanisms as a target for future treatment studies.

Innate cocaine-seeking vulnerability arising from loss of serotonin-mediated aversive effects of cocaine in rats

Cocaine blocks dopamine reuptake, thereby producing rewarding effects that are widely studied. However, cocaine also blocks serotonin uptake, which we show drives, in rats, individually variable aversive effects that depend on serotonin 2C receptors (5-HT2CRs) in the rostromedial tegmental nucleus (RMTg), a major GABAergic afferent to midbrain dopamine neurons. 5-HT2CRs produce depolarizing effects in RMTg neurons that are particularly strong in some rats, leading to aversive effects that reduce acquisition of and relapse to cocaine seeking. In contrast, 5-HT2CR signaling is largely lost after cocaine exposure in other rats, leading to reduced aversive effects and increased cocaine seeking. These results suggest a serotonergic biological marker of cocaine-seeking vulnerability that can be targeted to modulate drug seeking.

Fifty years on: Serotonin and depression

It has been over 50 years since the original serotonin hypothesis was proposed by the British Psychiatrist Alec Coppen. Recently, some authors have questioned the validity of the hypothesis. In this narrative review, we summarise the evidence for the serotonin hypothesis of depression, focusing on psychopharmacology and molecular imaging, as well as systems-level neuroscience.

Valence-partitioned learning signals drive choice behavior and phenomenal subjective experience in humans

How the human brain generates conscious phenomenal experience is a fundamental problem. In particular, it is unknown how variable and dynamic changes in subjective affect are driven by interactions with objective phenomena. We hypothesize a neurocomputational mechanism that generates valence-specific learning signals associated with 'what it is like' to be rewarded or punished. Our hypothesized model maintains a partition between appetitive and aversive information while generating independent and parallel reward and punishment learning signals. This valence-partitioned reinforcement learning (VPRL) model and its associated learning signals are shown to predict dynamic changes in 1) human choice behavior, 2) phenomenal subjective experience, and 3) BOLD-imaging responses that implicate a network of regions that process appetitive and aversive information that converge on the ventral striatum and ventromedial prefrontal cortex during moments of introspection. Our results demonstrate the utility of valence-partitioned reinforcement learning as a neurocomputational basis for investigating mechanisms that may drive conscious experience.

Learning predictive cognitive maps with spiking neurons during behavior and replays

The hippocampus has been proposed to encode environments using a representation that contains predictive information about likely future states, called the successor representation. However, it is not clear how such a representation could be learned in the hippocampal circuit. Here, we propose a plasticity rule that can learn this predictive map of the environment using a spiking neural network. We connect this biologically plausible plasticity rule to reinforcement learning, mathematically and numerically showing that it implements the TD-lambda algorithm. By spanning these different levels, we show how our framework naturally encompasses behavioral activity and replays, smoothly moving from rate to temporal coding, and allows learning over behavioral timescales with a plasticity rule acting on a timescale of milliseconds. We discuss how biological parameters such as dwelling times at states, neuronal firing rates and neuromodulation relate to the delay discounting parameter of the TD algorithm, and how they influence the learned representation. We also find that, in agreement with psychological studies and contrary to reinforcement learning theory, the discount factor decreases hyperbolically with time. Finally, our framework suggests a role for replays, in both aiding learning in novel environments and finding shortcut trajectories that were not experienced during behavior, in agreement with experimental data.

Influence of intra-abomasal administration of L-DOPA on circulating catecholamines and feed intake in cattle

Dopamine has multiple physiological functions including feed intake control in which it can act as an anorectic or orexigenic agent. This study had the objective to evaluate intra-abomasal administration of L-DOPA (levodopa; L-3,4-dihydroxyphenylalanine) from -Mucuna pruriens on circulating catecholamines, indicators of energy metabolism and feed intake in cattle. Eight Holstein steers (340 ± 20 kg) fitted with ruminal cannula were used in a replicated 4 x 4 Latin Square design experiment. Intra-abomasal infusion of L-DOPA at 0, 0.5, 1 and 2 mg/kg BW was carried out for seven days and blood samples were collected at 0, 30, 60, 120, 240 and 480 min from L-DOPA infusion on day 7. The area under the curve (AUC) of plasma L-DOPA and free dopamine increased quadratically with the administration of L-DOPA. However, the AUC of plasma total dopamine had a positive linear response with the increase of L-DOPA. Conversely, the serum 5-hydroxytriptophan (5-HTP), plasma serotonin, serum serotonin, serum tyrosine, plasma glucose and plasma free fatty acids were not affected by the intra-abomasal infusion of L-DOPA. The circulating concentration of the epinephrine, norepinephrine, serotonin, glucose and free fatty acids did not change with L-DOPA infusion. It can be concluded that intra-abomasal L-DOPA administration produced a strong increase in circulating dopamine with no change in energy metabolites and feed intake in cattle.

Temporal derivative computation in the dorsal raphe network revealed by an experimentally driven augmented integrate-and-fire modeling framework

By means of an expansive innervation, the serotonin (5-HT) neurons of the dorsal raphe nucleus (DRN) are positioned to enact coordinated modulation of circuits distributed across the entire brain in order to adaptively regulate behavior. Yet the network computations that emerge from the excitability and connectivity features of the DRN are still poorly understood. To gain insight into these computations, we began by carrying out a detailed electrophysiological characterization of genetically identified mouse 5-HT and somatostatin (SOM) neurons. We next developed a single-neuron modeling framework that combines the realism of Hodgkin-Huxley models with the simplicity and predictive power of generalized integrate-and-fire models. We found that feedforward inhibition of 5-HT neurons by heterogeneous SOM neurons implemented divisive inhibition, while endocannabinoid-mediated modulation of excitatory drive to the DRN increased the gain of 5-HT output. Our most striking finding was that the output of the DRN encodes a mixture of the intensity and temporal derivative of its input, and that the temporal derivative component dominates this mixture precisely when the input is increasing rapidly. This network computation primarily emerged from prominent adaptation mechanisms found in 5-HT neurons, including a previously undescribed dynamic threshold. By applying a bottom-up neural network modeling approach, our results suggest that the DRN is particularly apt to encode input changes over short timescales, reflecting one of the salient emerging computations that dominate its output to regulate behavior.

Memory decay and generalization following distinct motor learning mechanisms

Motor skill learning is considered to arise out of contributions from multiple learning mechanisms, including error-based learning (EBL), use-dependent learning (UDL), and reinforcement learning (RL). These learning mechanisms exhibit dissociable roles and engage different neural circuits during skill acquisition. However, it remains largely unknown how a newly formed motor memory acquired through each learning mechanism decays over time and whether distinct learning mechanisms produce different generalization patterns. Here, we used variants of reaching paradigms that dissociated these learning mechanisms to examine the time course of memory decay following each learning and the generalization patterns of each learning. We found that motor memories acquired through these learning mechanisms decayed as a function of time. Notably, 15 min, 6 h, and 24 h after acquisition, the memory of EBL decayed much greater than that of RL. The memory acquired through UDL faded away within a few minutes. Motor memories formed through EBL and RL for given movement directions generalized to untrained movement directions, with the generalization of EBL being greater than that of RL. In contrast, motor memory of UDL could not generalize to untrained movement directions. These results suggest that distinct learning mechanisms exhibit different patterns of memory decay and generalization.NEW & NOTEWORTHY Motor skill learning is likely to involve error-based learning, use-dependent plasticity, and operant reinforcement. Here, we showed that these dissociable learning mechanisms exhibited distinct patterns of memory decay and generalization. With a better understanding of the characteristics of these learning mechanisms, it becomes possible to regulate each learning process separately to improve neurological rehabilitation.

Regulation of social hierarchy learning by serotonin transporter availability

Learning one's status in a group is a fundamental process in building social hierarchies. Although animal studies suggest that serotonin (5-HT) signaling modulates learning social hierarchies, direct evidence in humans is lacking. Here we determined the relationship between serotonin transporter (SERT) availability and brain systems engaged in learning social ranks combining computational approaches with simultaneous PET-fMRI acquisition in healthy males. We also investigated the link between SERT availability and brain activity in a non-social control condition involving learning the payoffs of slot machines. Learning social ranks was modulated by the dorsal raphe nucleus (DRN) 5-HT function. BOLD ventral striatal response, tracking the rank of opponents, decreased with DRN SERT levels. Moreover, this link was specific to the social learning task. These findings demonstrate that 5-HT plays an influence on the computations required to learn social ranks.

Computational reinforcement learning, reward (and punishment), and dopamine in psychiatric disorders

In the DSM-5, psychiatric diagnoses are made based on self-reported symptoms and clinician-identified signs. Though helpful in choosing potential interventions based on the available regimens, this conceptualization of psychiatric diseases can limit basic science investigation into their underlying causes. The reward prediction error (RPE) hypothesis of dopamine neuron function posits that phasic dopamine signals encode the difference between the rewards a person expects and experiences. The computational framework from which this hypothesis was derived, temporal difference reinforcement learning (TDRL), is largely focused on reward processing rather than punishment learning. Many psychiatric disorders are characterized by aberrant behaviors, expectations, reward processing, and hypothesized dopaminergic signaling, but also characterized by suffering and the inability to change one's behavior despite negative consequences. In this review, we provide an overview of the RPE theory of phasic dopamine neuron activity and review the gains that have been made through the use of computational reinforcement learning theory as a framework for understanding changes in reward processing. The relative dearth of explicit accounts of punishment learning in computational reinforcement learning theory and its application in neuroscience is highlighted as a significant gap in current computational psychiatric research. Four disorders comprise the main focus of this review: two disorders of traditionally hypothesized hyperdopaminergic function, addiction and schizophrenia, followed by two disorders of traditionally hypothesized hypodopaminergic function, depression and post-traumatic stress disorder (PTSD). Insights gained from a reward processing based reinforcement learning framework about underlying dopaminergic mechanisms and the role of punishment learning (when available) are explored in each disorder. Concluding remarks focus on the future directions required to characterize neuropsychiatric disorders with a hypothesized cause of underlying dopaminergic transmission.

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T

The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

Edible insects prevent changes to brain monoamine profiles from malnourishment in weaned rats

Background: Childhood malnutrition can have devastating consequences on health, behavior, and cognition. Edible insects are sustainable low cost high protein and iron nutritious foods that can prevent malnutrition. However, it is unclear whether insect-based diets may help prevent changes to brain neurochemistry associated with malnutrition.Materials and Methods: Weanling male Sprague-Dawley rats were malnourished by feeding a low protein-iron diet (LPI, 5% protein and ∼2 ppm Fe) for 3 weeks or nourished by feeding a sufficient protein-iron diet (SPI, 15% protein 20 ppm FeSO₄) for the duration of the study. Following 3 weeks of LPI diet, three subsets of the malnourished rats were placed on repletion diets supplemented with cricket, palm weevil larvae, or the SPI diet for 2 weeks, while the remaining rats continued the LPI diet for an additional 2 weeks. Monoamine-related neurochemicals (e.g. serotonin (5-HT), dopamine (DA), norepinephrine) and select monoamine metabolites were measured in the hypothalamus, hippocampus, striatum, and prefrontal cortex using Ultra High-Performance Liquid Chromatography.Results: Five weeks of LPI diets disrupted brain monoamines, most notable in the hypothalamus. Two weeks supplementation with cricket and palm weevil larvae diets prevented changes to measures of 5-HT and DA turnover in the hippocampus and hypothalamus. Moreover, these insect diets prevented the malnutrition-induced imbalance of 5-HT and DA metabolites in the hippocampus, striatum, and hypothalamus.Conclusion: Edible insects such as cricket and palm weevil larvae could be sustainable nutrition intervention to prevent behavioral and cognitive impairment associated abnormal brain monoamine activities that results from early life malnutrition.

Flexibility in valenced reinforcement learning computations across development

Optimal integration of positive and negative outcomes during learning varies depending on an environment's reward statistics. The present study investigated the extent to which children, adolescents, and adults (N = 142 8-25 year-olds, 55% female, 42% White, 31% Asian, 17% mixed race, and 8% Black; data collected in 2021) adapt their weighting of better-than-expected and worse-than-expected outcomes when learning from reinforcement. Participants made choices across two contexts: one in which weighting positive outcomes more heavily than negative outcomes led to better performance, and one in which the reverse was true. Reinforcement learning modeling revealed that across age, participants shifted their valence biases in accordance with environmental structure. Exploratory analyses revealed strengthening of context-dependent flexibility with increasing age.

Understanding the effects of serotonin in the brain through its role in the gastrointestinal tract

The neuromodulatory arousal system imbues the nervous system with the flexibility and robustness required to facilitate adaptive behaviour. While there are well-understood mechanisms linking dopamine, noradrenaline and acetylcholine to distinct behavioural states, similar conclusions have not been as readily available for serotonin. Fascinatingly, despite clear links between serotonergic function and cognitive capacities as diverse as reward processing, exploration, and the psychedelic experience, over 95% of the serotonin in the body is released in the gastrointestinal tract, where it controls digestive muscle contractions (peristalsis). Here, we argue that framing neural serotonin as a rostral extension of the gastrointestinal serotonergic system dissolves much of the mystery associated with the central serotonergic system. Specifically, we outline that central serotonin activity mimics the effects of a digestion/satiety circuit mediated by hypothalamic control over descending serotonergic nuclei in the brainstem. We review commonalities and differences between these two circuits, with a focus on the heterogeneous expression of different classes of serotonin receptors in the brain. Much in the way that serotonin-induced peristalsis facilitates the work of digestion, serotonergic influences over cognition can be reframed as performing the work of cognition. Extending this analogy, we argue that the central serotonergic system allows the brain to arbitrate between different cognitive modes as a function of serotonergic tone: low activity facilitates cognitive automaticity, whereas higher activity helps to identify flexible solutions to problems, particularly if and when the initial responses fail. This perspective sheds light on otherwise disparate capacities mediated by serotonin, and also helps to understand why there are such pervasive links between serotonergic pathology and the symptoms of psychiatric disorders.

Different brain systems support learning from received and avoided pain during human pain-avoidance learning

Both unexpected pain and unexpected pain absence can drive avoidance learning, but whether they do so via shared or separate neural and neurochemical systems is largely unknown. To address this issue, we combined an instrumental pain-avoidance learning task with computational modeling, functional magnetic resonance imaging (fMRI), and pharmacological manipulations of the dopaminergic (100 mg levodopa) and opioidergic (50 mg naltrexone) systems (N = 83). Computational modeling provided evidence that untreated participants learned more from received than avoided pain. Our dopamine and opioid manipulations negated this learning asymmetry by selectively increasing learning rates for avoided pain. Furthermore, our fMRI analyses revealed that pain prediction errors were encoded in subcortical and limbic brain regions, whereas no-pain prediction errors were encoded in frontal and parietal cortical regions. However, we found no effects of our pharmacological manipulations on the neural encoding of prediction errors. Together, our results suggest that human pain-avoidance learning is supported by separate threat- and safety-learning systems, and that dopamine and endogenous opioids specifically regulate learning from successfully avoided pain.

Design Principles for Neurorobotics

In their book “How the Body Shapes the Way We Think: A New View of Intelligence,” Pfeifer and Bongard put forth an embodied approach to cognition. Because of this position, many of their robot examples demonstrated “intelligent” behavior despite limited neural processing. It is our belief that neurorobots should attempt to follow many of these principles. In this article, we discuss a number of principles to consider when designing neurorobots and experiments using robots to test brain theories. These principles are strongly inspired by Pfeifer and Bongard, but build on their design principles by grounding them in neuroscience and by adding principles based on neuroscience research. Our design principles fall into three categories. First, organisms must react quickly and appropriately to events. Second, organisms must have the ability to learn and remember over their lifetimes. Third, organisms must weigh options that are crucial for survival. We believe that by following these design principles a robot's behavior will be more naturalistic and more successful.

Dopamine and serotonin interplay for valence-based spatial learning

Dopamine (DA) and serotonin (5-HT) are important neuromodulators of synaptic plasticity that have been linked to learning from positive or negative outcomes or valence-based learning. In the hippocampus, both affect long-term plasticity but play different roles in encoding uncertainty or predicted reward. DA has been related to positive valence, from reward consumption or avoidance behavior, and 5-HT to aversive encoding. We propose DA produces overall LTP while 5-HT elicits LTD. Here, we compare two reward-modulated spike timing-dependent plasticity (R-STDP) rules to describe the action of these neuromodulators. We examined their role in cognitive performance and flexibility for computational models of the Morris water maze task and reversal learning. Our results show that the interplay of DA and 5-HT improves learning performance and can explain experimental evidence. This study reinforces the importance of neuromodulation in determining the direction of plasticity.

Optogenetic Stimulation of Midbrain Dopamine Neurons Produces Striatal Serotonin Release

Targeting neurons with light-driven opsins is widely used to investigate cell-specific responses. We transfected midbrain dopamine neurons with the excitatory opsin Chrimson. Extracellular basal and stimulated neurotransmitter levels in the dorsal striatum were measured by microdialysis in awake mice. Optical activation of dopamine cell bodies evoked terminal dopamine release in the striatum. Multiplexed analysis of dialysate samples revealed that the evoked dopamine was accompanied by temporally coupled increases in striatal 3-methoxytyramine, an extracellular dopamine metabolite, and in serotonin. We investigated a mechanism for dopamine-serotonin interactions involving striatal dopamine receptors. However, the evoked serotonin associated with optical stimulation of dopamine neurons was not abolished by striatal D1- or D2-like receptor inhibition. Although the mechanisms underlying the coupling of striatal dopamine and serotonin remain unclear, these findings illustrate advantages of multiplexed measurements for uncovering functional interactions between neurotransmitter systems. Furthermore, they suggest that the output of optogenetic manipulations may extend beyond opsin-expressing neuronal populations.

Anxiety predicts impulsive-compulsive behaviours in Parkinson's disease: Clinical relevance and theoretical implications

Patients with Parkinson's disease (PD) often present symptoms of anxiety, depression and apathy. These negative affect manifestations have been recently associated with the presence of impulsive compulsive behaviours (ICBs). However, their relation with the use of dopamine replacement therapy (DRT), a renewed risk factor for ICBs, is still not fully understood. Elucidating the role of these different ICBs predictors in PD could inform both prevention/intervention recommendations as well as theoretical models. In the present study, we have analysed data collected in 417 PD patients, 50 patients with Parkinsonian symptoms but with scan without evidence of dopaminergic deficit (SWEDD), and 185 healthy controls (HC). We examined each patient's clinical profile over a two-year time window, investigating the role of both negative affect and DRT on ICBs. Results confirmed the presence of higher levels of anxiety in both the clinical groups, and of higher level of ICBs in SWEDD patients, respect to both PD and HC. Mixed model analyses revealed a statistically significant association between anxiety and ICBs in the SWEDD patients who did not take any DRT. Findings suggest the independence between anxiety and DRT in ICBs development, and provide new evidence for the motivational opponency theoretical framework.

Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making

Nervous systems evolved to effectively navigate the dynamics of the environment to achieve their goals. One framework used to study this fundamental problem arose in the study of learning and decision-making. In this framework, the demands of effective behavior require slow dynamics-on the scale of seconds to minutes-of networks of neurons. Here, we review the phenomena and mechanisms involved. Using vignettes from a few species and areas of the nervous system, we view neuromodulators as key substrates for temporal scaling of neuronal dynamics. Expected final online publication date for the Annual Review of Neuroscience, Volume 45 is July 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

The role of state uncertainty in the dynamics of dopamine

Reinforcement learning models of the basal ganglia map the phasic dopamine signal to reward prediction errors (RPEs). Conventional models assert that, when a stimulus predicts a reward with fixed delay, dopamine activity during the delay should converge to baseline through learning. However, recent studies have found that dopamine ramps up before reward in certain conditions even after learning, thus challenging the conventional models. In this work, we show that sensory feedback causes an unbiased learner to produce RPE ramps. Our model predicts that when feedback gradually decreases during a trial, dopamine activity should resemble a "bump," whose ramp-up phase should, furthermore, be greater than that of conditions where the feedback stays high. We trained mice on a virtual navigation task with varying brightness, and both predictions were empirically observed. In sum, our theoretical and experimental results reconcile the seemingly conflicting data on dopamine behaviors under the RPE hypothesis.

A Predictive Coding Framework for Understanding Major Depression

Predictive coding models of brain processing propose that top-down cortical signals promote efficient neural signaling by carrying predictions about incoming sensory information. These "priors" serve to constrain bottom-up signal propagation where prediction errors are carried via feedforward mechanisms. Depression, traditionally viewed as a disorder characterized by negative cognitive biases, is associated with disrupted reward prediction error encoding and signaling. Accumulating evidence also suggests that depression is characterized by impaired local and long-range prediction signaling across multiple sensory domains. This review highlights the electrophysiological and neuroimaging evidence for disrupted predictive processing in depression. The discussion is framed around the manner in which disrupted generative predictions about the sensorium could lead to depressive symptomatology, including anhedonia and negative bias. In particular, the review focuses on studies of sensory deviance detection and reward processing, highlighting research evidence for both disrupted generative predictions and prediction error signaling in depression. The role of the monoaminergic and glutamatergic systems in predictive coding processes is also discussed. This review provides a novel framework for understanding depression using predictive coding principles and establishes a foundational roadmap for potential future research.

A dynamical systems view of neuroethology: Uncovering stateful computation in natural behaviors

State-dependent computation is key to cognition in both biological and artificial systems. Alan Turing recognized the power of stateful computation when he created the Turing machine with theoretically infinite computational capacity in 1936. Independently, by 1950, ethologists such as Tinbergen and Lorenz also began to implicitly embed rudimentary forms of state-dependent computation to create qualitative models of internal drives and naturally occurring animal behaviors. Here, we reformulate core ethological concepts in explicitly dynamical systems terms for stateful computation. We examine, based on a wealth of recent neural data collected during complex innate behaviors across species, the neural dynamics that determine the temporal structure of internal states. We will also discuss the degree to which the brain can be hierarchically partitioned into nested dynamical systems and the need for a multi-dimensional state-space model of the neuromodulatory system that underlies motivational and affective states.

Serotonin neurons modulate learning rate through uncertainty

Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change are difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.

Complex functional brain network properties in anorexia nervosa

BACKGROUND

Anorexia nervosa (AN) is a disorder characterized by an incapacitating fear of weight gain and by a disturbance in the way the body is experienced, facets that motivate dangerous weight loss behaviors. Multimodal neuroimaging studies highlight atypical neural activity in brain networks involved in interoceptive awareness and reward processing.

METHODS

The current study used resting-state neuroimaging to model the architecture of large-scale functional brain networks and characterize network properties of individual brain regions to clinical measures. Resting-state neuroimaging was conducted in 62 adolescents, 22 (21 female) with a history of AN and 40 (39 female) healthy controls (HCs). Sensorimotor and basal ganglia regions, as part of a 165-region whole-brain network, were investigated. Subject-specific functional brain networks were computed to index centrality. A contrast analysis within the general linear model covarying for age was performed. Correlations between network properties and behavioral measures were conducted (significance q < .05).

RESULTS

Compared to HCs, AN had lower connectivity from sensorimotor regions, and greater connectivity from the left caudate nucleus to the right postcentral gyrus. AN demonstrated lower sensorimotor centrality, but higher basal ganglia centrality. Sensorimotor connectivity dyads and centrality exhibited negative correlations with body dissatisfaction and drive for thinness, two essential features of AN.

CONCLUSIONS

These findings suggest that AN is associated with greater communication from the basal ganglia, and lower information propagation in sensorimotor cortices. This is consistent with the clinical presentation of AN, where individuals exhibit patterns of rigid habitual behavior that is not responsive to bodily needs, and seem "disconnected" from their bodies.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory

As individuals learn through trial and error, some are more influenced by good outcomes, while others weight bad outcomes more heavily. Such valence biases may also influence memory for past experiences. Here, we examined whether valence asymmetries in reinforcement learning change across adolescence, and whether individual learning asymmetries bias the content of subsequent memory. Participants ages 8-27 learned the values of 'point machines,' after which their memory for trial-unique images presented with choice outcomes was assessed. Relative to children and adults, adolescents overweighted worse-than-expected outcomes during learning. Individuals' valence biases modulated incidental memory, such that those who prioritized worse- (or better-) than-expected outcomes during learning were also more likely to remember images paired with these outcomes, an effect reproduced in an independent dataset. Collectively, these results highlight age-related changes in the computation of subjective value and demonstrate that a valence-asymmetric valuation process influences how information is prioritized in episodic memory.

Dorsal Raphe 5-HT Neurons Utilize, But Do Not Generate, Negative Aversive Prediction Errors

The dorsal raphe nucleus (DRN) contains the largest population of serotonin (5-HT) neurons in the central nervous system. 5-HT, synthesized via tryptophan hydroxylase 2 (Tph2), is a widely functioning neuromodulator implicated in fear learning. Here, we sought to investigate whether DRN 5-HT is necessary to reduce fear via negative prediction error (–PE). Using male and female TPH2-cre rats, DRN^tph2+ cells were selectively deleted via cre-caspase (rAAV5-Flex-taCasp3-TEVp) in experiment 1. Rats then underwent fear discrimination during which three cues were associated with unique foot shock probabilities: safety p = 0.00, uncertainty p = 0.375, and danger p = 1.00. Rats then received selective extinction to the uncertainty cue, a behavioral manipulation designed to probe –PE. Deleting DRN^tph2+ cells had no impact on initial discrimination but slowed selective extinction. In experiment 2, we used a within-subjects optogenetic inhibition design to causally implicate DRN^tph2+ cells in prediction error signaling. Male and female TPH2-cre rats received intra-DRN infusions of cre-dependent halorhodopsin (rAAV5-Ef1a-DIO-eNpHR3.0-eYFP) or cre-YFP. DRN^tph2+ cells were inhibited specifically during the time of prediction error or a control period. Illumination during either positive prediction error (+PE) or control periods had no impact on fear to the uncertainty cue. Inhibition of DRN^tph2+ cells at the time of –PE did not impact immediate fear, but facilitated selective extinction in postillumination sessions. Together, these results demonstrate a role for DRN^tph2+ cells in using, but not generating, –PE to weaken cue-shock associations.

Impaired Learning From Negative Feedback in Stimulant Use Disorder: Dopaminergic Modulation

BACKGROUND

Drug-induced alterations to the dopamine system in stimulant use disorder (SUD) are hypothesized to impair reinforcement learning (RL). Computational modeling enables the investigation of the latent processes of RL in SUD patients, which could elucidate the nature of their impairments.

METHODS

We investigated RL in 44 SUD patients and 41 healthy control participants using a probabilistic RL task that assesses learning from reward and punishment separately. In an independent sample, we determined the modulatory role of dopamine in RL following a single dose of the dopamine D2/3 receptor antagonist amisulpride (400 mg) and the agonist pramipexole (0.5 mg) in a randomised, double-blind, placebo-controlled, crossover design. We analyzed task performance using computational modelling and hypothesized that RL impairments in SUD patients would be differentially modulated by a dopamine D2/3 receptor antagonist and agonist.

RESULTS

Computational analyses in both samples revealed significantly reduced learning rates from punishment in SUD patients compared with healthy controls, whilst their reward learning rates were not measurably impaired. In addition, the dopaminergic receptor agents modulated RL parameters differentially in both groups. Both amisulpride and pramipexole impaired RL parameters in healthy participants, but ameliorated learning from punishment in SUD patients.

CONCLUSION

Our findings suggest that RL impairments seen in SUD patients are associated with altered dopamine function.