A role for reward valuation in the serotonergic modulation of impulsivity

Striatal Serotonin Release Signals Reward Value

Serotonin modulates diverse phenotypes and functions including depressive, aggressive, impulsive, and feeding behaviors, all of which have reward-related components. To date, research has focused on understanding these effects by measuring and manipulating dorsal raphe serotonin neurons and using single-receptor approaches. These studies have led to a better understanding of the heterogeneity of serotonin actions on behavior; however, they leave open many questions about the timing and location of serotonin's actions modulating the neural circuits that drive these behaviors. Recent advances in genetically encoded fluorescent biosensors, including the GPCR activation-based sensor for serotonin (GRAB-5-HT), enable the measurement of serotonin release in mice on a timescale compatible with a single rewarding event without corelease confounds. Given substantial evidence from slice electrophysiology experiments showing that serotonin influences neural activity of the striatal circuitry, and the known role of the dorsal medial striatal (DMS) in reward-directed behavior, we focused on understanding the parameters and timing that govern serotonin release in the DMS in the context of reward consumption, external reward value, internal state, and cued reward. Overall, we found that serotonin release is associated with each of these and encodes reward anticipation, value, approach, and consumption in the DMS.

Reward value and internal state differentially drive impulsivity and motivation

Goal-directed behavior is influenced by both reward value as well as internal state. A large body of research has focused on how reward value and internal drives such as hunger influence motivation in rodent models, however less work has focused on how these factors may differentially affect impulsivity. In these studies, we tested the effect of internal drive versus reward value on different facets of reward-related behavior including impulsive action, impulsive choice and, motivation. We varied reward value by changing the concentration of sucrose in the reward outcome, and varied internal drive by manipulating thirst through water restriction. Consistent with the literature we found that both internal state and reward value influenced motivation. However, we found that in high effort paradigms, only internal state influenced motivation with minimal effects of reward value. Interestingly, we found that internal state and reward value differentially influence different subtypes of impulsivity. Internal state, and to a lesser extent, reward value, influenced impulsive action as measured by premature responding. On the other hand, there were minimal effects of either reward value or homeostatic state on impulsive choice as measured by delay discounting. Overall, these studies begin to address how internal state and reward value differentially drive impulsive behavior. Understanding how these factors influence impulsivity is important for developing behavioral interventions and treatment targets for patients with dysregulated motivated or impulsive behavior.

More widespread and rigid neuronal representation of reward expectation underlies impulsive choices

Impulsive choices prioritize smaller, more immediate rewards over larger, delayed, or potentially uncertain rewards. Impulsive choices are a critical aspect of substance use disorders and maladaptive decision-making across the lifespan. Here, we sought to understand the neuronal underpinnings of expected reward and risk estimation on a trial-by-trial basis during impulsive choices. To do so, we acquired electrical recordings from the human brain while participants carried out a risky decision-making task designed to measure choice impulsivity. Behaviorally, we found a reward-accuracy tradeoff, whereby more impulsive choosers were more accurate at the task, opting for a more immediate reward while compromising overall task performance. We then examined how neuronal populations across frontal, temporal, and limbic brain regions parametrically encoded reinforcement learning model variables, namely reward and risk expectation and surprise, across trials. We found more widespread representations of reward value expectation and prediction error in more impulsive choosers, whereas less impulsive choosers preferentially represented risk expectation. A regional analysis of reward and risk encoding highlighted the anterior cingulate cortex for value expectation, the anterior insula for risk expectation and surprise, and distinct regional encoding between impulsivity groups. Beyond describing trial-by-trial population neuronal representations of reward and risk variables, these results suggest impaired inhibitory control and model-free learning underpinnings of impulsive choice. These findings shed light on neural processes underlying reinforced learning and decision-making in uncertain environments and how these processes may function in psychiatric disorders.

Reduced serotonergic transmission alters sensitivity to cost and reward via 5-HT1A and 5-HT1B receptors in monkeys

Serotonin (5-HT) deficiency is a core biological pathology underlying depression and other psychiatric disorders whose key symptoms include decreased motivation. However, the exact role of 5-HT in motivation remains controversial and elusive. Here, we pharmacologically manipulated the 5-HT system in macaque monkeys and quantified the effects on motivation for goal-directed actions in terms of incentives and costs. Reversible inhibition of 5-HT synthesis increased errors and reaction times on goal-directed tasks, indicating reduced motivation. Analysis found incentive-dependent and cost-dependent components of this reduction. To identify the receptor subtypes that mediate cost and incentive, we systemically administered antagonists specific to 4 major 5-HT receptor subtypes: 5-HT1A, 5-HT1B, 5-HT2A, and 5-HT4. Positron emission tomography (PET) visualized the unique distribution of each subtype in limbic brain regions and determined the systemic dosage for antagonists that would achieve approximately 30% occupancy. Only blockade of 5-HT1A decreased motivation through changes in both expected cost and incentive; sensitivity to future workload and time delay to reward increased (cost) and reward value decreased (incentive). Blocking the 5-HT1B receptor also reduced motivation through decreased incentive, although it did not affect expected cost. These results suggest that 5-HT deficiency disrupts 2 processes, the subjective valuation of costs and rewards, via 5-HT1A and 5-HT1B receptors, thus leading to reduced motivation.

The contributions of neonatal inhalation of copper to air pollution-induced neurodevelopmental outcomes in mice

Exposures to ambient ultrafine particle (UFP) air pollution (AP) during the early postnatal period in mice (equivalent to human third trimester brain development) produce male-biased changes in brain structure, including ventriculomegaly, reduced brain myelination, alterations in neurotransmitters and glial activation, as well as impulsive-like behavioral characteristics, all of which are also features characteristic of male-biased neurodevelopmental disorders (NDDs). The purpose of this study was to ascertain the extent to which inhaled Cu, a common contaminant of AP that is also dysregulated across multiple NDDs, might contribute to these phenotypes. For this purpose, C57BL/6J mice were exposed from postnatal days 4-7 and 10-13 for 4 hr/day to inhaled copper oxide (CuxOy) nanoparticles at an environmentally relevant concentration averaging 171.9 ng/m3. Changes in brain metal homeostasis and neurotransmitter levels were determined following termination of exposure (postnatal day 14), while behavioral changes were assessed in adulthood. CuxOy inhalation modified cortical metal homeostasis and produced male-biased disruption of striatal neurotransmitters, with marked increases in dopaminergic function, as well as excitatory/inhibitory imbalance and reductions in serotonergic function. Impulsive-like behaviors in a fixed ratio (FR) waiting-for-reward schedule and a fixed interval (FI) schedule of food reward occurred in both sexes, but more prominently in males, effects which could not be attributed to altered locomotor activity or short-term memory. Inhaled Cu as from AP exposures, at environmentally relevant levels experienced during development, may contribute to impaired brain function, as shown by its ability to disrupt brain metal homeostasis and striatal neurotransmission. In addition, its ability to evoke impulsive-like behavior, particularly in male offspring, may be related to striatal dopaminergic dysfunction that is known to mediate such behaviors. As such, regulation of air Cu levels may be protective of public health.

A Role for Serotonin in Modulating Opposing Drive and Brake Circuits of Impulsivity

Impulsivity generally refers to a deficit in inhibition, with a focus on understanding the neural circuits which constitute the "brake" on actions and gratification. It is likely that increased impulsivity can arise not only from reduced inhibition, but also from a heightened or exaggerated excitatory "drive." For example, an action which has more vigor, or is fueled by either increased incentive salience or a stronger action-outcome association, may be harder to inhibit. From this perspective, this review focuses on impulse control as a competition over behavioral output between an initially learned response-reward outcome association, and a subsequently acquired opposing inhibitory association. Our goal is to present a synthesis of research from humans and animal models that supports this dual-systems approach to understanding the behavioral and neural substrates that contribute to impulsivity, with a focus on the neuromodulatory role of serotonin. We review evidence for the role of serotonin signaling in mediating the balance of the "drive" and "brake" circuits. Additionally, we consider parallels of these competing instrumental systems in impulsivity within classical conditioning processes (e.g., extinction) in order to point us to potential behavioral and neural mechanisms that may modulate the competing instrumental associations. Finally, we consider how the balance of these competing associations might contribute to, or be extracted from, our experimental assessments of impulsivity. A careful understanding of the underlying behavioral and circuit level contributions to impulsivity is important for understanding the pathogenesis of increased impulsivity present in a number of psychiatric disorders. Pathological levels of impulsivity in such disorders are likely subserved by deficits in the balance of motivational and inhibitory processes.

Serotonin 1B receptor effects on response inhibition are independent of inhibitory learning

Impulsivity is defined in terms of deficits in instrumental response inhibition, when the inability to withhold an action produces a negative outcome. However, there are many behavioral and cognitive constructs which theoretically could contribute to disordered impulsivity, including Pavlovian responding, which few studies have considered in this context. In the present set of studies, we examine Pavlovian inhibitory learning and excitatory responding in a mouse model for dysregulated impulsivity, specifically, mice lacking the serotonin 1B receptor (5-HT_1BR). Consistent with previous results, we show that these mice display increased impulsivity as measured by premature responding in the operant 5-choice serial reaction time test. In a Pavlovian conditioned inhibition paradigm, they also show a decreased ability to withhold responding, but importantly have an intact ability to learn inhibitory associations. In a Pavlovian appetitive conditioning experiment, 5-HT_1BR knockout mice show normal responding under a positive contingency schedule, however, they display increased responding to cues presented on an independent schedule from reinforcement in a zero contingency schedule. Interestingly this difference does not occur when the cues are explicitly unpaired in a negative contingency schedule, nor during a 25% reinforcement schedule. Overall, while our results show that the deficits in operant response inhibition in mice lacking 5-HT_1BR are likely not due to Pavlovian inhibitory or excitatory learning, it is relevant to consider associative learning in the context of dysregulated impulsive behavior.