Serotonergic neurons signal reward and punishment on multiple timescales

Global Transcriptome Changes That Accompany Alterations in Serotonin Levels in Caenorhabditis elegans

Serotonin (5-hydroxytryptamine, 5-HT), is a phylogenetically ancient molecule best characterized as a neurotransmitter that modulates multiple aspects of mood and social cognition. The roles that 5-HT plays in normal and abnormal behavior are not fully understood but have been posited to be due to its common function as a 'defense signal'. However, 5-HT levels also systemically impact cell physiology, modulating cell division, migration, apoptosis, mitochondrial biogenesis, cellular metabolism and differentiation. Whether these diverse cellular effects of 5-HT also share a common basis is unclear. C. elegans provides an ideal system to interrogate the systemic effects of 5-HT, since lacking a blood-brain barrier, 5-HT synthesized and released by neurons permeates the organism to modulate neuronal as well as non-neuronal cells throughout the body. Here we used RNA-Seq to characterize the systemic changes in gene expression that occur in C. elegans upon altering 5-HT levels, and compared the transcriptomes to published datasets. We find that an acute increase in 5-HT is accompanied by a global decrease in gene expression levels, upregulation of genes involved in stress pathways, changes that significantly correlate with the published transcriptomes of animals that have activated defense and immune responses, and an increase in levels of phosphorylated eukaryotic initiation factor, eIF2α. In 5-HT deficient animals lacking tryptophan hydroxylase (tph-1(mg280)II) there is a net increase in gene expression, with an overrepresentation of genes related to development and chromatin. Surprisingly, the transcriptomes of animals with acute increases in 5-HT levels, and 5-HT deficiency do not overlap with transcriptomes of mutants with whom they share striking physiological resemblance. These studies are the first to catalog systemic transcriptome changes that occur upon alterations in 5-HT levels. They further show that in C. elegans changes in gene expression upon altering 5-HT levels, and changes in physiology, are not directly correlated.

Open-Source Joystick Manipulandum for Decision-Making, Reaching, and Motor Control Studies in Mice

To make full use of optogenetic and molecular techniques in the study of motor control, rich behavioral paradigms for rodents must rise to the same level of sophistication and applicability. We describe the layout, construction, use and analysis of data from joystick-based reaching in a head-fixed mouse. The step-by-step guide is designed for both experienced rodent motor labs and new groups looking to enter into this research space. Using this platform, mice learn to consistently perform large, easily-quantified reaches, including during a two-armed bandit probabilistic learning task. The metrics of performance (reach trajectory, amplitude, speed, duration, and inter-reach interval) can be used to quantify behavior or administer stimulation in closed loop with behavior. We provide a highly customizable, low cost and reproducible open-source behavior training platform for studying motor control, decision-making, and reaching reaction time. The development of this software and hardware platform enables behavioral work to complement recent advances in rodents, while remaining accessible to smaller institutions and labs, thus providing a high-throughput method to study unexplored features of action selection, motivation, and value-based decisions.

Reward Timing and Its Expression by Inhibitory Interneurons in the Mouse Primary Visual Cortex

The primary sensory cortex has historically been studied as a low-level feature detector, but has more recently been implicated in many higher-level cognitive functions. For instance, after an animal learns that a light predicts water at a fixed delay, neurons in the primary visual cortex (V1) can produce "reward timing activity" (i.e., spike modulation of various forms that relate the interval between the visual stimulus and expected reward). Local manipulations to V1 implicate it as a site of learning reward timing activity (as opposed to simply reporting timing information from another region via feedback input). However, the manner by which V1 then produces these representations is unknown. Here, we combine behavior, in vivo electrophysiology, and optogenetics to investigate the characteristics of and circuit mechanisms underlying V1 reward timing in the head-fixed mouse. We find that reward timing activity is present in mouse V1, that inhibitory interneurons participate in reward timing, and that these representations are consistent with a theorized network architecture. Together, these results deepen our understanding of V1 reward timing and the manner by which it is produced.

Embracing diversity in the 5-HT neuronal system

Neurons that synthesize and release 5-hydroxytryptamine (5-HT; serotonin) express a core set of genes that establish and maintain this neurotransmitter phenotype and distinguish these neurons from other brain cells. Beyond a shared 5-HTergic phenotype, these neurons display divergent cellular properties in relation to anatomy, morphology, hodology, electrophysiology and gene expression, including differential expression of molecules supporting co-transmission of additional neurotransmitters. This diversity suggests that functionally heterogeneous subtypes of 5-HT neurons exist, but linking subsets of these neurons to particular functions has been technically challenging. We discuss recent data from molecular genetic, genomic and functional methods that, when coupled with classical findings, yield a reframing of the 5-HT neuronal system as a conglomeration of diverse subsystems with potential to inspire novel, more targeted therapies for clinically distinct 5-HT-related disorders.

Transient and sustained effects of dopamine and serotonin signaling in motivation-related behavior

Pharmacological studies of antidepressants and atypical antipsychotics have suggested a role of dopamine and serotonin signaling in depression. However, depressive symptoms and treatment effects are difficult to explain based simply on brain-wide decrease or increase in the concentrations of these molecules. Recent animal studies using advanced neuronal manipulation and observation techniques have revealed detailed dopamine and serotonin dynamics that regulate diverse aspects of motivation-related behavior. Dopamine and serotonin transiently modulate moment-to-moment behavior at timescales ranging from sub-second to minutes and also produce persistent effects, such as reward-related learning and stress responses that last longer than several days. Transient and sustained effects often exhibit specific roles depending on the projection sites, where distinct synaptic and cellular mechanisms are required to process the neurotransmitters for each transient and sustained timescale. Therefore, it appears that specific aspects of motivation-related behavior are regulated by distinct synaptic and cellular mechanisms in specific brain regions that underlie the transient and sustained effects of dopamine and serotonin signaling. Recent clinical studies have implied that subjects with depressive symptoms show impaired transient and sustained signaling functions; moreover, they exhibit heterogeneity in depressive symptoms and neuronal dysfunction. Depressive symptoms may be explained by the dysfunction of each transient and sustained signaling mechanism, and distinct patterns of impairment in the relevant mechanisms may explain the heterogeneity of symptoms. Thus, detailed understanding of dopamine and serotonin signaling may provide new insight into depressive symptoms.

Animal affect and decision-making

The scientific study of animal affect (emotion) is an area of growing interest. Whilst research on mechanism and causation has predominated, the study of function is less advanced. This is not due to a lack of hypotheses; in both humans and animals, affective states are frequently proposed to play a pivotal role in coordinating adaptive responses and decisions. However, exactly how they might do this (what processes might implement this function) is often left rather vague. Here we propose a framework for integrating animal affect and decision-making that is couched in modern decision theory and employs an operational definition that aligns with dimensional concepts of core affect and renders animal affect empirically tractable. We develop a model of how core affect, including short-term (emotion-like) and longer-term (mood-like) states, influence decision-making via processes that we label affective options, affective predictions, and affective outcomes and which correspond to similar concepts in schema of the links between human emotion and decision-making. Our framework is generalisable across species and generates questions for future research.

Opposing Contributions of GABAergic and Glutamatergic Ventral Pallidal Neurons to Motivational Behaviors

The ventral pallidum (VP) is critical for invigorating reward seeking and is also involved in punishment avoidance, but how it contributes to such opposing behavioral actions remains unclear. Here, we show that GABAergic and glutamatergic VP neurons selectively control behavior in opposing motivational contexts. In vivo recording combined with optogenetics in mice revealed that these two populations oppositely encode positive and negative motivational value, are differentially modulated by animal's internal state, and determine the behavioral response during motivational conflict. Furthermore, GABAergic VP neurons are essential for movements toward reward in a positive motivational context but suppress movements in an aversive context. In contrast, glutamatergic VP neurons are essential for movements to avoid a threat but suppress movements in an appetitive context. Our results indicate that GABAergic and glutamatergic VP neurons encode the drive for approach and avoidance, respectively, with the balance between their activities determining the type of motivational behavior.

Serotonergic control of GABAergic inhibition in the lateral amygdala

Much evidence implicates the serotonergic regulation of the amygdala in anxiety. Thus the present study was undertaken to characterize the influence of serotonin (5-HT) on principal neurons (PNs) of the rat lateral amygdala (LA), using whole cell recordings in vitro. Because inhibition is a major determinant of PN activity, we focused on the control of GABAergic transmission by 5-HT. IPSCs were elicited by local electrical stimulation of LA in the presence of glutamate receptor antagonists. We found that 5-HT reduces GABA_A inhibitory postsynaptic currents (IPSCs) via presynaptic 5-HT_1B receptors. While the presynaptic inhibition of GABA release also attenuated GABA_B currents, this effect was less pronounced than for GABA_A currents because 5-HT also induced a competing postsynaptic enhancement of GABA_B currents. That is, GABA_B currents elicited by pressure application of GABA or baclofen were enhanced by 5-HT. In addition, we obtained evidence suggesting that 5-HT differentially regulates distinct subsets of GABAergic synapses. Indeed, GABA_A IPSCs were comprised of two components: a relatively 5-HT-insensitive IPSC that had a fast time course and a 5-HT-sensitive component that had a slower time course. Because the relative contribution of these two components varied depending on whether neurons were recorded at proximity versus at a distance from the stimulating electrodes, we speculate that distinct subtypes of local-circuit cells contribute the two contingents of GABAergic synapses. Overall, our results indicate that 5-HT is a potent regulator of synaptic inhibition in LA.NEW & NOTEWORTHY We report that 5-HT, acting via presynaptic 5-HT_1B receptors, attenuates GABA_A IPSCs by reducing GABA release in the lateral amygdala (LA). In parallel, 5-HT enhances GABA_B currents postsynaptically, such that GABA_B inhibitory postsynaptic currents (IPSCs) are relatively preserved from the presynaptic inhibition of GABA release. We also found that the time course of 5-HT-sensitive and -insensitive GABA_A IPSCs differ. Together, these results indicate that 5-HT is a potent regulator of synaptic inhibition in LA.

Neuromodulators and Long-Term Synaptic Plasticity in Learning and Memory: A Steered-Glutamatergic Perspective

The molecular pathways underlying the induction and maintenance of long-term synaptic plasticity have been extensively investigated revealing various mechanisms by which neurons control their synaptic strength. The dynamic nature of neuronal connections combined with plasticity-mediated long-lasting structural and functional alterations provide valuable insights into neuronal encoding processes as molecular substrates of not only learning and memory but potentially other sensory, motor and behavioural functions that reflect previous experience. However, one key element receiving little attention in the study of synaptic plasticity is the role of neuromodulators, which are known to orchestrate neuronal activity on brain-wide, network and synaptic scales. We aim to review current evidence on the mechanisms by which certain modulators, namely dopamine, acetylcholine, noradrenaline and serotonin, control synaptic plasticity induction through corresponding metabotropic receptors in a pathway-specific manner. Lastly, we propose that neuromodulators control plasticity outcomes through steering glutamatergic transmission, thereby gating its induction and maintenance.

Single-cell transcriptomes and whole-brain projections of serotonin neurons in the mouse dorsal and median raphe nuclei

Serotonin neurons of the dorsal and median raphe nuclei (DR, MR) collectively innervate the entire forebrain and midbrain, modulating diverse physiology and behavior. To gain a fundamental understanding of their molecular heterogeneity, we used plate-based single-cell RNA-sequencing to generate a comprehensive dataset comprising eleven transcriptomically distinct serotonin neuron clusters. Systematic in situ hybridization mapped specific clusters to the principal DR, caudal DR, or MR. These transcriptomic clusters differentially express a rich repertoire of neuropeptides, receptors, ion channels, and transcription factors. We generated novel intersectional viral-genetic tools to access specific subpopulations. Whole-brain axonal projection mapping revealed that DR serotonin neurons co-expressing vesicular glutamate transporter-3 preferentially innervate the cortex, whereas those co-expressing thyrotropin-releasing hormone innervate subcortical regions in particular the hypothalamus. Reconstruction of 50 individual DR serotonin neurons revealed diverse and segregated axonal projection patterns at the single-cell level. Together, these results provide a molecular foundation of the heterogenous serotonin neuronal phenotypes.

Characterization of transgenic mouse models targeting neuromodulatory systems reveals organizational principles of the dorsal raphe

The dorsal raphe (DR) is a heterogeneous nucleus containing dopamine (DA), serotonin (5HT), γ-aminobutyric acid (GABA) and glutamate neurons. Consequently, investigations of DR circuitry require Cre-driver lines that restrict transgene expression to precisely defined cell populations. Here, we present a systematic evaluation of mouse lines targeting neuromodulatory cells in the DR. We find substantial differences in specificity between lines targeting DA neurons, and in penetrance between lines targeting 5HT neurons. Using these tools to map DR circuits, we show that populations of neurochemically distinct DR neurons are arranged in a stereotyped topographical pattern, send divergent projections to amygdala subnuclei, and differ in their presynaptic inputs. Importantly, targeting DR DA neurons using different mouse lines yielded both structural and functional differences in the neural circuits accessed. These results provide a refined model of DR organization and support a comparative, case-by-case evaluation of the suitability of transgenic tools for any experimental application.

Recent advances in understanding the role of phasic dopamine activity

The latest animal neurophysiology has revealed that the dopamine reward prediction error signal drives neuronal learning in addition to behavioral learning and reflects subjective reward representations beyond explicit contingency. The signal complies with formal economic concepts and functions in real-world consumer choice and social interaction. An early response component is influenced by physical impact, reward environment, and novelty but does not fully code prediction error. Some dopamine neurons are activated by aversive stimuli, which may reflect physical stimulus impact or true aversiveness, but they do not seem to code general negative value or aversive prediction error. The reward prediction error signal is complemented by distinct, heterogeneous, smaller and slower changes reflecting sensory and motor contributors to behavioral activation, such as substantial movement (as opposed to precise motor control), reward expectation, spatial choice, vigor, and motivation. The different dopamine signals seem to defy a simple unifying concept and should be distinguished to better understand phasic dopamine functions.

Prior Activation of 5-HT7 Receptors Modulates the Conditioned Place Preference With Methylphenidate

The serotonin receptor subtype 7 (5-HT7R) is clearly involved in behavioral functions such as learning/memory, mood regulation and circadian rhythm. Recent discoveries proposed modulatory physiological roles for serotonergic systems in reward-guided behavior. However, the interplay between serotonin (5-HT) and dopamine (DA) in reward-related behavioral adaptations needs to be further assessed. TP-22 is a recently developed arylpiperazine-based 5-HT7R agonist, which is also showing high affinity and selectivity towards D1 receptors. Here, we report that TP-22 displays D1 receptor antagonist activity. Moreover, we describe the first in vivo tests with TP-22: first, a pilot experiment (assessing dosage and timing of action) identified the 0.25 mg/kg i.v. dosage for locomotor stimulation of rats. Then, a conditioned place preference (CPP) test with the DA-releasing psychostimulant drug, methylphenidate (MPH), involved three rat groups: prior i.v. administration of TP-22 (0.25 mg/kg), or vehicle (VEH), 90 min before MPH (5 mg/kg), was intended for modulation of conditioning to the white chamber (saline associated to the black chamber); control group (SAL) was conditioned with saline in both chambers. Prior TP-22 further increased the stimulant effect of MPH on locomotor activity. During the place-conditioning test, drug-free activity of TP-22+MPH subjects remained steadily elevated, while VEH+MPH subjects showed a decline. Finally, after a priming injection of TP-22 in MPH-free conditions, rats showed a high preference for the MPH-associated white chamber, which conversely had vanished in VEH-primed MPH-conditioned subjects. Overall, the interaction between MPH and pre-treatment with TP-22 seems to improve both locomotor stimulation and the conditioning of motivational drives to environmental cues. Together with recent studies, a main modulatory role of 5-HT7R for the processing of rewards can be suggested. In the present study, TP-22 proved to be a useful psychoactive tool to better elucidate the role of 5-HT7R and its interplay with DA in reward-related behavior.

Control of synaptic plasticity in deep cortical networks

Humans and many other animals have an enormous capacity to learn about sensory stimuli and to master new skills. However, many of the mechanisms that enable us to learn remain to be understood. One of the greatest challenges of systems neuroscience is to explain how synaptic connections change to support maximally adaptive behaviour. Here, we provide an overview of factors that determine the change in the strength of synapses, with a focus on synaptic plasticity in sensory cortices. We review the influence of neuromodulators and feedback connections in synaptic plasticity and suggest a specific framework in which these factors can interact to improve the functioning of the entire network.

Molecular and anatomical organization of the dorsal raphe nucleus

The dorsal raphe nucleus (DRN) is an important source of neuromodulators and has been implicated in a wide variety of behavioral and neurological disorders. The DRN is subdivided into distinct anatomical subregions comprised of multiple cell types, and its complex cellular organization has impeded efforts to investigate the distinct circuit and behavioral functions of its subdomains. Here we used single-cell RNA sequencing, in situ hybridization, anatomical tracing, and spatial correlation analysis to map the transcriptional and spatial profiles of cells from the mouse DRN. Our analysis of 39,411 single-cell transcriptomes revealed at least 18 distinct neuron subtypes and 5 serotonergic neuron subtypes with distinct molecular and anatomical properties, including a serotonergic neuron subtype that preferentially innervates the basal ganglia. Our study lays out the molecular organization of distinct serotonergic and non-serotonergic subsystems, and will facilitate the design of strategies for further dissection of the DRN and its diverse functions.

Medial prefrontal cortex in neurological diseases

The medial prefrontal cortex (mPFC) is a crucial cortical region that integrates information from numerous cortical and subcortical areas and converges updated information to output structures. It plays essential roles in the cognitive process, regulation of emotion, motivation, and sociability. Dysfunction of the mPFC has been found in various neurological and psychiatric disorders, such as depression, anxiety disorders, schizophrenia, autism spectrum disorders, Alzheimer's disease, Parkinson's disease, and addiction. In the present review, we summarize the preclinical and clinical studies to illustrate the role of the mPFC in these neurological diseases.

Serotonin Subsystems Modulate Diverse and Opposite Behavioral Functions

Pioneering work showed that serotonin (5-HT) neurons have the unique capacity to engage in different and opposed aspects of motivated behaviors such as reward and punishment responses. These findings provided strong evidence about the functional heterogeneity of 5-HT neurons, and their possible engagement in multiple and behaviorally distinct neural subsystems. A recent study provides further compelling evidence supporting this notion, in which two ascending 5-HT circuits modulate opposed aspects of motivated behaviors.

Stable Representations of Decision Variables for Flexible Behavior

Decisions occur in dynamic environments. In the framework of reinforcement learning, the probability of performing an action is influenced by decision variables. Discrepancies between predicted and obtained rewards (reward prediction errors) update these variables, but they are otherwise stable between decisions. Although reward prediction errors have been mapped to midbrain dopamine neurons, it is unclear how the brain represents decision variables themselves. We trained mice on a dynamic foraging task in which they chose between alternatives that delivered reward with changing probabilities. Neurons in the medial prefrontal cortex, including projections to the dorsomedial striatum, maintained persistent firing rate changes over long timescales. These changes stably represented relative action values (to bias choices) and total action values (to bias response times) with slow decay. In contrast, decision variables were weakly represented in the anterolateral motor cortex, a region necessary for generating choices. Thus, we define a stable neural mechanism to drive flexible behavior.

The Serotonergic Raphe Promote Sleep in Zebrafish and Mice

The role of serotonin (5-HT) in sleep is controversial: early studies suggested a sleep-promoting role, but eventually the paradigm shifted toward a wake-promoting function for the serotonergic raphe. Here, we provide evidence from zebrafish and mice that the raphe are critical for the initiation and maintenance of sleep. In zebrafish, genetic ablation of 5-HT production by the raphe reduces sleep, sleep depth, and the homeostatic response to sleep deprivation. Pharmacological inhibition or ablation of the raphe reduces sleep, while optogenetic stimulation increases sleep. Similarly, in mice, ablation of the raphe increases wakefulness and impairs the homeostatic response to sleep deprivation, whereas tonic optogenetic stimulation at a rate similar to baseline activity induces sleep. Interestingly, burst optogenetic stimulation induces wakefulness in accordance with previously described burst activity of the raphe during arousing stimuli. These results indicate that the serotonergic system promotes sleep in both diurnal zebrafish and nocturnal rodents. VIDEO ABSTRACT.

A Discrete Dorsal Raphe to Basal Amygdala 5-HT Circuit Calibrates Aversive Memory

Despite a wealth of clinical and preclinical data implicating the serotonin (5-HT) system in fear-related affective disorders, a precise definition of this neuromodulator's role in fear remains elusive. Using convergent anatomical and functional approaches, we interrogate the contribution to fear of basal amygdala (BA) 5-HT inputs from the dorsal raphe nucleus (DRN). We show the DRN→BA 5-HT pathway is engaged during fear memory formation and retrieval, and activity of these projections facilitates fear and impairs extinction. The DRN→BA 5-HT pathway amplifies fear-associated BA neuronal firing and theta power and phase-locking. Although fear recruits 5-HT and VGluT3 co-expressing DRN neurons, the fear-potentiating influence of the DRN→BA 5-HT pathway requires signaling at BA 5-HT1A/2A receptors. Input-output mapping illustrates how the DRN→BA 5-HT pathway is anatomically distinct and connected with other brain regions that mediate fear. These findings reveal how a discrete 5-HT circuit orchestrates a broader neural network to calibrate aversive memory.

Modulation of anterior cingulate cortex reward and penalty signalling in medication-naive young-adult subjects with depressive symptoms following acute dose lurasidone

BACKGROUND

Aberrations in reward and penalty processing are implicated in depression and putatively reflect altered dopamine signalling. This study exploits the advantages of a placebo-controlled design to examine how a novel D2 antagonist with adjunctive antidepressant properties modifies activity in the brain's reward network in depression.

METHODS

We recruited 43 medication-naïve subjects across the range of depression severity (Beck's Depression Inventory-II score range: 0-43), including healthy volunteers, as well as people meeting full-criteria for major depressive disorder. In a double-blind placebo-controlled cross-over design, all subjects received either placebo or lurasidone (20 mg) across two visits separated by 1 week. Functional magnetic resonance imaging with the Monetary Incentive Delay (MID) task assessed reward functions via neural responses during anticipation and receipt of gains and losses. Arterial spin labelling measured cerebral blood flow (CBF) at rest.

RESULTS

Lurasidone altered fronto-striatal activity during anticipation and outcome phases of the MID task. A significant three-way Medication-by-Depression severity-by-Outcome interaction emerged in the anterior cingulate cortex (ACC) after correction for multiple comparisons. Follow-up analyses revealed significantly higher ACC activation to losses in high- v. low depression participants in the placebo condition, with a normalisation by lurasidone. This effect could not be accounted for by shifts in resting CBF.

CONCLUSIONS

Lurasidone acutely normalises reward processing signals in individuals with depressive symptoms. Lurasidone's antidepressant effects may arise from reducing responses to penalty outcomes in individuals with depressive symptoms.

Dissociable dopamine dynamics for learning and motivation

The dopamine projection from ventral tegmental area (VTA) to nucleus accumbens (NAc) is critical for motivation to work for rewards and reward-driven learning. How dopamine supports both functions is unclear. Dopamine cell spiking can encode prediction errors, which are vital learning signals in computational theories of adaptive behaviour. By contrast, dopamine release ramps up as animals approach rewards, mirroring reward expectation. This mismatch might reflect differences in behavioural tasks, slower changes in dopamine cell spiking or spike-independent modulation of dopamine release. Here we compare spiking of identified VTA dopamine cells with NAc dopamine release in the same decision-making task. Cues that indicate an upcoming reward increased both spiking and release. However, NAc core dopamine release also covaried with dynamically evolving reward expectations, without corresponding changes in VTA dopamine cell spiking. Our results suggest a fundamental difference in how dopamine release is regulated to achieve distinct functions: broadcast burst signals promote learning, whereas local control drives motivation.

RETRACTED: Chronic fluoxetine treatment in vivo enhances excitatory synaptic transmission in the hippocampus

This article has been retracted: please see Elsevier Policy on Article Withdrawal (https://www.elsevier.com/about/our-business/policies/article-withdrawal). This article has been retracted at the request of the Authors. After publication, Scott M. Thompson found significant concerns about the data and duly notified The University of Maryland. The University of Maryland conducted an internal investigation which confirmed that the article was compromised. Namely in Figure 2B, the Investigation Committee determined that the western blots used to create the figure were not the ones used for the quantification and concluded that the figure was falsified to fit the hypothesis. In Figure 2C and 2D, the Investigation Committee determined that the densitometry data (pCaMKII, pS831, CamKII and GluA1) used to create the histogram were falsified to fit the hypothesis.

Striatal Low-Threshold Spiking Interneurons Regulate Goal-Directed Learning

The dorsomedial striatum (DMS) is critically involved in motor control and reward processing, but the specific neural circuit mediators are poorly understood. Recent evidence highlights the extensive connectivity of low-threshold spiking interneurons (LTSIs) within local striatal circuitry; however, the in vivo function of LTSIs remains largely unexplored. We employed fiber photometry to assess LTSI calcium activity in a range of DMS-mediated behaviors, uncovering specific reward-related activity that is down-modulated during goal-directed learning. Using two mechanistically distinct manipulations, we demonstrated that this down-modulation of LTSI activity is critical for acquisition of novel contingencies, but not for their modification. In contrast, continued LTSI activation slowed instrumental learning. Similar manipulations of fast-spiking interneurons did not reproduce these effects, implying a specific function of LTSIs. Finally, we revealed a role for the γ-aminobutyric acid (GABA)ergic functions of LTSIs in learning. Together, our data provide new insights into this striatal interneuron subclass as important gatekeepers of goal-directed learning.

Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales

Behavior deviating from our normative expectations often appears irrational. For example, even though behavior following the so-called matching law can maximize reward in a stationary foraging task, actual behavior commonly deviates from matching. Such behavioral deviations are interpreted as a failure of the subject; however, here we instead suggest that they reflect an adaptive strategy, suitable for uncertain, non-stationary environments. To prove it, we analyzed the behavior of primates that perform a dynamic foraging task. In such nonstationary environment, learning on both fast and slow timescales is beneficial: fast learning allows the animal to react to sudden changes, at the price of large fluctuations (variance) in the estimates of task relevant variables. Slow learning reduces the fluctuations but costs a bias that causes systematic behavioral deviations. Our behavioral analysis shows that the animals solved this bias-variance tradeoff by combining learning on both fast and slow timescales, suggesting that learning on multiple timescales can be a biologically plausible mechanism for optimizing decisions under uncertainty.

Recent experience can only provide limited information to guide decisions in a volatile environment. Here, the authors report that the choices made by a monkey in a dynamic foraging task can be better explained by a model that combines learning on both fast and slow timescales.

Dorsal raphe serotonin neurons inhibit operant responding for reward via inputs to the ventral tegmental area but not the nucleus accumbens: evidence from studies combining optogenetic stimulation and serotonin reuptake inhibition

The monoamine neurotransmitter serotonin (5-hydroxytryptamine; 5-HT) exerts an inhibitory influence over motivation, but the circuits mediating this are unknown. Here, we used an optogenetic approach to isolate the contribution of dorsal raphe nucleus (DRN) 5-HT neurons and 5-HT innervation of the mesolimbic dopamine (DA) system to motivated behavior in mice. We found that optogenetic stimulation of DRN 5-HT neurons enhanced downstream 5-HT release, but this was not sufficient to inhibit operant responding for saccharin, a measure of motivated behavior. However, combining optogenetic stimulation of DRN 5-HT neurons with a low dose of the selective serotonin reuptake inhibitor (SSRI) citalopram synergistically reduced operant responding. We then examined whether these effects could be recapitulated if optogenetic stimulation specifically targeted 5-HT terminals in the ventral tegmental area (VTA) or nucleus accumbens (NAc) of the mesolimbic DA system. Optogenetic stimulation of 5-HT input to the VTA combined with citalopram treatment produced a synergistic decrease in responding for saccharin, resembling the changes produced by targeting 5-HT neurons in the DRN. However, this effect was not observed when optogenetic stimulation targeted 5-HT terminals in the NAc. Taken together, these results suggest that DRN 5-HT neurons exert an inhibitory influence over operant responding for reward through a direct interaction with the mesolimbic DA system at the level of the VTA. These studies support an oppositional interaction between 5-HT and DA systems in controlling motivation and goal-directed behavior, and have important implications for the development and refinement of treatment strategies for psychiatric disorders such as depression and addiction.

Multiple Dopamine Systems: Weal and Woe of Dopamine

The ability to predict future outcomes increases the fitness of the animal. Decades of research have shown that dopamine neurons broadcast reward prediction error (RPE) signals-the discrepancy between actual and predicted reward-to drive learning to predict future outcomes. Recent studies have begun to show, however, that dopamine neurons are more diverse than previously thought. In this review, we will summarize a series of our studies that have shown unique properties of dopamine neurons projecting to the posterior "tail" of the striatum (TS) in terms of anatomy, activity, and function. Specifically, TS-projecting dopamine neurons are activated by a subset of negative events including threats from a novel object, send prediction errors for external threats, and reinforce avoidance behaviors. These results indicate that there are at least two axes of dopamine-mediated reinforcement learning in the brain-one learning from canonical RPEs and another learning from threat prediction errors. We argue that the existence of multiple learning systems is an adaptive strategy that makes possible each system optimized for its own needs. The compartmental organization in the mammalian striatum resembles that of a dopamine-recipient area in insects (mushroom body), pointing to a principle of dopamine function conserved across phyla.

Disrupted reinforcement learning during post-error slowing in ADHD

ADHD is associated with altered dopamine regulated reinforcement learning on prediction errors. Despite evidence of categorically altered error processing in ADHD, neuroimaging advances have largely investigated models of normal reinforcement learning in greater detail. Further, although reinforcement leaning critically relies on ventral striatum exerting error magnitude related thresholding influences on substantia nigra (SN) and dorsal striatum, these thresholding influences have never been identified with neuroimaging. To identify such thresholding influences, we propose that error magnitude related activities must first be separated from opposite activities in overlapping neural regions during error detection. Here we separate error detection from magnitude related adjustment (post-error slowing) during inhibition errors in the stop signal task in typically developing (TD) and ADHD adolescents using fMRI. In TD, we predicted that: 1) deactivation of dorsal striatum on error detection interrupts ongoing processing, and should be proportional to right frontoparietal response phase activity that has been observed in the SST; 2) deactivation of ventral striatum on post-error slowing exerts thresholding influences on, and should be proportional to activity in dorsal striatum. In ADHD, we predicted that ventral striatum would instead correlate with heightened amygdala responses to errors. We found deactivation of dorsal striatum on error detection correlated with response-phase activity in both groups. In TD, post-error slowing deactivation of ventral striatum correlated with activation of dorsal striatum. In ADHD, ventral striatum correlated with heightened amygdala activity. Further, heightened activities in locus coeruleus (norepinephrine), raphe nucleus (serotonin) and medial septal nuclei (acetylcholine), which all compete for control of DA, and are altered in ADHD, exhibited altered correlations with SN. All correlations in TD were replicated in healthy adults. Results in TD are consistent with dopamine regulated reinforcement learning on post-error slowing. In ADHD, results are consistent with heightened activities in the amygdala and non-dopaminergic neurotransmitter nuclei preventing reinforcement learning.

Two eARCHT3.0 Lines for Optogenetic Silencing of Dopaminergic and Serotonergic Neurons

Dopaminergic and serotonergic neurons modulate and control processes ranging from reward signaling to regulation of motor outputs. Further, dysfunction of these neurons is involved in both degenerative and psychiatric disorders. Elucidating the roles of these neurons has been greatly facilitated by bacterial artificial chromosome (BAC) transgenic mouse lines expressing channelrhodopsin to readily enable cell-type specific activation. However, corresponding lines to silence these monoaminergic neurons have been lacking. We have generated two BAC transgenic mouse lines expressing the outward proton pump, enhanced ArchT3.0 (eArchT3.0), and GFP under control of the regulatory elements of either the dopamine transporter (DAT; Jax# 031663) or the tryptophan hydroxylase 2 (TPH2; Jax# 031662) gene locus. We demonstrate highly faithful and specific expression of these lines in dopaminergic and serotonergic neurons respectively. Additionally we validate effective and sensitive eArchT3.0-mediated silencing of these neurons using slice electrophysiology as well as with a well-established behavioral assay. These new transgenic tools will help expedite the study of dopaminergic and serotonergic system function in normal behavior and disease.

Dorsal Raphe Dual Serotonin-Glutamate Neurons Drive Reward by Establishing Excitatory Synapses on VTA Mesoaccumbens Dopamine Neurons

Dorsal raphe (DR) serotonin neurons provide a major input to the ventral tegmental area (VTA). Here, we show that DR serotonin transporter (SERT) neurons establish both asymmetric and symmetric synapses on VTA dopamine neurons, but most of these synapses are asymmetric. Moreover, the DR-SERT terminals making asymmetric synapses on VTA dopamine neurons coexpress vesicular glutamate transporter 3 (VGluT3; transporter for accumulation of glutamate for its synaptic release), suggesting the excitatory nature of these synapses. VTA photoactivation of DR-SERT fibers promotes conditioned place preference, elicits excitatory currents on mesoaccumbens dopamine neurons, increases their firing, and evokes dopamine release in nucleus accumbens. These effects are blocked by VTA inactivation of glutamate and serotonin receptors, supporting the idea of glutamate release in VTA from dual DR SERT-VGluT3 inputs. Our findings suggest a path-specific input from DR serotonergic neurons to VTA that promotes reward by the release of glutamate and activation of mesoaccumbens dopamine neurons.

Nucleus Accumbens Modulation in Reward and Aversion

The nucleus accumbens (NAc) is a key node of the brain’s circuitry that is responsible for translating motivation into action. It has been implicated in playing critical roles in virtually all forms of adaptive and pathological motivated behaviors. It is subject to modulation by a broad array of inputs that influence NAc activity in complex ways that are still poorly understood. Here, we briefly review current knowledge about the behavioral consequences of NAc modulation, focusing on recent studies that use novel techniques developed and implemented over the last decade.

Lesions of lateral habenula attenuate win-stay but not lose-shift responses in a competitive choice task

Multiple neural systems contribute to choice adaptation following reinforcement. Recent evidence suggests that the lateral habenula (LHb) plays a key role in such adaptations, particularly when reinforcements are worse than expected. Here, we investigated the effects of bilateral LHb lesions on responding in a binary choice task with no discriminatory cues. LHb lesions in rats decreased win-stay responses but surprisingly left lose-shift responses intact. This same dissociated effect was also observed after systemic administration of d-amphetamine in a separate cohort of animals. These results suggest that at least some behavioural responses triggered by reward omission do not depend on an intact LHb.

Rostral and Caudal Ventral Tegmental Area GABAergic Inputs to Different Dorsal Raphe Neurons Participate in Opioid Dependence

Both the ventral tegmental area (VTA) and dorsal raphe nucleus (DRN) are involved in affective control and reward-related behaviors. Moreover, the neuronal activities of the VTA and DRN are modulated by opioids. However, the precise circuits from the VTA to DRN and how opioids modulate these circuits remain unknown. Here, we found that neurons projecting from the VTA to DRN are primarily GABAergic. Rostral VTA (rVTA) GABAergic neurons preferentially innervate DRN GABAergic neurons, thus disinhibiting DRN serotonergic neurons. Optogenetic activation of this circuit induces aversion. In contrast, caudal VTA (cVTA) GABAergic neurons mainly target DRN serotonergic neurons, and activation of this circuit promotes reward. Importantly, μ-opioid receptors (MOPs) are selectively expressed at rVTA→DRN GABAergic synapses, and morphine depresses the synaptic transmission. Chronically elevating the activity of the rVTA→DRN pathway specifically interrupts morphine-induced conditioned place preference. This opioid-modulated inhibitory circuit may yield insights into morphine reward and dependence pathogenesis.

Chronic methylphenidate treatment during adolescence has long-term effects on monoaminergic function

BACKGROUND

Psychostimulants like methylphenidate or D-amphetamine are often prescribed for attention deficit and hyperactivity disorders in children. Whether such drugs can be administered into a developing brain without consequences in adulthood is still an open question.

METHODS

Here, using in vivo extracellular electrophysiology in anesthetised preparations, combined with behavioural assays, we have examined the long-term consequences in adulthood of a chronic methylphenidate oral administration (5 mg/kg/day, 15 days) in early adolescent (post-natal day 28) and late adolescent (post-natal day 42) rats, by evaluating body weight change, sucrose preference (indicator of anhedonia), locomotor sensitivity to D-amphetamine and electrical activities of ventral tegmental area dopamine and dorsal raphe nucleus serotonin neurons.

RESULTS

Chronic methylphenidate treatment during early or late adolescence did not induce weight deficiencies and anhedonia-like behaviours at adulthood. However, it increased bursting activities of dorsal raphe nucleus serotonin neurons. Furthermore, chronic methylphenidate treatment during early but not during late adolescence enhanced D-amphetamine-induced rearing activity, as well as ventral tegmental area dopamine cell excitability (firing, burst and population activity), associated with a partial desensitisation of dopamine D₂ auto-receptors.

CONCLUSIONS

We have demonstrated here that early, but not late, adolescent exposure to oral methylphenidate may induce long-lasting effects on monoamine neurotransmission. The possible clinical implication of these data will be discussed.

ASICs Mediate Food Responses in an Enteric Serotonergic Neuron that Controls Foraging Behaviors

Animals must respond to the ingestion of food by generating adaptive behaviors, but the role of gut-brain signaling in behavioral regulation is poorly understood. Here, we identify conserved ion channels in an enteric serotonergic neuron that mediate its responses to food ingestion and decipher how these responses drive changes in foraging behavior. We show that the C. elegans serotonergic neuron NSM acts as an enteric sensory neuron that acutely detects food ingestion. We identify the novel and conserved acid-sensing ion channels (ASICs) DEL-7 and DEL-3 as NSM-enriched channels required for feeding-dependent NSM activity, which in turn drives slow locomotion while animals feed. Point mutations that alter the DEL-7 channel change NSM dynamics and associated behavioral dynamics of the organism. This study provides causal links between food ingestion, molecular and physiological properties of an enteric serotonergic neuron, and adaptive feeding behaviors, yielding a new view of how enteric neurons control behavior.

Control of movement vigor and decision making during foraging

During foraging, animals decide how long to stay at a patch and harvest reward, and then, they move with certain vigor to another location. How does the brain decide when to leave, and how does it determine the speed of the ensuing movement? Here, we considered the possibility that both the decision-making and the motor control problems aimed to maximize a single normative utility: the sum of all rewards acquired minus all efforts expended divided by total time. This optimization could be achieved if the brain compared a local measure of utility with its history. To test the theory, we examined behavior of people as they gazed at images: they chose how long to look at the image (harvesting information) and then moved their eyes to another image, controlling saccade speed. We varied reward via image content and effort via image eccentricity, and then, we measured how these changes affected decision making (gaze duration) and motor control (saccade speed). After a history of low rewards, people increased gaze duration and decreased saccade speed. In anticipation of future effort, they lowered saccade speed and increased gaze duration. After a history of high effort, they elevated their saccade speed and increased gaze duration. Therefore, the theory presented a principled way with which the brain may control two aspects of behavior: movement speed and harvest duration. Our experiments confirmed many (but not all) of the predictions, suggesting that harvest duration and movement speed, fundamental aspects of behavior during foraging, may be governed by a shared principle of control.

Systematic Analysis of Transmitter Coexpression Reveals Organizing Principles of Local Interneuron Heterogeneity

Broad neuronal classes are surprisingly heterogeneous across many parameters, and subclasses often exhibit partially overlapping traits including transmitter coexpression. However, the extent to which transmitter coexpression occurs in predictable, consistent patterns is unknown. Here, we demonstrate that pairwise coexpression of GABA and multiple neuropeptide families by olfactory local interneurons (LNs) of the moth Manduca sexta is highly heterogeneous, with a single LN capable of expressing neuropeptides from at least four peptide families and few instances in which neuropeptides are consistently coexpressed. Using computational modeling, we demonstrate that observed coexpression patterns cannot be explained by independent probabilities of expression of each neuropeptide. Our analyses point to three organizing principles that, once taken into consideration, allow replication of overall coexpression structure: (1) peptidergic neurons are highly likely to coexpress GABA; (2) expression probability of allatotropin depends on myoinhibitory peptide expression; and (3) the all-or-none coexpression patterns of tachykinin neurons with several other neuropeptides. For other peptide pairs, the presence of one peptide was not predictive of the presence of the other, and coexpression probability could be replicated by independent probabilities. The stochastic nature of these coexpression patterns highlights the heterogeneity of transmitter content among LNs and argues against clear-cut definition of subpopulation types based on the presence of single neuropeptides. Furthermore, the receptors for all neuropeptides and GABA were expressed within each population of principal neuron type in the antennal lobe (AL). Thus, activation of any given LN results in a dynamic cocktail of modulators that have the potential to influence every level of olfactory processing within the AL.

Pavlovian-instrumental interactions in active avoidance: The bark of neutral trials

In active avoidance tasks, subjects have to learn to execute particular actions in order to avoid an aversive stimulus, such as a shock. Such paradigms pose a number of psychological and neural enigmas, and so have attracted substantial computational interest. However, the ratio of conjecture to confirmation remains high. Here, we perform a theoretical inquiry into a recent experiment by Gentry, Lee, and Roesch ('Phasic dopamine release in the rat nucleus accumbens predicts approach and avoidance performance', Nat. Commun., 7:13154) who measured phasic dopamine concentrations in the nucleus accumbens core of rats whilst they avoided shocks, acquired food, or acted to gain no programmed outcome. These last, neutral, trials turned out to be a perfect probe for the workings of avoidance, partly because of the substantial differences between subjects and sessions revealed in the experiment. We suggest a way to interpret this probe, gaining support for opponency-, safety-, and Pavlovian-influenced treatments of avoidance.

Motor thalamus supports striatum-driven reinforcement

Reinforcement has long been thought to require striatal synaptic plasticity. Indeed, direct striatal manipulations such as self-stimulation of direct-pathway projection neurons (dMSNs) are sufficient to induce reinforcement within minutes. However, it’s unclear what role, if any, is played by downstream circuitry. Here, we used dMSN self-stimulation in mice as a model for striatum-driven reinforcement and mapped the underlying circuitry across multiple basal ganglia nuclei and output targets. We found that mimicking the effects of dMSN activation on downstream circuitry, through optogenetic suppression of basal ganglia output nucleus substantia nigra reticulata (SNr) or activation of SNr targets in the brainstem or thalamus, was also sufficient to drive rapid reinforcement. Remarkably, silencing motor thalamus—but not other selected targets of SNr—was the only manipulation that reduced dMSN-driven reinforcement. Together, these results point to an unexpected role for basal ganglia output to motor thalamus in striatum-driven reinforcement.

Anatomically Defined and Functionally Distinct Dorsal Raphe Serotonin Sub-systems

The dorsal raphe (DR) constitutes a major serotonergic input to the forebrain and modulates diverse functions and brain states, including mood, anxiety, and sensory and motor functions. Most functional studies to date have treated DR serotonin neurons as a single population. Using viral-genetic methods, we found that subcortical- and cortical-projecting serotonin neurons have distinct cell-body distributions within the DR and differentially co-express a vesicular glutamate transporter. Further, amygdala- and frontal-cortex-projecting DR serotonin neurons have largely complementary whole-brain collateralization patterns, receive biased inputs from presynaptic partners, and exhibit opposite responses to aversive stimuli. Gain- and loss-of-function experiments suggest that amygdala-projecting DR serotonin neurons promote anxiety-like behavior, whereas frontal-cortex-projecting neurons promote active coping in the face of challenge. These results provide compelling evidence that the DR serotonin system contains parallel sub-systems that differ in input and output connectivity, physiological response properties, and behavioral functions.

Serotonin regulates dynamics of cerebellar granule cell activity by modulating tonic inhibition

Understanding how afferent information is integrated by cortical structures requires identifying the factors shaping excitation and inhibition within their input layers. The input layer of the cerebellar cortex integrates diverse sensorimotor information to enable learned associations that refine the dynamics of movement. Specifically, mossy fiber afferents relay sensorimotor input into the cerebellum to excite granule cells, whose activity is regulated by inhibitory Golgi cells. To test how this integration can be modulated, we have used an acute brain slice preparation from young adult rats and found that encoding of mossy fiber input in the cerebellar granule cell layer can be regulated by serotonin (5-hydroxytryptamine, 5-HT) via a specific action on Golgi cells. We find that 5-HT depolarizes Golgi cells, likely by activating 5-HT2A receptors, but does not directly act on either granule cells or mossy fibers. As a result of Golgi cell depolarization, 5-HT significantly increases tonic inhibition onto both granule cells and Golgi cells. 5-HT-mediated Golgi cell depolarization is not sufficient, however, to alter the probability or timing of mossy fiber-evoked feed-forward inhibition onto granule cells. Together, increased granule cell tonic inhibition paired with normal feed-forward inhibition acts to reduce granule cell spike probability without altering spike timing. Hence, these data provide a circuit mechanism by which 5-HT can reduce granule cell activity without altering temporal representations of mossy fiber input. Such changes in network integration could enable flexible, state-specific suppression of cerebellar sensorimotor input that should not be learned or enable reversal learning for unwanted associations. NEW & NOTEWORTHY Serotonin (5-hydroxytryptamine, 5-HT) regulates synaptic integration at the input stage of cerebellar processing by increasing tonic inhibition of granule cells. This circuit mechanism reduces the probability of granule cell spiking without altering spike timing, thus suppressing cerebellar input without altering its temporal representation in the granule cell layer.

Monoaminergic Neuromodulation of Sensory Processing

All neuronal circuits are subject to neuromodulation. Modulatory effects on neuronal processing and resulting behavioral changes are most commonly reported for higher order cognitive brain functions. Comparatively little is known about how neuromodulators shape processing in sensory brain areas that provide the signals for downstream regions to operate on. In this article, we review the current knowledge about how the monoamine neuromodulators serotonin, dopamine and noradrenaline influence the representation of sensory stimuli in the mammalian sensory system. We review the functional organization of the monoaminergic brainstem neuromodulatory systems in relation to their role for sensory processing and summarize recent neurophysiological evidence showing that monoamines have diverse effects on early sensory processing, including changes in gain and in the precision of neuronal responses to sensory inputs. We also highlight the substantial evidence for complementarity between these neuromodulatory systems with different patterns of innervation across brain areas and cortical layers as well as distinct neuromodulatory actions. Studying the effects of neuromodulators at various target sites is a crucial step in the development of a mechanistic understanding of neuronal information processing in the healthy brain and in the generation and maintenance of mental diseases.

An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals

Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed two distinct choice strategies. Choices after short inter-trial-intervals (ITIs) depended only on the last trial outcome and followed a win-stay-lose-switch pattern. In contrast, choices after long ITIs reflected outcome history over multiple trials, as described by reinforcement learning models. We found that optogenetic stimulation during a trial significantly boosted the rate of learning that occurred due to the outcome of that trial, but these effects were only exhibited on choices after long ITIs. This suggests that serotonin neurons modulate reinforcement learning rates, and that this influence is masked by alternate, unaffected, decision mechanisms. These results provide insight into the role of serotonin in treating psychiatric disorders, particularly its modulation of neural plasticity and learning.

Serotonin (5-HT) plays many important roles in reward, punishment, patience and beyond, and optogenetic stimulation of 5-HT neurons has not crisply parsed them. The authors report a novel analysis of a reward-based decision-making experiment, and show that 5-HT stimulation increases the learning rate, but only on a select subset of choices.

Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience

Recent experiments have shown that optogenetic activation of serotonin neurons in the dorsal raphe nucleus (DRN) in mice enhances patience in waiting for future rewards. Here, we show that serotonin effect in promoting waiting is maximized by both high probability and high timing uncertainty of reward. Optogenetic activation of serotonergic neurons prolongs waiting time in no-reward trials in a task with 75% food reward probability, but not with 50 or 25% reward probabilities. Serotonin effect in promoting waiting increases when the timing of reward presentation becomes unpredictable. To coherently explain the experimental data, we propose a Bayesian decision model of waiting that assumes that serotonin neuron activation increases the prior probability or subjective confidence of reward delivery. The present data and modeling point to the possibility of a generalized role of serotonin in resolving trade-offs, not only between immediate and delayed rewards, but also between sensory evidence and subjective confidence.

What does dopamine mean?

Dopamine is a critical modulator of both learning and motivation. This presents a problem: how can target cells know whether increased dopamine is a signal to learn or to move? It is often presumed that motivation involves slow ('tonic') dopamine changes, while fast ('phasic') dopamine fluctuations convey reward prediction errors for learning. Yet recent studies have shown that dopamine conveys motivational value and promotes movement even on subsecond timescales. Here I describe an alternative account of how dopamine regulates ongoing behavior. Dopamine release related to motivation is rapidly and locally sculpted by receptors on dopamine terminals, independently from dopamine cell firing. Target neurons abruptly switch between learning and performance modes, with striatal cholinergic interneurons providing one candidate switch mechanism. The behavioral impact of dopamine varies by subregion, but in each case dopamine provides a dynamic estimate of whether it is worth expending a limited internal resource, such as energy, attention, or time.