Distinct roles of striatal direct and indirect pathways in value-based decision making

Control of goal-directed and inflexible actions by dorsal striatal melanocortin systems, in coordination with the central nucleus of the amygdala

The dorsomedial striatum (DMS) is associated with flexible goal seeking, as opposed to routinized habits. Whether local mechanisms brake this function, for instance when habits may be adaptive, is incompletely understood. We find that a sub-population of dopamine D1 receptor-containing striatal neurons express the melanocortin-4 receptor (MC4R) for α-melanocyte stimulating hormone. These neurons within the DMS are necessary and sufficient for controlling the capacity of mice to flexibly adjust actions based on the likelihood that they will be rewarded. In investigating MC4R function, we found that it suppresses immediate-early gene levels in the DMS and concurrently, flexible goal seeking. MC4R+ neurons receive input from the central nucleus of the amygdala, and behavioral experiments indicate that they are functionally integrated into an amygdalo-striatal circuit that suppresses action flexibility in favor of routine. Publicly available spatial transcriptomics datasets were analyzed for gene transcript correlates of Mc4r expression across the striatal subregions, revealing considerable co-variation in dorsal structures. This insight led to the discovery that the function of MC4R in the dorsolateral striatum complements that in the DMS, in this case suppressing habit-like behavior. Altogether, our findings suggest that striatal MC4R controls the capacity for goal-directed and inflexible actions alike.

Chronic Ethanol Exposure Produces Persistent Impairment in Cognitive Flexibility and Decision Signals in the Striatum

Lack of cognitive flexibility is a hallmark of substance use disorders and has been associated with drug-induced synaptic plasticity in the dorsomedial striatum (DMS). Yet the possible impact of altered plasticity on real-time striatal neural dynamics during decision-making is unclear. Here, we identified persistent impairments induced by chronic ethanol (EtOH) exposure on cognitive flexibility and striatal decision signals. After a substantial withdrawal period from prior EtOH vapor exposure, male, but not female, rats exhibited reduced adaptability and exploratory behavior during a dynamic decision-making task. Reinforcement learning models showed that prior EtOH exposure enhanced learning from rewards over omissions. Notably, neural signals in the DMS related to the decision outcome were enhanced, while those related to choice and choice-outcome conjunction were reduced, in EtOH-treated rats compared to the controls. These findings highlight the profound impact of chronic EtOH exposure on adaptive decision-making, pinpointing specific changes in striatal representations of actions and outcomes as underlying mechanisms for cognitive deficits.

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.

Front and center: Maturational dysregulation of frontal lobe functional neuroanatomic connections in attention deficit hyperactivity disorder

Frontal lobe function may not universally explain all forms of attention deficit hyperactivity disorder (ADHD) but the frontal lobe hypothesis described supports an internally consistent model for integrating the numerous behaviors associated with ADHD. The paper examines the developmental trajectories of frontal and prefrontal lobe development, framing ADHD as maturational dysregulation concluding that the cognitive, motor, and behavioral abilities of the presumptive majority of ADHD children may not primarily be disordered or dysfunctional but reflect maturational dysregulation that is inconsistent with the psychomotor and cognitive expectations for the child’s chronological and mental age. ADHD children demonstrate decreased activation of the right and middle prefrontal cortex. Prefrontal and frontal lobe regions have an exuberant network of shared pathways with the diencephalic region, also having a regulatory function in arousal as well as with the ascending reticular formation which has a capacity for response suppression to task-irrelevant stimuli. Prefrontal lesions oftentimes are associated with the regulatory breakdown of goal-directed activity and impulsivity. In conclusion, a presumptive majority of childhood ADHD may result from maturational dysregulation of the frontal lobes with effects on the direct, indirect and/or, hyperdirect pathways.

Activation, but not inhibition, of the indirect pathway disrupts choice rejection in a freely moving, multiple-choice foraging task

The dorsomedial striatum (DMS) plays a key role in action selection, but less is known about how direct and indirect pathway spiny projection neurons (dSPNs and iSPNs, respectively) contribute to choice rejection in freely moving animals. Here, we use pathway-specific chemogenetic manipulation during a serial choice foraging task to test the role of dSPNs and iSPNs in learned choice rejection. We find that chemogenetic activation, but not inhibition, of iSPNs disrupts rejection of nonrewarded choices, contrary to predictions of a simple "select/suppress" heuristic. Our findings suggest that iSPNs' role in stopping and freezing does not extend in a simple fashion to choice rejection in an ethological, freely moving context. These data may provide insights critical for the successful design of interventions for addiction or other conditions in which it is desirable to strengthen choice rejection.

Androglobin, a chimeric mammalian globin, is required for male fertility

Spermatogenesis is a highly specialized differentiation process driven by a dynamic gene expression program and ending with the production of mature spermatozoa. Whereas hundreds of genes are known to be essential for male germline proliferation and differentiation, the contribution of several genes remains uncharacterized. The predominant expression of the latest globin family member, androglobin (Adgb), in mammalian testis tissue prompted us to assess its physiological function in spermatogenesis. Adgb knockout mice display male infertility, reduced testis weight, impaired maturation of elongating spermatids, abnormal sperm shape, and ultrastructural defects in microtubule and mitochondrial organization. Epididymal sperm from Adgb knockout animals display multiple flagellar malformations including coiled, bifid or shortened flagella, and erratic acrosomal development. Following immunoprecipitation and mass spectrometry, we could identify septin 10 (Sept10) as interactor of Adgb. The Sept10-Adgb interaction was confirmed both in vivo using testis lysates and in vitro by reciprocal co-immunoprecipitation experiments. Furthermore, the absence of Adgb leads to mislocalization of Sept10 in sperm, indicating defective manchette and sperm annulus formation. Finally, in vitro data suggest that Adgb contributes to Sept10 proteolysis in a calmodulin-dependent manner. Collectively, our results provide evidence that Adgb is essential for murine spermatogenesis and further suggest that Adgb is required for sperm head shaping via the manchette and proper flagellum formation.

Towards an expanded neuroscientific understanding of social play

Play has been recognized as a complex and diverse set of behaviors that has been difficult to define. Play can range from rough and tumble play among rats to a human child playing a computer game. Play has been understood to exist in multiple forms such as social, object, and locomotor (Burghardt, 2005). In this article we review the literatures on the neural basis of social play, on heart rate variability, on behavioral switching and set-shifting, on prepulse inhibition of the acoustic startle reflex, and on learning at the level of the basal ganglia. Each of these neuronal pathways, aside from heart rate variability, is rooted in the parafascicular nucleus of the thalamus, an important neural substrate for social play. We argue that social play optimally balances a number of opposing neural pathways by engaging systems involved in safety versus danger (heart rate variability), automatized reactions versus learned reactions to new stimuli (behavioral switching and set-shifting), and gating relevant versus less relevant stimuli (prepulse inhibition of the acoustic startle reflex). The idea that play, in addition to its role in interpersonal adaptation to social life, may have a central role in optimizing flexibility and creativity in individual response to novelty has been explored by previous authors (Huizinga, 1955; Spinka et al., 2001; Pellegrini et al., 2007; Pellis and Pellis, 2017). In this paper we explore the possible underlying neural basis for this function of play, having to do with balancing various neural networks, and in doing so propose an expanded understanding of the nature and function of social play.

NOX1/NADPH Oxidase Promotes Synaptic Facilitation Induced by Repeated D2 Receptor Stimulation: Involvement in Behavioral Repetition

Repetitive behavior is a widely observed neuropsychiatric symptom. Abnormal dopaminergic signaling in the striatum is one of the factors associated with behavioral repetition; however, the molecular mechanisms underlying the induction of repetitive behavior remain unclear. Here, we demonstrated that the NOX1 isoform of the superoxide-producing enzyme NADPH oxidase regulated repetitive behavior in mice by facilitating excitatory synaptic inputs in the central striatum (CS). In male C57Bl/6J mice, repeated stimulation of D2 receptors induced abnormal behavioral repetition and perseverative behavior. Nox1 deficiency or acute pharmacological inhibition of NOX1 significantly shortened repeated D2 receptor stimulation-induced repetitive behavior without affecting motor responses to a single D2 receptor stimulation. Among brain regions, Nox1 showed enriched expression in the striatum, and repeated dopamine D2 receptor stimulation further increased Nox1 expression levels in the CS, but not in the dorsal striatum. Electrophysiological analyses revealed that repeated D2 receptor stimulation facilitated excitatory inputs in the CS indirect pathway medium spiny neurons (iMSNs), and this effect was suppressed by the genetic deletion or pharmacological inhibition of NOX1. Nox1 deficiency potentiated protein tyrosine phosphatase activity and attenuated the accumulation of activated Src kinase, which is required for the synaptic potentiation in CS iMSNs. Inhibition of NOX1 or β-arrestin in the CS was sufficient to ameliorate repetitive behavior. Striatal-specific Nox1 knockdown also ameliorated repetitive and perseverative behavior. Collectively, these results indicate that NOX1 acts as an enhancer of synaptic facilitation in CS iMSNs and plays a key role in the molecular link between abnormal dopamine signaling and behavioral repetition and perseveration.SIGNIFICANCE STATEMENT Behavioral repetition is a form of compulsivity, which is one of the core symptoms of psychiatric disorders, such as obsessive-compulsive disorder. Perseveration is also a hallmark of such disorders. Both clinical and animal studies suggest important roles of abnormal dopaminergic signaling and striatal hyperactivity in compulsivity; however, the precise molecular link between them remains unclear. Here, we demonstrated the contribution of NOX1 to behavioral repetition induced by repeated stimulation of D2 receptors. Repeated stimulation of D2 receptors upregulated Nox1 mRNA in a striatal subregion-specific manner. The upregulated NOX1 promoted striatal synaptic facilitation in iMSNs by enhancing phosphorylation signaling. These results provide a novel mechanism for D2 receptor-mediated excitatory synaptic facilitation and indicate the therapeutic potential of NOX1 inhibition in compulsivity.

Disruption of Nrxn1α within excitatory forebrain circuits drives value-based dysfunction

Goal-directed behaviors are essential for normal function and significantly impaired in neuropsychiatric disorders. Despite extensive associations between genetic mutations and these disorders, the molecular contributions to goal-directed dysfunction remain unclear. We examined mice with constitutive and brain region-specific mutations in Neurexin1α, a neuropsychiatric disease-associated synaptic molecule, in value-based choice paradigms. We found Neurexin1α knockouts exhibited reduced selection of beneficial outcomes and impaired avoidance of costlier options. Reinforcement modeling suggested that this was driven by deficits in updating and representation of value. Disruption of Neurexin1α within telencephalic excitatory projection neurons, but not thalamic neurons, recapitulated choice abnormalities of global Neurexin1α knockouts. Furthermore, this selective forebrain excitatory knockout of Neurexin1α perturbed value-modulated neural signals within striatum, a central node in feedback-based reinforcement learning. By relating deficits in value-based decision-making to region-specific Nrxn1α disruption and changes in value-modulated neural activity, we reveal potential neural substrates for the pathophysiology of neuropsychiatric disease-associated cognitive dysfunction.

Striatal direct and indirect pathway neurons differentially control the encoding and updating of goal-directed learning

The posterior dorsomedial striatum (pDMS) is necessary for goal-directed action; however, the role of the direct (dSPN) and indirect (iSPN) spiny projection neurons in the pDMS in such actions remains unclear. In this series of experiments, we examined the role of pDMS SPNs in goal-directed action in rats and found that whereas dSPNs were critical for goal-directed learning and for energizing the learned response, iSPNs were involved in updating that learning to support response flexibility. Instrumental training elevated expression of the plasticity marker Zif268 in dSPNs only, and chemogenetic suppression of dSPN activity during training prevented goal-directed learning. Unilateral optogenetic inhibition of dSPNs induced an ipsilateral response bias in goal-directed action performance. In contrast, although initial goal-directed learning was unaffected by iSPN manipulations, optogenetic inhibition of iSPNs, but not dSPNs, impaired the updating of this learning and attenuated response flexibility after changes in the action-outcome contingency.

Spatial organization of functional clusters representing reward and movement information in the striatal direct and indirect pathways

To obtain insights into striatal neural processes underlying reward-based learning and movement control, we examined spatial organizations of striatal neurons related to movement and reward-based learning. For this, we recorded the activity of direct- and indirect-pathway neurons (D1 and A2a receptor-expressing neurons, respectively) in mice engaged in probabilistic classical conditioning and open-field free exploration. We found broadly organized functional clusters of striatal neurons in the direct as well as indirect pathways for both movement- and reward-related variables. Functional clusters for different variables were partially overlapping in both pathways, but the overlap between outcome- and value-related functional clusters was greater in the indirect than direct pathway. Also, value-related spatial clusters were progressively refined during classical conditioning. Our study shows the broad and learning-dependent spatial organization of functional clusters of dorsal striatal neurons in the direct and indirect pathways. These findings further argue against the classic model of the basal ganglia and support the importance of spatiotemporal patterns of striatal neuronal ensemble activity in the control of behavior.

Exercise-Induced Adaptations to the Mouse Striatal Adenosine System

Adenosine acts as a key regulator of striatum activity, in part, through the antagonistic modulation of dopamine activity. Exercise can increase adenosine activity in the brain, which may impair dopaminergic functions in the striatum. Therefore, long-term repeated bouts of exercise may subsequently generate plasticity in striatal adenosine systems in a manner that promotes dopaminergic activity. This study investigated the effects of long-term voluntary wheel running on adenosine 1 (A₁R), adenosine 2A (A_2AR), dopamine 1 (D₁R), and dopamine 2 (D₂R) receptor protein expression in adult mouse dorsal and ventral striatum structures using immunohistochemistry. In addition, equilibrative nucleoside transporter 1 (ENT1) protein expression was examined after wheel running, as ENT1 regulates the bidirectional flux of adenosine between intra- and extracellular space. The results suggest that eight weeks of running wheel access spared age-related increases of A₁R and A_2AR protein concentrations across the dorsal and ventral striatal structures. Wheel running mildly reduced ENT1 protein levels in ventral striatum subregions. Moreover, wheel running mildly increased D₂R protein density within striatal subregions in the dorsal medial striatum, nucleus accumbens core, and the nucleus accumbens shell. However, D₁R protein expression in the striatum was unchanged by wheel running. These data suggest that exercise promotes adaptations to striatal adenosine systems. Exercise-reduced A₁R and A_2AR and exercise-increased D₂R protein levels may contribute to improved dopaminergic signaling in the striatum. These findings may have implications for cognitive and behavioral processes, as well as motor and psychiatric diseases that involve the striatum.

Distinct roles of striatal direct and indirect pathways in value-based decision making

The striatum is critically involved in value-based decision making. However, it is unclear how striatal direct and indirect pathways work together to make optimal choices in a dynamic and uncertain environment. Here, we examined the effects of selectively inactivating D1 receptor (D1R)- or D2 receptor (D2R)-expressing dorsal striatal neurons (corresponding to direct- and indirect-pathway neurons, respectively) on mouse choice behavior in a reversal task with progressively increasing reversal frequency and a dynamic two-armed bandit task. Inactivation of either D1R- or D2R-expressing striatal neurons impaired performance in both tasks, but the pattern of altered choice behavior differed between the two animal groups. A reinforcement learning model-based analysis indicated that inactivation of D1R- and D2R-expressing striatal neurons selectively impairs value-dependent action selection and value learning, respectively. Our results suggest differential contributions of striatal direct and indirect pathways to two distinct steps in value-based decision making.