A competitive model for striatal action selection

Cocaine and sucrose rewards recruit different seeking ensembles in the nucleus accumbens core

Poorly regulated reward seeking is a central feature of substance use disorder. Recent research shows that rewarding drug-related experiences induce synchronous activation of a discrete number of neurons in the nucleus accumbens that are causally linked to reward-related contexts. Here we comprehensively characterize the specific ensemble of neurons built through experience that are linked to seeking behavior. We additionally address the question of whether or not addictive drugs usurp the neuronal networks recruited by natural rewards by evaluating cocaine- and sucrose-associated ensembles within the same mouse. We used FosCreERT2/+/Ai14 transgenic mice to tag cells activated by and potentially encoding cocaine and sucrose seeking. We tagged ~1% of neurons in the core subregion of the accumbens (NAcore) activated during cue-induced seeking for cocaine or sucrose. The majority of tagged cells in the seeking ensembles were D1-MSNs, and specifically activated during seeking, not during extinction or when mice remained in the home cage. To compare different reward-specific ensembles within the same mouse, we used a dual cocaine and sucrose self-administration protocol allowing reward-specific seeking. Using this model, we found ~70% distinction between the cells constituting the cocaine- compared to the sucrose-seeking ensemble. Establishing that cocaine recruits an ensemble of NAcore neurons largely distinct from neurons recruited into an ensemble coding for sucrose seeking suggest a finely tuned specificity of ensembles. The findings allow further exploration of the mechanisms that transform reward-based positive reinforcement into maladaptive drug seeking.

Adiposity covaries with signatures of asymmetric feedback learning during adaptive decisions

Unhealthy weight gain relates, in part, to how people make decisions based on prior experience. Here we conducted post hoc analysis on an archival data set to evaluate whether individual differences in adiposity, an anthropometric construct encompassing a spectrum of body types, from lean to obese, associate with signatures of asymmetric feedback learning during value-based decision-making. In a sample of neurologically healthy adults (N = 433), ventral striatal responses to rewards, measured using fMRI, were not directly associated with adiposity, but rather moderated its relationship with feedback-driven learning in the Iowa gambling task, tested outside the scanner. Using a biologically inspired model of basal ganglia-dependent decision processes, we found this moderating effect of reward reactivity to be explained by an asymmetrical use of feedback to drive learning; that is, with more plasticity for gains than for losses, stronger reward reactivity leads to decisions that minimize exploration for maximizing long-term outcomes. Follow-up analysis confirmed that individual differences in adiposity correlated with signatures of asymmetric use of feedback cues during learning, suggesting that reward reactivity may especially relate to adiposity, and possibly obesity risk, when gains impact future decisions more than losses.

Dopaminergic Control over the Tripartite Synapse

In this issue of Neuron, Corkrum et al. (2020) demonstrate an unexpected role for dopamine D₁ receptors on astrocytes located in the nucleus accumbens, a key structure of the brain's reward system. Activation of these receptors mediates dopamine-evoked depression of excitatory synaptic transmission, which contributes to amphetamine's psychomotor effects.

Opposing patterns of abnormal D1 and D2 receptor dependent cortico-striatal plasticity explain increased risk taking in patients with DYT1 dystonia

Patients with DYT1 dystonia caused by the mutated TOR1A gene exhibit risk neutral behaviour compared to controls who are risk averse in the same reinforcement learning task. It is unclear whether this behaviour can be linked to changes in cortico-striatal plasticity demonstrated in animal models which share the same TOR1A mutation. We hypothesised that we could reproduce the experimental risk taking behaviour using a model of the basal ganglia under conditions where cortico-striatal plasticity was abnormal. As dopamine exerts opposing effects on cortico-striatal plasticity via different receptors expressed on medium spiny neurons (MSN) of the direct (D1R dominant, dMSNs) and indirect (D2R dominant, iMSNs) pathways, we tested whether abnormalities in cortico-striatal plasticity in one or both of these pathways could explain the patient's behaviour. Our model could generate simulated behaviour indistinguishable from patients when cortico-striatal plasticity was abnormal in both dMSNs and iMSNs in opposite directions. The risk neutral behaviour of the patients was replicated when increased cortico-striatal long term potentiation in dMSN's was in combination with increased long term depression in iMSN's. This result is consistent with previous observations in rodent models of increased cortico-striatal plasticity at in dMSNs, but contrasts with the pattern reported in vitro of dopamine D2 receptor dependant increases in cortico-striatal LTP and loss of LTD at iMSNs. These results suggest that additional factors in patients who manifest motor symptoms may lead to divergent effects on D2 receptor dependant cortico-striatal plasticity that are not apparent in rodent models of this disease.

Prepubertal gonadectomy reveals sex differences in approach-avoidance behavior in adult mice

Adolescence is a developmental period that is associated with physical, cognitive, and affective maturation and a time when sex biases in multiple psychiatric diseases emerge. While puberty onset marks the initiation of adolescence, it is unclear whether the pubertal rise in gonadal hormones generates sex differences in approach-avoidance behaviors that may impact psychiatric vulnerability. To examine the influence of pubertal development on adult behavior, we removed the gonads or performed sham surgery in male and female mice just prior to puberty onset and assessed performance in an odor-guided foraging task and anxiety-related behaviors in adulthood. We observed no significant sex differences in foraging or anxiety-related behaviors between intact adult male and female mice but found significant differences between adult male and female mice that had been gonadectomized (GDX) prior to puberty onset. GDX males failed to acquire the odor-guided foraging task, showed reduced locomotion, and exhibited increased anxiety-like behavior, while GDX females showed the opposite pattern of behavior. These data suggest that puberty may minimize rather than drive differences in approach-avoidance phenotypes in male and female mice.

Continuous Representations of Speed by Striatal Medium Spiny Neurons

The striatum is critical for controlling motor output. However, it remains unclear how striatal output neurons encode and facilitate movement. A prominent theory suggests that striatal units encode movements in bursts of activity near specific events, such as the start or end of actions. These bursts are theorized to gate or permit specific motor actions, thereby encoding and facilitating complex sequences of actions. An alternative theory has suggested that striatal neurons encode continuous changes in sensory or motor information with graded changes in firing rate. Supporting this theory, many striatal neurons exhibit such graded changes without bursting near specific actions. Here, we evaluated these two theories in the same recordings of mice (both male and female). We recorded single-unit and multiunit activity from the dorsomedial striatum of mice as they spontaneously explored an arena. We observed both types of encoding, although continuous encoding was more prevalent than bursting near movement initiation or termination. The majority of recorded units did not exhibit positive linear relationships with speed but instead exhibited nonlinear relationships that peaked at a range of locomotor speeds. Bulk calcium recordings of identified direct and indirect pathway neurons revealed similar speed tuning profiles, indicating that the heterogeneity in response profiles was not due to this genetic distinction. We conclude that continuous encoding of speed is a central component of movement encoding in the striatum.SIGNIFICANCE STATEMENT The striatum is a structure that is linked to volitional movements and is a primary site of pathology in movement disorders. It remains unclear how striatal neurons encode motor parameters and use them to facilitate movement. Here, we evaluated two models for this: a "discrete encoding model" in which striatal neurons facilitate movements with brief burst of activity near the start and end of movements, and a "continuous encoding model," in which striatal neurons encode the sensory or motor state of the animal with continuous changes in firing. We found evidence primarily in support of the continuous encoding model. This may have implications for understanding the striatal control of movement, as well as informing therapeutic approaches for treating movement disorders.

Striatal circuits for reward learning and decision-making

The striatum is essential for learning which actions lead to reward and for implementing those actions. Decades of experimental and theoretical work have led to several influential theories and hypotheses about how the striatal circuit mediates these functions. However, owing to technical limitations, testing these hypotheses rigorously has been difficult. In this Review, we briefly describe some of the classic ideas of striatal function. We then review recent studies in rodents that take advantage of optical and genetic methods to test these classic ideas by recording and manipulating identified cell types within the circuit. This new body of work has provided experimental support of some longstanding ideas about the striatal circuit and has uncovered critical aspects of the classic view that are incorrect or incomplete.

Dorsal Striatal Circuits for Habits, Compulsions and Addictions

Here, we review the neural circuit bases of habits, compulsions, and addictions, behaviors which are all characterized by relatively automatic action performance. We discuss relevant studies, primarily from the rodent literature, and describe how major headway has been made in identifying the brain regions and neural cell types whose activity is modulated during the acquisition and performance of these automated behaviors. The dorsal striatum and cortical inputs to this structure have emerged as key players in the wider basal ganglia circuitry encoding behavioral automaticity, and changes in the activity of different neuronal cell-types in these brain regions have been shown to co-occur with the formation of automatic behaviors. We highlight how disordered functioning of these neural circuits can result in neuropsychiatric disorders, such as obsessive-compulsive disorder (OCD) and drug addiction. Finally, we discuss how the next phase of research in the field may benefit from integration of approaches for access to cells based on their genetic makeup, activity, connectivity and precise anatomical location.

High-Frequency Activation of Nucleus Accumbens D1-MSNs Drives Excitatory Potentiation on D2-MSNs

Subtypes of nucleus accumbens medium spiny neurons (MSNs) promote dichotomous outcomes in motivated behaviors. However, recent reports indicate enhancing activity of either nucleus accumbens (NAc) core MSN subtype augments reward, suggesting coincident MSN activity may underlie this outcome. Here, we report a collateral excitation mechanism in which high-frequency, NAc core dopamine 1 (D1)-MSN activation causes long-lasting potentiation of excitatory transmission (LLP) on dopamine receptor 2 (D2)-MSNs. Our mechanistic investigation demonstrates that this form of plasticity requires release of the excitatory peptide substance P from D1-MSNs and robust cholinergic interneuron activation through neurokinin receptor stimulation. We also reveal that D2-MSN LLP requires muscarinic 1 receptor activation, intracellular calcium signaling, and GluR2-lacking AMPAR insertion. This study uncovers a mechanism for shaping NAc core activity through the transfer of excitatory information from D1-MSNs to D2-MSNs and may provide a means for altering goal-directed behavior through coordinated MSN activity.

Reward-driven changes in striatal pathway competition shape evidence evaluation in decision-making

Cortico-basal-ganglia-thalamic (CBGT) networks are critical for adaptive decision-making, yet how changes to circuit-level properties impact cognitive algorithms remains unclear. Here we explore how dopaminergic plasticity at corticostriatal synapses alters competition between striatal pathways, impacting the evidence accumulation process during decision-making. Spike-timing dependent plasticity simulations showed that dopaminergic feedback based on rewards modified the ratio of direct and indirect corticostriatal weights within opposing action channels. Using the learned weight ratios in a full spiking CBGT network model, we simulated neural dynamics and decision outcomes in a reward-driven decision task and fit them with a drift diffusion model. Fits revealed that the rate of evidence accumulation varied with inter-channel differences in direct pathway activity while boundary height varied with overall indirect pathway activity. This multi-level modeling approach demonstrates how complementary learning and decision computations can emerge from corticostriatal plasticity.

Cognitive process models such as reinforcement learning (RL) and the drift diffusion model (DDM) have helped to elucidate the basic algorithms underlying error-corrective learning and the evaluation of accumulating decision evidence leading up to a choice. While these relatively abstract models help to guide experimental and theoretical probes into associated phenomena, they remain uninformative about the actual physical mechanics by which learning and decision algorithms are carried out in a neurobiological substrate during adaptive choice behavior. Here we present an “upwards mapping” approach to bridging neural and cognitive models of value-based decision-making, showing how dopaminergic feedback alters the network-level dynamics of cortico-basal-ganglia-thalamic (CBGT) pathways during learning to bias behavioral choice towards more rewarding actions. By mapping “up” the levels of analysis, this approach yields specific predictions about aspects of neuronal activity that map to the quantities appearing in the cognitive decision-making framework.

Errors in Action Timing and Inhibition Facilitate Learning by Tuning Distinct Mechanisms in the Underlying Decision Process

Goal-directed behavior requires integrating action selection processes with learning systems that adapt control using environmental feedback. These functions are known to intersect at a common neural substrate with multiple known targets of plasticity (the cortico-basal ganglia-thalamic network), suggesting that feedback signals have a multifaceted impact on future decisions. Using a hybrid of accumulation-to-bound decision models and reinforcement learning, we modeled the performance of humans in a stop signal task where participants (N 75: 37 males, 38 females) learned the prior distribution of the timing of a stop signal through trial-and-error feedback. Changes in the drift rate of the action execution process were driven by errors in action timing, whereas adaptation in the boundary height served to increase caution following failed stops. These findings highlight two interactive learning mechanisms for adapting the control of goal-directed actions based on dissociable dimensions of feedback error.SIGNIFICANCE STATEMENT Many complex behavioral goals rely on the ability to regulate the timing of action execution while also maintaining enough control to cancel actions in response to "Stop" cues in the environment. Here we examined how these fundamental components of behavior become tuned to the control demands of the environment by combining principles of reinforcement learning with accumulation-to-bound models. Model fits to behavioral data in an adaptive stop signal task revealed two adaptive mechanisms: (1) timing error-related changes in the rate of the execution signal; and (2) an increase in the execution boundary after failed stops. These findings demonstrate unique effects of timing and control errors on the underlying mechanisms of control, the rate and threshold of accumulating action signals.