Spatial organization of functional clusters representing reward and movement information in the striatal direct and indirect pathways

Dynamics of striatal action selection and reinforcement learning

Spiny projection neurons (SPNs) in dorsal striatum are often proposed as a locus of reinforcement learning in the basal ganglia. Here, we identify and resolve a fundamental inconsistency between striatal reinforcement learning models and known SPN synaptic plasticity rules. Direct-pathway (dSPN) and indirect-pathway (iSPN) neurons, which promote and suppress actions, respectively, exhibit synaptic plasticity that reinforces activity associated with elevated or suppressed dopamine release. We show that iSPN plasticity prevents successful learning, as it reinforces activity patterns associated with negative outcomes. However, this pathological behavior is reversed if functionally opponent dSPNs and iSPNs, which promote and suppress the current behavior, are simultaneously activated by efferent input following action selection. This prediction is supported by striatal recordings and contrasts with prior models of SPN representations. In our model, learning and action selection signals can be multiplexed without interference, enabling learning algorithms beyond those of standard temporal difference models.

Striatal Neurons Are Recruited Dynamically into Collective Representations of Self-Initiated and Learned Actions in Freely Moving Mice

Striatal spiny projection neurons are hyperpolarized-at-rest (HaR) and driven to action potential threshold by a small number of powerful inputs-an input-output configuration that is detrimental to response reliability. Because the striatum is important for habitual behaviors and goal-directed learning, we conducted a microendoscopic imaging in freely moving mice that express a genetically encoded Ca2+ indicator sparsely in striatal HaR neurons to evaluate their response reliability during self-initiated movements and operant conditioning. The sparse expression was critical for longitudinal studies of response reliability, and for studying correlations among HaR neurons while minimizing spurious correlations arising from contamination by the background signal. We found that HaR neurons are recruited dynamically into action representation, with distinct neuronal subsets being engaged in a moment-by-moment fashion. While individual neurons respond with little reliability, the population response remained stable across days. Moreover, we found evidence for the temporal coupling between neuronal subsets during conditioned (but not innate) behaviors.

Feasibility of dopamine as a vector-valued feedback signal in the basal ganglia

It is well established that midbrain dopaminergic neurons support reinforcement learning (RL) in the basal ganglia by transmitting a reward prediction error (RPE) to the striatum. In particular, different computational models and experiments have shown that a striatum-wide RPE signal can support RL over a small discrete set of actions (e.g., no/no-go, choose left/right). However, there is accumulating evidence that the basal ganglia functions not as a selector between predefined actions but rather as a dynamical system with graded, continuous outputs. To reconcile this view with RL, there is a need to explain how dopamine could support learning of continuous outputs, rather than discrete action values. Inspired by the recent observations that besides RPE, the firing rates of midbrain dopaminergic neurons correlate with motor and cognitive variables, we propose a model in which dopamine signal in the striatum carries a vector-valued error feedback signal (a loss gradient) instead of a homogeneous scalar error (a loss). We implement a local, "three-factor" corticostriatal plasticity rule involving the presynaptic firing rate, a postsynaptic factor, and the unique dopamine concentration perceived by each striatal neuron. With this learning rule, we show that such a vector-valued feedback signal results in an increased capacity to learn a multidimensional series of real-valued outputs. Crucially, we demonstrate that this plasticity rule does not require precise nigrostriatal synapses but remains compatible with experimental observations of random placement of varicosities and diffuse volume transmission of dopamine.

Antipsychotic drug efficacy correlates with the modulation of D1 rather than D2 receptor-expressing striatal projection neurons

Elevated dopamine transmission in psychosis is assumed to unbalance striatal output through D1- and D2-receptor-expressing spiny-projection neurons (SPNs). Antipsychotic drugs are thought to re-balance this output by blocking D2 receptors (D2Rs). In this study, we found that amphetamine-driven dopamine release unbalanced D1-SPN and D2-SPN Ca2+ activity in mice, but that antipsychotic efficacy was associated with the reversal of abnormal D1-SPN, rather than D2-SPN, dynamics, even for drugs that are D2R selective or lacking any dopamine receptor affinity. By contrast, a clinically ineffective drug normalized D2-SPN dynamics but exacerbated D1-SPN dynamics under hyperdopaminergic conditions. Consistent with antipsychotic effect, selective D1-SPN inhibition attenuated amphetamine-driven changes in locomotion, sensorimotor gating and hallucination-like perception. Notably, antipsychotic efficacy correlated with the selective inhibition of D1-SPNs only under hyperdopaminergic conditions-a dopamine-state-dependence exhibited by D1R partial agonism but not non-antipsychotic D1R antagonists. Our findings provide new insights into antipsychotic drug mechanism and reveal an important role for D1-SPN modulation.

Septotemporal variations in hippocampal value and outcome processing

A large body of evidence indicates functional variations along the hippocampal longitudinal axis. To investigate whether and how value and outcome processing vary between the dorsal (DH) and the ventral hippocampus (VH), we examined neuronal activity and inactivation effects of the DH and VH in mice performing probabilistic classical conditioning tasks. Inactivation of either structure disrupts value-dependent anticipatory licking, and value-coding neurons are found in both structures, indicating their involvement in value processing. However, the DH neuronal population increases activity as a function of value, while the VH neuronal population is preferentially responsive to the highest-value sensory cue. Also, signals related to outcome-dependent value learning are stronger in the DH. VH neurons instead show rapid responses to punishment and strongly biased responses to negative prediction error. These findings suggest that the DH faithfully represents the external value landscape, whereas the VH preferentially represents behaviorally relevant, salient features of experienced events.

To be and not to be: wide-field Ca2+ imaging reveals neocortical functional segmentation combines stability and flexibility

The stability and flexibility of the functional parcellation of the cerebral cortex is fundamental to how familiar and novel information is both represented and stored. We leveraged new advances in Ca2+ sensors and microscopy to understand the dynamics of functional segmentation in the dorsal cerebral cortex. We performed wide-field Ca2+ imaging in head-fixed mice and used spatial independent component analysis (ICA) to identify independent spatial sources of Ca2+ fluorescence. The imaging data were evaluated over multiple timescales and discrete behaviors including resting, walking, and grooming. When evaluated over the entire dataset, a set of template independent components (ICs) were identified that were common across behaviors. Template ICs were present across a range of timescales, from days to 30 seconds, although with lower occurrence probability at shorter timescales, highlighting the stability of the functional segmentation. Importantly, unique ICs emerged at the shorter duration timescales that could act to transiently refine the cortical network. When data were evaluated by behavior, both common and behavior-specific ICs emerged. Each behavior is composed of unique combinations of common and behavior-specific ICs. These observations suggest that cerebral cortical functional segmentation exhibits considerable spatial stability over time and behaviors while retaining the flexibility for task-dependent reorganization.

Spontaneous behaviour is structured by reinforcement without explicit reward

Spontaneous animal behaviour is built from action modules that are concatenated by the brain into sequences1,2. However, the neural mechanisms that guide the composition of naturalistic, self-motivated behaviour remain unknown. Here we show that dopamine systematically fluctuates in the dorsolateral striatum (DLS) as mice spontaneously express sub-second behavioural modules, despite the absence of task structure, sensory cues or exogenous reward. Photometric recordings and calibrated closed-loop optogenetic manipulations during open field behaviour demonstrate that DLS dopamine fluctuations increase sequence variation over seconds, reinforce the use of associated behavioural modules over minutes, and modulate the vigour with which modules are expressed, without directly influencing movement initiation or moment-to-moment kinematics. Although the reinforcing effects of optogenetic DLS dopamine manipulations vary across behavioural modules and individual mice, these differences are well predicted by observed variation in the relationships between endogenous dopamine and module use. Consistent with the possibility that DLS dopamine fluctuations act as a teaching signal, mice build sequences during exploration as if to maximize dopamine. Together, these findings suggest a model in which the same circuits and computations that govern action choices in structured tasks have a key role in sculpting the content of unconstrained, high-dimensional, spontaneous behaviour.

Cable-free brain imaging for multiple free-moving animals with miniature wireless microscopes

SIGNIFICANCE

Although several miniature microscope systems have been developed to allow researchers to image brain neuron activities of free moving rodents, they generally require a long cable connecting to the miniature microscope. It not only limits the behavior of the animal, but also makes it challenging to study multiple animals simultaneously.

AIM

The aim of this work is to develop a fully wireless miniature microscope that would break constraints from the connecting cables so that the animals could move completely freely, allowing neuroscience researchers to study more of animals' behaviors simultaneously, such as social behavior.

APPROACH

We present a wireless mini-microscope (wScope) that enables simultaneously real-time brain imaging preview from multiple free-moving animals. The wScope has a mass of 2.7 g and a maximum frame rate of 25 Hz at 750    μ m × 450    μ m field of view with 1.8 - μ m resolution.

RESULTS

The performance of the wScope is validated via real-time imaging of the cerebral blood flow and the activity of neurons in the primary visual cortex (V1) of different mice.

CONCLUSIONS

The wScope provides a powerful tool for brain imaging of multiple free moving animals in their much larger spaces and more naturalistic environments.

Place cells are nonrandomly clustered by field location in CA1 hippocampus

A challenge in both modern and historic neuroscience has been achieving an understanding of neuron circuits, and determining the computational and organizational principles that underlie these circuits. Deeper understanding of the organization of brain circuits and cell types, including in the hippocampus, is required for advances in behavioral and cognitive neuroscience, as well as for understanding principles governing brain development and evolution. In this manuscript, we pioneer a new method to analyze the spatial clustering of active neurons in the hippocampus. We use calcium imaging and a rewarded navigation task to record from 100 s of place cells in the CA1 of freely moving rats. We then use statistical techniques developed for and in widespread use in geographic mapping studies, global Moran's I, and local Moran's I to demonstrate that cells that code for similar spatial locations tend to form small spatial clusters. We present evidence that this clustering is not the result of artifacts from calcium imaging, and show that these clusters are primarily formed by cells that have place fields around previously rewarded locations. We go on to show that, although cells with similar place fields tend to form clusters, there is no obvious topographic mapping of environmental location onto the hippocampus, such as seen in the visual cortex. Insights into hippocampal organization, as in this study, can elucidate mechanisms underlying motivational behaviors, spatial navigation, and memory formation.

Wide-Field Calcium Imaging of Neuronal Network Dynamics In Vivo

A central tenet of neuroscience is that sensory, motor, and cognitive behaviors are generated by the communications and interactions among neurons, distributed within and across anatomically and functionally distinct brain regions. Therefore, to decipher how the brain plans, learns, and executes behaviors requires characterizing neuronal activity at multiple spatial and temporal scales. This includes simultaneously recording neuronal dynamics at the mesoscale level to understand the interactions among brain regions during different behavioral and brain states. Wide-field Ca²⁺ imaging, which uses single photon excitation and improved genetically encoded Ca²⁺ indicators, allows for simultaneous recordings of large brain areas and is proving to be a powerful tool to study neuronal activity at the mesoscopic scale in behaving animals. This review details the techniques used for wide-field Ca²⁺ imaging and the various approaches employed for the analyses of the rich neuronal-behavioral data sets obtained. Also discussed is how wide-field Ca²⁺ imaging is providing novel insights into both normal and altered neural processing in disease. Finally, we examine the limitations of the approach and new developments in wide-field Ca²⁺ imaging that are bringing new capabilities to this important technique for investigating large-scale neuronal dynamics.

Misconfigured striatal connectivity profiles in smokers

Dysregulation of frontal cortical inputs to the striatum is foundational in the neural basis of substance use disorder (SUD). Neuroanatomical and electrophysiological data increasingly show that striatal nodes receive appreciable input from numerous cortical areas, and that the combinational properties of these multivariate "connectivity profiles" play a predominant role in shaping striatal activity and function. Yet, how abnormal configuration of striatal connectivity profiles might contribute to SUD is unknown. Here, we implemented a novel "connectivity profile analysis" (CPA) approach using resting-state functional connectivity data to facilitate detection of different types of connectivity profile "misconfiguration" that may reflect distinct forms of aberrant circuit plasticity in SUD. We examined 46 nicotine-dependent smokers and 33 non-smokers and showed that both dorsal striatum (DS) and ventral striatum (VS) connectivity profiles with frontal cortex were misconfigured in smokers-but in doubly distinct fashions. DS misconfigurations were stable across sated and acute abstinent states (indicative of a "trait" circuit adaptation) whereas VS misconfigurations emerged only during acute abstinence (indicative of a "state" circuit adaptation). Moreover, DS misconfigurations involved abnormal connection strength rank order arrangement, whereas VS misconfigurations involved abnormal aggregate strength. We found that caudal ventral putamen in smokers uniquely displayed multiple types of connectivity profile misconfiguration, whose interactive magnitude was linked to dependence severity, and that VS misconfiguration magnitude correlated positively with withdrawal severity during acute abstinence. Findings underscore the potential for approaches that more aptly model the neurobiological composition of corticostriatal circuits to yield deeper insights into the neural basis of SUD.

From ensembles to meta-ensembles: Specific reward encoding by correlated network activity

Neuronal ensembles are local, sparsely distributed populations of neurons that are reliably re-activated by a specific stimulus, context or task. Such discrete cell populations can be defined either functionally, by electrophysiological recordings or in vivo calcium imaging, or anatomically, using the expression of markers such as the immediate early gene cFos. A typical example of tasks that involve the formation of neuronal ensembles is reward learning, such as the cue-reward pairing during operant conditioning. These ensembles are re-activated during cue-presentation and increasing evidence suggests that this re-activation is the neurophysiological basis for the execution of reward-seeking behavior. Whilst the pursuit of rewards is a common daily activity, it is also related to the consumption of drugs, such as alcohol, and may result in problematic behaviors including addiction. Recent research has identified neuronal ensembles in several reward-related brain regions that control distinct aspects of a conditioned response, e.g., contextual information about the availability of a specific reward or the actions needed to retrieve this reward under the given circumstances. Here, we review studies using the activity marker cFos to identify and characterize neuronal ensembles related to alcohol and non-drug rewards with a special emphasis on the discrimination between different rewards by meta-ensembles, i.e., by dynamic co-activation of multiple ensembles across different brain areas.

In Vivo Multi-Day Calcium Imaging of CA1 Hippocampus in Freely Moving Rats Reveals a High Preponderance of Place Cells with Consistent Place Fields

Calcium imaging using GCaMP indicators and miniature microscopes has been used to image cellular populations during long timescales and in different task phases, as well as to determine neuronal circuit topology and organization. Because the hippocampus (HPC) is essential for tasks of memory, spatial navigation, and learning, calcium imaging of large populations of HPC neurons can provide new insight on cell changes over time during these tasks. All reported HPC in vivo calcium imaging experiments have been done in mouse. However, rats have many behavioral and physiological experimental advantages over mice. In this paper, we present the first (to our knowledge) in vivo calcium imaging from CA1 HPC in freely moving male rats. Using the UCLA Miniscope, we demonstrate that, in rat, hundreds of cells can be visualized and held across weeks. We show that calcium events in these cells are highly correlated with periods of movement, with few calcium events occurring during periods without movement. We additionally show that an extremely large percent of cells recorded during a navigational task are place cells (77.3 ± 5.0%, surpassing the percent seen during mouse calcium imaging), and that these cells enable accurate decoding of animal position and can be held over days with consistent place fields in a consistent spatial map. A detailed protocol is included, and implications of these advancements on in vivo imaging and place field literature are discussed.SIGNIFICANCE STATEMENT In vivo calcium imaging in freely moving animals allows the visualization of cellular activity across days. In this paper, we present the first in vivo Ca2+ recording from CA1 hippocampus (HPC) in freely moving rats. We demonstrate that hundreds of cells can be visualized and held across weeks, and that calcium activity corresponds to periods of movement. We show that a high percentage (77.3 ± 5.0%) of imaged cells are place cells, and that these place cells enable accurate decoding and can be held stably over days with little change in field location. Because the HPC is essential for many tasks involving memory, navigation, and learning, imaging of large populations of HPC neurons can shed new insight on cellular activity changes and organization.

Parallel processing of working memory and temporal information by distinct types of cortical projection neurons

It is unclear how different types of cortical projection neurons work together to support diverse cortical functions. We examined the discharge characteristics and inactivation effects of intratelencephalic (IT) and pyramidal tract (PT) neurons-two major types of cortical excitatory neurons that project to cortical and subcortical structures, respectively-in the deep layer of the medial prefrontal cortex in mice performing a delayed response task. We found stronger target-dependent firing of IT than PT neurons during the delay period. We also found the inactivation of IT neurons, but not PT neurons, impairs behavioral performance. In contrast, PT neurons carry more temporal information than IT neurons during the delay period. Our results indicate a division of labor between IT and PT projection neurons in the prefrontal cortex for the maintenance of working memory and for tracking the passage of time, respectively.

Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment

Significant evidence supports the view that dopamine shapes learning by encoding reward prediction errors. However, it is unknown whether striatal targets receive tailored dopamine dynamics based on regional functional specialization. Here, we report wave-like spatiotemporal activity patterns in dopamine axons and release across the dorsal striatum. These waves switch between activational motifs and organize dopamine transients into localized clusters within functionally related striatal subregions. Notably, wave trajectories were tailored to task demands, propagating from dorsomedial to dorsolateral striatum when rewards are contingent on animal behavior and in the opponent direction when rewards are independent of behavioral responses. We propose a computational architecture in which striatal dopamine waves are sculpted by inference about agency and provide a mechanism to direct credit assignment to specialized striatal subregions. Supporting model predictions, dorsomedial dopamine activity during reward-pursuit signaled the extent of instrumental control and interacted with reward waves to predict future behavioral adjustments.