Single-cell activity tracking reveals that orbitofrontal neurons acquire and maintain a long-term memory to guide behavioral adaptation

Formation of an Enduring Ensemble of Accumbens Neurons Leads to Prepotent Seeking for Cocaine Over Natural Reward Cues

Neuronal activity in the nucleus accumbens core (NAcore) is necessary for reward-seeking behaviors. We hypothesized that the differential encoding of natural and drug rewards in the NAcore contributes to substance use disorder. We leveraged single-cell calcium imaging of dopamine D1- and D2-receptor-expressing medium spiny neurons (MSNs) in the NAcore of mice to examine differences between sucrose and cocaine rewarded (self-administration) and unrewarded (abstinent and cue-induced) seeking. Activity was time-locked to nose-poking for reward, clustered, and compared between sucrose and cocaine. Only in cocaine-trained mice were excited D1-MSNs securely stable, capable of decoding nose-poking in all rewarded and unrewarded sessions and correlated with the intensity of nose-poking for unrewarded seeking. Furthermore, D1-MSNs formed a stable ensemble predictive of seeking behavior after extended cocaine, but not sucrose abstinence. The excited D1-MSN ensemble uniquely drives cue-induced cocaine seeking and may contribute to why drug seeking is prepotent over natural reward seeking in cocaine use disorder.

Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time

The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference learning (TD) learning, whereby certain units signal reward prediction errors (RPE). The TD algorithm has been traditionally mapped onto the dopaminergic system, as firing properties of dopamine neurons can resemble RPEs. However, certain predictions of TD learning are inconsistent with experimental results, and previous implementations of the algorithm have made unscalable assumptions regarding stimulus-specific fixed temporal bases. We propose an alternate framework to describe dopamine signaling in the brain, FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, dopamine release is similar, but not identical to RPE, leading to predictions that contrast to those of TD. While FLEX itself is a general theoretical framework, we describe a specific, biophysically plausible implementation, the results of which are consistent with a preponderance of both existing and reanalyzed experimental data.

Corticostriatal ensemble dynamics across heroin self-administration to reinstatement

Corticostriatal projection neurons from prelimbic medial prefrontal cortex to the nucleus accumbens core critically regulate drug-seeking behaviors, yet the underlying encoding dynamics whereby these neurons contribute to drug seeking remain elusive. Here we use two-photon calcium imaging to visualize the activity of corticostriatal neurons in mice from the onset of heroin use to relapse. We find that the activity of these neurons is highly heterogeneous during heroin self-administration and seeking, with at least 8 distinct neuronal ensembles that display both excitatory and inhibitory encoding dynamics. These neuronal ensembles are particularly apparent during relapse, where excitatory responses are amplified compared to heroin self-administration. Moreover, we find that optogenetic inhibition of corticostriatal projection neurons attenuates heroin seeking regardless of the relapse trigger. Our results reveal the precise corticostriatal activity dynamics underlying drug-seeking behaviors and support a key role for this circuit in mediating relapse to drug seeking.

Body Satisfaction, Exercise Dependence, and White Matter Microstructure in Young Adults

BACKGROUND

Self-body satisfaction is considered a psychological factor for exercise dependence (EXD). However, the potential neuropsychological mechanisms underlying this association remain unclear.

PURPOSE

To investigate the role of white matter microstructure in the association between body satisfaction and EXD.

STUDY TYPE

Prospective.

POPULATION

One hundred eight regular exercisers (age 22.11 ± 2.62 years; 58 female).

FIELD STRENGTH/SEQUENCE

3.0 Tesla; diffusion-weighted echo planar imaging with 30 directions.

ASSESSMENT

The Body Shape Satisfaction (BSS) and Exercise Dependence Scale (EDS); whole-brain tract-based spatial statistics (TBSS) and correlational tractography analyses; average fractional anisotropy (FA) and quantitative anisotropy (QA) values of obtained tracts.

STATISTICAL TESTS

The whole-brain regression model, mediation analysis, and simple slope analysis. P values <0.05 were defined as statistically significant.

RESULTS

The BSS and EDS scores were 37.33 ± 6.32 and 68.22 ± 13.88, respectively. TBSS showed negative correlations between EDS and FA values in the bilateral corticospinal tract (CST, r = -0.41), right cingulum (r = -0.41), and left superior thalamic radiation (STR, r = -0.50). Correlational tractography showed negative associations between EDS and QA values of the left inferior frontal occipital fasciculus (r = -0.35), STR (r = -0.42), CST (r = -0.31), and right cingulum (r = -0.28). The FA values, rather than QA values, mediated the BSS-EDS association (indirect effects = 0.30). The BSS was significantly associated with the EDS score at both low (β = 1.02) and high (β = 0.43) levels of FA value, while the association was significant only at the high level of QA value (β = 1.26).

DATA CONCLUSION

EXD was correlated with white matter in frontal-subcortical and sensorimotor networks, and these tracts mediated the body satisfaction-EXD association. White matter microstructure could be a promising neural signature for understanding the underlying neuropsychological mechanisms of EXD.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY: Stage 1.

Adolescent alcohol exposure persistently alters orbitofrontal cortical encoding of Pavlovian conditional stimulus components in female rats

Exposure to alcohol during adolescence impacts cortical and limbic brain regions undergoing maturation. In rodent models, long-term effects on behavior and neurophysiology have been described after adolescent intermittent ethanol (AIE), especially in males. We hypothesized that AIE in female rats increases conditional approach to a reward-predictive cue and corresponding neuronal activity in the orbitofrontal cortex (OFC) and nucleus accumbens (NAc). We evaluated behavior and neuronal firing after AIE (5 g/kg intragastric) or water (CON) in adult female rats. Both AIE and CON groups expressed a ST phenotype, and AIE marginally increased sign-tracking (ST) and decreased goal-tracking (GT) metrics. NAc neurons exhibited phasic firing patterns to the conditional stimulus (CS), with no differences between groups. In contrast, neuronal firing in the OFC of AIE animals was greater at CS onset and offset than in CON animals. During reward omission, OFC responses to CS offset normalized to CON levels, but enhanced OFC firing to CS onset persisted in AIE. We suggest that the enhanced OFC neural activity observed in AIE rats to the CS could contribute to behavioral inflexibility. Ultimately, AIE persistently impacts the neurocircuitry of reward-motivated behavior in female rats.

Whole-brain Mapping of Inputs and Outputs of Specific Orbitofrontal Cortical Neurons in Mice

The orbitofrontal cortex (ORB), a region crucial for stimulus-reward association, decision-making, and flexible behaviors, extensively connects with other brain areas. However, brain-wide inputs to projection-defined ORB neurons and the distribution of inhibitory neurons postsynaptic to neurons in specific ORB subregions remain poorly characterized. Here we mapped the inputs of five types of projection-specific ORB neurons and ORB outputs to two types of inhibitory neurons. We found that different projection-defined ORB neurons received inputs from similar cortical and thalamic regions, albeit with quantitative variations, particularly in somatomotor areas and medial groups of the dorsal thalamus. By counting parvalbumin (PV) or somatostatin (SST) interneurons innervated by neurons in specific ORB subregions, we found a higher fraction of PV neurons in sensory cortices and a higher fraction of SST neurons in subcortical regions targeted by medial ORB neurons. These results provide insights into understanding and investigating the function of specific ORB neurons.

Hyperactivity of indirect pathway-projecting spiny projection neurons promotes compulsive behavior

Compulsive behaviors are a hallmark symptom of obsessive compulsive disorder (OCD). Striatal hyperactivity has been linked to compulsive behavior generation in correlative studies in humans and causal studies in rodents. However, the contribution of the two distinct striatal output populations to the generation and treatment of compulsive behavior is unknown. These populations of direct and indirect pathway-projecting spiny projection neurons (SPNs) have classically been thought to promote or suppress actions, respectively, leading to a long-held hypothesis that increased output of direct relative to indirect pathway promotes compulsive behavior. Contrary to this hypothesis, here we find that indirect pathway hyperactivity is associated with compulsive grooming in the Sapap3-knockout mouse model of OCD-relevant behavior. Furthermore, we show that suppression of indirect pathway activity using optogenetics or treatment with the first-line OCD pharmacotherapy fluoxetine is associated with reduced grooming in Sapap3-knockouts. Together, these findings highlight the striatal indirect pathway as a potential treatment target for compulsive behavior.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects are often willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No Info) vs. a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%) and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) in the anterior cingulate cortex, orbitofrontal cortex, basolateral amygdala, or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of the anterior cingulate cortex destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that previous choice predicted current choice in all conditions, however previously rewarded Info trials strongly predicted preference in all conditions except in female rats following anterior cingulate cortex inhibition. The results reveal a causal, sex-dependent role for the anterior cingulate cortex in decisions involving information.

An open-source behavior controller for associative learning and memory (B-CALM)

Associative learning and memory, i.e., learning and remembering the associations between environmental stimuli, self-generated actions, and outcomes such as rewards or punishments, are critical for the well-being of animals. Hence, the neural mechanisms underlying these processes are extensively studied using behavioral tasks in laboratory animals. Traditionally, these tasks have been controlled using commercial hardware and software, which limits scalability and accessibility due to their cost. More recently, due to the revolution in microcontrollers or microcomputers, several general-purpose and open-source solutions have been advanced for controlling neuroscientific behavioral tasks. While these solutions have great strength due to their flexibility and general-purpose nature, for the same reasons, they suffer from some disadvantages including the need for considerable programming expertise, limited online visualization, or slower than optimal response latencies for any specific task. Here, to mitigate these concerns, we present an open-source behavior controller for associative learning and memory (B-CALM). B-CALM provides an integrated suite that can control a host of associative learning and memory behaviors. As proof of principle for its applicability, we show data from head-fixed mice learning Pavlovian conditioning, operant conditioning, discrimination learning, as well as a timing task and a choice task. These can be run directly from a user-friendly graphical user interface (GUI) written in MATLAB that controls many independently running Arduino Mega microcontrollers in parallel (one per behavior box). In sum, B-CALM will enable researchers to execute a wide variety of associative learning and memory tasks in a scalable, accurate, and user-friendly manner.

A persistent prefrontal reference frame across time and task rules

Behavior can be remarkably consistent, even over extended time periods, yet whether this is reflected in stable or 'drifting' neuronal responses to task features remains controversial. Here, we find a persistently active ensemble of neurons in the medial prefrontal cortex (mPFC) of mice that reliably maintains trajectory-specific tuning over several weeks while performing an olfaction-guided spatial memory task. This task-specific reference frame is stabilized during learning, upon which repeatedly active neurons show little representational drift and maintain their trajectory-specific tuning across long pauses in task exposure and across repeated changes in cue-target location pairings. These data thus suggest a 'core ensemble' of prefrontal neurons forming a reference frame of task-relevant space for the performance of consistent behavior over extended periods of time.

Restoration of a paraventricular thalamo-accumbal behavioral suppression circuit prevents reinstatement of heroin seeking

Lack of behavioral suppression typifies substance use disorders, yet the neural circuit underpinnings of drug-induced behavioral disinhibition remain unclear. Here, we employ deep-brain two-photon calcium imaging in heroin self-administering mice, longitudinally tracking adaptations within a paraventricular thalamus to nucleus accumbens behavioral inhibition circuit from the onset of heroin use to reinstatement. We find that select thalamo-accumbal neuronal ensembles become profoundly hypoactive across the development of heroin seeking and use. Electrophysiological experiments further reveal persistent adaptations at thalamo-accumbal parvalbumin interneuronal synapses, whereas functional rescue of these synapses prevents multiple triggers from initiating reinstatement of heroin seeking. Finally, we find an enrichment of μ-opioid receptors in output- and cell-type-specific paraventricular thalamic neurons, which provide a mechanism for heroin-induced synaptic plasticity and behavioral disinhibition. These findings reveal key circuit adaptations that underlie behavioral disinhibition in opioid dependence and further suggest that recovery of this system would reduce relapse susceptibility.

Midbrain signaling of identity prediction errors depends on orbitofrontal cortex networks

Outcome-guided behavior requires knowledge about the identity of future rewards. Previous work across species has shown that the dopaminergic midbrain responds to violations in expected reward identity and that the lateral orbitofrontal cortex (OFC) represents reward identity expectations. Here we used network-targeted transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) during a trans-reinforcer reversal learning task to test the hypothesis that outcome expectations in the lateral OFC contribute to the computation of identity prediction errors (iPE) in the midbrain. Network-targeted TMS aiming at lateral OFC reduced the global connectedness of the lateral OFC and impaired reward identity learning in the first block of trials. Critically, TMS disrupted neural representations of expected reward identity in the OFC and modulated iPE responses in the midbrain. These results support the idea that iPE signals in the dopaminergic midbrain are computed based on outcome expectations represented in the lateral OFC.

Holographic stimulation of opposing amygdala ensembles bidirectionally modulates valence-specific behavior via mutual inhibition

The basolateral amygdala (BLA) is an evolutionarily conserved brain region, well known for valence processing. Despite this central role, the relationship between activity of BLA neuronal ensembles in response to appetitive and aversive stimuli and the subsequent expression of valence-specific behavior has remained elusive. Here, we leverage two-photon calcium imaging combined with single-cell holographic photostimulation through an endoscopic lens to demonstrate a direct causal role for opposing ensembles of BLA neurons in the control of oppositely valenced behavior in mice. We report that targeted photostimulation of either appetitive or aversive BLA ensembles results in mutual inhibition and shifts behavioral responses to promote consumption of an aversive tastant or reduce consumption of an appetitive tastant, respectively. Here, we identify that neuronal encoding of valence in the BLA is graded and relies on the relative proportion of individual BLA neurons recruited in a stable appetitive or quinine ensemble.

The role of prospective contingency in the control of behavior and dopamine signals during associative learning

Associative learning depends on contingency, the degree to which a stimulus predicts an outcome. Despite its importance, the neural mechanisms linking contingency to behavior remain elusive. Here we examined the dopamine activity in the ventral striatum - a signal implicated in associative learning - in a Pavlovian contingency degradation task in mice. We show that both anticipatory licking and dopamine responses to a conditioned stimulus decreased when additional rewards were delivered uncued, but remained unchanged if additional rewards were cued. These results conflict with contingency-based accounts using a traditional definition of contingency or a novel causal learning model (ANCCR), but can be explained by temporal difference (TD) learning models equipped with an appropriate inter-trial-interval (ITI) state representation. Recurrent neural networks trained within a TD framework develop state representations like our best 'handcrafted' model. Our findings suggest that the TD error can be a measure that describes both contingency and dopaminergic activity.

Neural signatures of opioid-induced risk-taking behavior in the prelimbic prefrontal cortex

Opioid use disorder occurs alongside impaired risk-related decision-making, but the underlying neural correlates are unclear. We developed a novel approach-avoidance conflict model using a modified conditioned place preference paradigm to study neural signals of risky opioid seeking in the prefrontal cortex, a region implicated in executive decision making. Upon establishment of morphine conditioned place preference, rats underwent a subsequent conflict test in which fear-inducing cat odor was introduced in the previously drug-paired side of the apparatus. While the saline control group avoided the cat odor side, the morphine group maintained preference for the paired side despite the presence of cat odor. K-means clustering identified two subsets of morphine-treated rats that exhibited either persistent drug seeking (Risk-Takers) or increased avoidance (Risk-Avoiders) during conflict. Single-unit recordings from the prelimbic cortex (PL) revealed decreased neuronal firing rates upon acute morphine exposure in both Risk-Takers and Risk-Avoiders, but this firing rate suppression was absent after repeated administration. Risk-Avoiders also displayed distinct post-morphine excitation in PL which persisted across conditioning. During the preference test, subpopulations of PL neurons in all groups were either excited or inhibited when rats entered the paired side. Interestingly, while this inhibitory signal was lost during the subsequent conflict test in both saline and Risk-Avoider groups, these inhibitory responses persisted in Risk-Takers. Our results suggest that loss of PL inhibition after opioid conditioning is associated with the formation of contextual reward memory. Furthermore, persistent PL inhibitory signaling in the drug-associated context during conflict may underlie increased risk taking following opioid exposure.

Identifying and modulating neural signatures of stress susceptibility and resilience enables control of anhedonia

Anhedonia is a core aspect of major depressive disorder. Traditionally viewed as a blunted emotional state in which individuals are unable to experience joy, anhedonia also diminishes the drive to seek rewards and the ability to value and learn about them 1-4.The neural underpinnings of anhedonia and how this emotional state drives related behavioral changes remain unclear. Here, we investigated these questions by taking advantage of the fact that when mice are exposed to traumatic social stress, susceptible animals become socially withdrawn and anhedonic, where they cease to seek high-value rewards, while others remain resilient. By performing high density electrophysiological recordings and comparing neural activity patterns of these groups in the basolateral amygdala (BLA) and ventral CA1 (vCA1) of awake behaving animals, we identified neural signatures of susceptibility and resilience to anhedonia. When animals actively sought rewards, BLA activity in resilient mice showed stronger discrimination between upcoming reward choices. In contrast, susceptible mice displayed a rumination-like signature, where BLA neurons encoded the intention to switch or stay on a previously chosen reward. When animals were at rest, the spontaneous BLA activity of susceptible mice was higher dimensional than in controls, reflecting a greater number of distinct neural population states. Notably, this spontaneous activity allowed us to decode group identity and to infer if a mouse had a history of stress better than behavioral outcomes alone. Finally, targeted manipulation of vCA1 inputs to the BLA in susceptible mice rescued dysfunctional neural dynamics, amplified dynamics associated with resilience, and reversed their anhedonic behavior. This work reveals population-level neural signatures that explain individual differences in responses to traumatic stress, and suggests that modulating vCA1-BLA inputs can enhance resilience by regulating these dynamics.

Opioid-driven disruption of the septal complex reveals a role for neurotensin-expressing neurons in withdrawal

Because opioid withdrawal is an intensely aversive experience, persons with opioid use disorder (OUD) often relapse to avoid it. The lateral septum (LS) is a forebrain structure that is important in aversion processing, and previous studies have linked the lateral septum (LS) to substance use disorders. It is unclear, however, which precise LS cell types might contribute to the maladaptive state of withdrawal. To address this, we used single-nucleus RNA-sequencing to interrogate cell type specific gene expression changes induced by chronic morphine and withdrawal. We discovered that morphine globally disrupted the transcriptional profile of LS cell types, but Neurotensin-expressing neurons (Nts; LS-Nts neurons) were selectively activated by naloxone. Using two-photon calcium imaging and ex vivo electrophysiology, we next demonstrate that LS-Nts neurons receive enhanced glutamatergic drive in morphine-dependent mice and remain hyperactivated during opioid withdrawal. Finally, we showed that activating and silencing LS-Nts neurons during opioid withdrawal regulates pain coping behaviors and sociability. Together, these results suggest that LS-Nts neurons are a key neural substrate involved in opioid withdrawal and establish the LS as a crucial regulator of adaptive behaviors, specifically pertaining to OUD.

Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including the highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, the unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory designer receptors exclusively activated by designer drugs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-/post-reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of the ventrolateral orbitofrontal cortex (vlOFC), but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Neuronal tuning to threat exposure remains stable in the mouse prefrontal cortex over multiple days

Intense threat elicits action in the form of active and passive coping. The medial prefrontal cortex (mPFC) executes top-level control over the selection of threat coping strategies, but the dynamics of mPFC activity upon continuing threat encounters remain unexplored. Here, we used 1-photon calcium imaging in mice to probe the activity of prefrontal pyramidal cells during repeated exposure to intense threat in a tail suspension (TS) paradigm. A subset of prefrontal neurons displayed selective activation during TS, which was stably maintained over days. During threat, neurons showed specific tuning to active or passive coping. These responses were unrelated to general motion tuning and persisted over days. Moreover, the neural manifold traversed by low-dimensional population activity remained stable over subsequent days of TS exposure and was preserved across individuals. These data thus reveal a specific, temporally, and interindividually conserved repertoire of prefrontal tuning to behavioral responses under threat.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects often are willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No-info) vs a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%), and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory DREADDs in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), basolateral amygdala (BLA), or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of ACC destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that the previous choice strongly predicted preference in control animals, but not in female rats following ACC inhibition. The results reveal a causal, sex-dependent role for ACC in decisions involving information.

Deep-brain optical recording of neural dynamics during behavior

In vivo fluorescence recording techniques have produced landmark discoveries in neuroscience, providing insight into how single cell and circuit-level computations mediate sensory processing and generate complex behaviors. While much attention has been given to recording from cortical brain regions, deep-brain fluorescence recording is more complex because it requires additional measures to gain optical access to harder to reach brain nuclei. Here we discuss detailed considerations and tradeoffs regarding deep-brain fluorescence recording techniques and provide a comprehensive guide for all major steps involved, from project planning to data analysis. The goal is to impart guidance for new and experienced investigators seeking to use in vivo deep fluorescence optical recordings in awake, behaving rodent models.

Meta-reinforcement learning via orbitofrontal cortex

The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new environments. It has been hypothesized that the prefrontal cortex may mediate meta-RL in the brain, but the evidence is scarce. Here we show that the orbitofrontal cortex (OFC) mediates meta-RL. We trained mice and deep RL models on a probabilistic reversal learning task across sessions during which they improved their trial-by-trial RL policy through meta-learning. Ca2+/calmodulin-dependent protein kinase II-dependent synaptic plasticity in OFC was necessary for this meta-learning but not for the within-session trial-by-trial RL in experts. After meta-learning, OFC activity robustly encoded value signals, and OFC inactivation impaired the RL behaviors. Longitudinal tracking of OFC activity revealed that meta-learning gradually shapes population value coding to guide the ongoing behavioral policy. Our results indicate that two distinct RL algorithms with distinct neural mechanisms and timescales coexist in OFC to support adaptive decision-making.

Neurocircuits for motivation

The nervous system coordinates various motivated behaviors such as feeding, drinking, and escape to promote survival and evolutionary fitness. Although the precise behavioral repertoires required for distinct motivated behaviors are diverse, common features such as approach or avoidance suggest that common brain substrates are required for a wide range of motivated behaviors. In this Review, I describe a framework by which neural circuits specified for some innate drives regulate the activity of ventral tegmental area (VTA) dopamine neurons to reinforce ongoing or planned actions to fulfill motivational demands. This framework may explain why signaling from VTA dopamine neurons is ubiquitously involved in many types of diverse volitional motivated actions, as well as how sensory and interoceptive cues can initiate specific goal-directed actions.

Neural signatures of stress susceptibility and resilience in the amygdala-hippocampal network

The neural dynamics that underlie divergent anhedonic responses to stress remain unclear. Here, we identified neuronal dynamics in an amygdala-hippocampal circuit that distinguish stress resilience and susceptibility. In a reward-choice task, basolateral amygdala (BLA) activity in resilient mice showed enhanced discrimination of upcoming reward choices. In contrast, a rumination-like signature emerged in the BLA of susceptible mice; a linear decoder could classify the intention to switch or stay on a previously chosen reward. Spontaneous activity in the BLA of susceptible mice was higher dimensional than controls, reflecting the exploration of a larger number of distinct neural states. Manipulation of vCA1-BLA inputs rescued dysfunctional neural dynamics and anhedonia in susceptible mice, suggesting that targeting this pathway can enhance BLA circuit function and ameliorate of depression-related behaviors.

Dissociable contributions of basolateral amygdala and ventrolateral orbitofrontal cortex to flexible learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Learning to Express Reward Prediction Error-like Dopaminergic Activity Requires Plastic Representations of Time

The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference (TD) reinforcement learning. The TD framework predicts that some neuronal elements should represent the reward prediction error (RPE), which means they signal the difference between the expected future rewards and the actual rewards. The prominence of the TD theory arises from the observation that firing properties of dopaminergic neurons in the ventral tegmental area appear similar to those of RPE model-neurons in TD learning. Previous implementations of TD learning assume a fixed temporal basis for each stimulus that might eventually predict a reward. Here we show that such a fixed temporal basis is implausible and that certain predictions of TD learning are inconsistent with experiments. We propose instead an alternative theoretical framework, coined FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, feature specific representations of time are learned, allowing for neural representations of stimuli to adjust their timing and relation to rewards in an online manner. In FLEX dopamine acts as an instructive signal which helps build temporal models of the environment. FLEX is a general theoretical framework that has many possible biophysical implementations. In order to show that FLEX is a feasible approach, we present a specific biophysically plausible model which implements the principles of FLEX. We show that this implementation can account for various reinforcement learning paradigms, and that its results and predictions are consistent with a preponderance of both existing and reanalyzed experimental data.

Orbitofrontal cortex control of striatum leads economic decision-making

Animals must continually evaluate stimuli in their environment to decide which opportunities to pursue, and in many cases these decisions can be understood in fundamentally economic terms. Although several brain regions have been individually implicated in these processes, the brain-wide mechanisms relating these regions in decision-making are unclear. Using an economic decision-making task adapted for rats, we find that neural activity in both of two connected brain regions, the ventrolateral orbitofrontal cortex (OFC) and the dorsomedial striatum (DMS), was required for economic decision-making. Relevant neural activity in both brain regions was strikingly similar, dominated by the spatial features of the decision-making process. However, the neural encoding of choice direction in OFC preceded that of DMS, and this temporal relationship was strongly correlated with choice accuracy. Furthermore, activity specifically in the OFC projection to the DMS was required for appropriate economic decision-making. These results demonstrate that choice information in the OFC is relayed to the DMS to lead accurate economic decision-making.

Classification of temporal lobe epilepsy based on neuropsychological tests and exploration of its underlying neurobiology

Objective

To assist improving long-term postoperative seizure-free rate, we aimed to use machine learning algorithms based on neuropsychological data to differentiate temporal lobe epilepsy (TLE) from extratemporal lobe epilepsy (extraTLE), as well as explore the relationship between magnetic resonance imaging (MRI) and neuropsychological tests.

Methods

Twenty-three patients with TLE and 23 patients with extraTLE underwent neuropsychological tests and MRI scans before surgery. The least absolute shrinkage and selection operator were firstly employed for feature selection, and a machine learning approach with neuropsychological tests was employed to classify TLE using leave-one-out cross-validation. A generalized linear model was used to analyze the relationship between brain alterations and neuropsychological tests.

Results

We found that logistic regression with the selected neuropsychological tests generated classification accuracies of 87.0%, with an area under the receiver operating characteristic curve (AUC) of 0.89. Three neuropsychological tests were acquired as significant neuropsychological signatures for the diagnosis of TLE. We also found that the Right-Left Orientation Test difference was related to the superior temporal and the banks of the superior temporal sulcus (bankssts). The Conditional Association Learning Test (CALT) was associated with the cortical thickness difference in the lateral orbitofrontal area between the two groups, and the Component Verbal Fluency Test was associated with the cortical thickness difference in the lateral occipital cortex between the two groups.

Conclusion

These results showed that machine learning-based classification with the selected neuropsychological data can successfully classify TLE with high accuracy compared to previous studies, which could provide kind of warning sign for surgery candidate of TLE patients. In addition, understanding the mechanism of cognitive behavior by neuroimaging information could assist doctors in the presurgical evaluation of TLE.

Whole-brain mapping of long-range inputs to the VIP-expressing inhibitory neurons in the primary motor cortex

The primary motor cortex (MOp) is an important site for motor skill learning. Interestingly, neurons in MOp possess reward-related activity, presumably to facilitate reward-based motor learning. While pyramidal neurons (PNs) and different subtypes of GABAergic inhibitory interneurons (INs) in MOp all undergo cell-type specific plastic changes during motor learning, the vasoactive intestinal peptide-expressing inhibitory interneurons (VIP-INs) in MOp have been shown to preferentially respond to reward and play a critical role in the early phases of motor learning by triggering local circuit plasticity. To understand how VIP-INs might integrate various streams of information, such as sensory, pre-motor, and reward-related inputs, to regulate local plasticity in MOp, we performed monosynaptic rabies tracing experiments and employed an automated cell counting pipeline to generate a comprehensive map of brain-wide inputs to VIP-INs in MOp. We then compared this input profile to the brain-wide inputs to somatostatin-expressing inhibitory interneurons (SST-INs) and parvalbumin-expressing inhibitory interneurons (PV-INs) in MOp. We found that while all cell types received major inputs from sensory, motor, and prefrontal cortical regions, as well as from various thalamic nuclei, VIP-INs received more inputs from the orbital frontal cortex (ORB) - a region associated with reinforcement learning and value predictions. Our findings provide insight on how the brain leverages microcircuit motifs by both integrating and partitioning different streams of long-range input to modulate local circuit activity and plasticity.

Neural dynamics underlying associative learning in the dorsal and ventral hippocampus

Animals associate cues with outcomes and update these associations as new information is presented. This requires the hippocampus, yet how hippocampal neurons track changes in cue-outcome associations remains unclear. Using two-photon calcium imaging, we tracked the same dCA1 and vCA1 neurons across days to determine how responses evolve across phases of odor-outcome learning. Initially, odors elicited robust responses in dCA1, whereas, in vCA1, odor responses primarily emerged after learning and embedded information about the paired outcome. Population activity in both regions rapidly reorganized with learning and then stabilized, storing learned odor representations for days, even after extinction or pairing with a different outcome. Additionally, we found stable, robust signals across CA1 when mice anticipated outcomes under behavioral control but not when mice anticipated an inescapable aversive outcome. These results show how the hippocampus encodes, stores and updates learned associations and illuminates the unique contributions of dorsal and ventral hippocampus.

Few-shot learning: temporal scaling in behavioral and dopaminergic learning

How do we learn associations in the world (e.g., between cues and rewards)? Cue-reward associative learning is controlled in the brain by mesolimbic dopamine1-4. It is widely believed that dopamine drives such learning by conveying a reward prediction error (RPE) in accordance with temporal difference reinforcement learning (TDRL) algorithms5. TDRL implementations are "trial-based": learning progresses sequentially across individual cue-outcome experiences. Accordingly, a foundational assumption-often considered a mere truism-is that the more cue-reward pairings one experiences, the more one learns this association. Here, we disprove this assumption, thereby falsifying a foundational principle of trial-based learning algorithms. Specifically, when a group of head-fixed mice received ten times fewer experiences over the same total time as another, a single experience produced as much learning as ten experiences in the other group. This quantitative scaling also holds for mesolimbic dopaminergic learning, with the increase in learning rate being so high that the group with fewer experiences exhibits dopaminergic learning in as few as four cue-reward experiences and behavioral learning in nine. An algorithm implementing reward-triggered retrospective learning explains these findings. The temporal scaling and few-shot learning observed here fundamentally changes our understanding of the neural algorithms of associative learning.

Medial orbitofrontal cortical regulation of different aspects of Pavlovian and instrumental reward seeking

RATIONALE

The medial subregion of the orbitofrontal cortex (mOFC) is thought to play an important role representing the expected outcome of a given course of action, as lesioning or inactivating this cortical region results in the adoption of choice strategies based more on observable (rather than previously learned) information. Despite this, its role in mediating basic associative learning remains to be fully clarified.

OBJECTIVE

The present series of experiments examined the role of the mOFC in (1) Pavlovian conditioned approach, (2) conditioned reinforcement, (3) extinction, and (4) cue-induced reinstatement of food-seeking behavior.

METHODS

Separate cohorts of rats went through Pavlovian or instrumental training. Intra-mOFC infusions of either saline or GABA agonists (to temporarily inactivate neural activity) were given prior to Pavlovian approach, conditioned reinforcement, first or second day of instrumental extinction training, or cue-induced reinstatement test days.

RESULTS

mOFC inactivation increased lever-CS contacts in Pavlovian conditioned approach and (2) had no effect on conditioned reinforcement. These manipulations (3) accelerated within-session instrumental extinction during the initial extinction session, but impaired subsequent extinction learning on drug-free days. (4) mOFC inactivation induced differential effects on reinstatement that depended on baseline performance. mOFC inactivation abolished reinstatement in "Reinstater" rats (who displayed robust responding under control conditions) and robustly increased reinstatement in "Non-Reinstater" rats (who showed little reinstatement under control conditions) suggesting that individual differences in reinstatement may be supported by differences in mOFC mediated representations of expected outcomes.

CONCLUSIONS

These findings have important implications for understanding how the mOFC uses stimulus-outcome and action-outcome expectancies to guide behavior, and how dysfunction within this region may contribute to pathological patterns of reward seeking.

Foundations of a temporal RL

Recent advances in neuroscience and psychology show that the brain has access to timelines of both the past and the future. Spiking across populations of neurons in many regions of the mammalian brain maintains a robust temporal memory, a neural timeline of the recent past. Behavioral results demonstrate that people can estimate an extended temporal model of the future, suggesting that the neural timeline of the past could extend through the present into the future. This paper presents a mathematical framework for learning and expressing relationships between events in continuous time. We assume that the brain has access to a temporal memory in the form of the real Laplace transform of the recent past. Hebbian associations with a diversity of synaptic time scales are formed between the past and the present that record the temporal relationships between events. Knowing the temporal relationships between the past and the present allows one to predict relationships between the present and the future, thus constructing an extended temporal prediction for the future. Both memory for the past and the predicted future are represented as the real Laplace transform, expressed as the firing rate over populations of neurons indexed by different rate constants s. The diversity of synaptic timescales allows for a temporal record over the much larger time scale of trial history. In this framework, temporal credit assignment can be assessed via a Laplace temporal difference. The Laplace temporal difference compares the future that actually follows a stimulus to the future predicted just before the stimulus was observed. This computational framework makes a number of specific neurophysiological predictions and, taken together, could provide the basis for a future iteration of RL that incorporates temporal memory as a fundamental building block.

Transformation of acoustic information to sensory decision variables in the parietal cortex

The process by which sensory evidence contributes to perceptual choices requires an understanding of its transformation into decision variables. Here, we address this issue by evaluating the neural representation of acoustic information in the auditory cortex-recipient parietal cortex, while gerbils either performed a two-alternative forced-choice auditory discrimination task or while they passively listened to identical acoustic stimuli. During task engagement, stimulus identity decoding performance from simultaneously recorded parietal neurons significantly correlated with psychometric sensitivity. In contrast, decoding performance during passive listening was significantly reduced. Principal component and geometric analyses revealed the emergence of low-dimensional encoding of linearly separable manifolds with respect to stimulus identity and decision, but only during task engagement. These findings confirm that the parietal cortex mediates a transition of acoustic representations into decision-related variables. Finally, using a clustering analysis, we identified three functionally distinct subpopulations of neurons that each encoded task-relevant information during separate temporal segments of a trial. Taken together, our findings demonstrate how parietal cortex neurons integrate and transform encoded auditory information to guide sound-driven perceptual decisions.

Global Instance Tracking: Locating Target More Like Humans

Target tracking, the essential ability of the human visual system, has been simulated by computer vision tasks. However, existing trackers perform well in austere experimental environments but fail in challenges like occlusion and fast motion. The massive gap indicates that researches only measure tracking performance rather than intelligence. How to scientifically judge the intelligence level of trackers? Distinct from decision-making problems, lacking three requirements (a challenging task, a fair environment, and a scientific evaluation procedure) makes it strenuous to answer the question. In this article, we first propose the global instance tracking (GIT) task, which is supposed to search an arbitrary user-specified instance in a video without any assumptions about camera or motion consistency, to model the human visual tracking ability. Whereafter, we construct a high-quality and large-scale benchmark VideoCube to create a challenging environment. Finally, we design a scientific evaluation procedure using human capabilities as the baseline to judge tracking intelligence. Additionally, we provide an online platform with toolkit and an updated leaderboard. Although the experimental results indicate a definite gap between trackers and humans, we expect to take a step forward to generate authentic human-like trackers. The database, toolkit, evaluation server, and baseline results are available at http://videocube.aitestunion.com.

Bringing Bayes and Shannon to the Study of Behavioural and Neurobiological Timing and Associative Learning

Bayesian parameter estimation and Shannon’s theory of information provide tools for analysing and understanding data from behavioural and neurobiological experiments on interval timing—and from experiments on Pavlovian and operant conditioning, because timing plays a fundamental role in associative learning. In this tutorial, we explain basic concepts behind these tools and show how to apply them to estimating, on a trial-by-trial, reinforcement-by-reinforcement and response-by-response basis, important parameters of timing behaviour and of the neurobiological manifestations of timing in the brain. These tools enable quantification of relevant variables in the trade-off between acting as an ideal observer should act and acting as an ideal agent should act, which is also known as the trade-off between exploration (information gathering) and exploitation (information utilization) in reinforcement learning. They enable comparing the strength of the evidence for a measurable association to the strength of the behavioural evidence that the association has been perceived. A GitHub site and an OSF site give public access to well-documented Matlab and Python code and to raw data to which these tools have been applied.

Mesolimbic dopamine release conveys causal associations

Learning to predict rewards based on environmental cues is essential for survival. It is believed that animals learn to predict rewards by updating predictions whenever the outcome deviates from expectations, and that such reward prediction errors (RPEs) are signaled by the mesolimbic dopamine system-a key controller of learning. However, instead of learning prospective predictions from RPEs, animals can infer predictions by learning the retrospective cause of rewards. Hence, whether mesolimbic dopamine instead conveys a causal associative signal that sometimes resembles RPE remains unknown. We developed an algorithm for retrospective causal learning and found that mesolimbic dopamine release conveys causal associations but not RPE, thereby challenging the dominant theory of reward learning. Our results reshape the conceptual and biological framework for associative learning.

Cognition and the single neuron: How cell types construct the dynamic computations of frontal cortex

Frontal cortex is thought to underlie many advanced cognitive capacities, from self-control to long term planning. Reflecting these diverse demands, frontal neural activity is notoriously idiosyncratic, with tuning properties that are correlated with endless numbers of behavioral and task features. This menagerie of tuning has made it difficult to extract organizing principles that govern frontal neural activity. Here, we contrast two successful yet seemingly incompatible approaches that have begun to address this challenge. Inspired by the indecipherability of single-neuron tuning, the first approach casts frontal computations as dynamical trajectories traversed by arbitrary mixtures of neurons. The second approach, by contrast, attempts to explain the functional diversity of frontal activity with the biological diversity of cortical cell-types. Motivated by the recent discovery of functional clusters in frontal neurons, we propose a consilience between these population and cell-type-specific approaches to neural computations, advancing the conjecture that evolutionarily inherited cell-type constraints create the scaffold within which frontal population dynamics must operate.

An opioid-gated thalamoaccumbal circuit for the suppression of reward seeking in mice

Suppression of dangerous or inappropriate reward-motivated behaviors is critical for survival, whereas therapeutic or recreational opioid use can unleash detrimental behavioral actions and addiction. Nevertheless, the neuronal systems that suppress maladaptive motivated behaviors remain unclear, and whether opioids disengage those systems is unknown. In a mouse model using two-photon calcium imaging in vivo, we identify paraventricular thalamostriatal neuronal ensembles that are inhibited upon sucrose self-administration and seeking, yet these neurons are tonically active when behavior is suppressed by a fear-provoking predator odor, a pharmacological stressor, or inhibitory learning. Electrophysiological, optogenetic, and chemogenetic experiments reveal that thalamostriatal neurons innervate accumbal parvalbumin interneurons through synapses enriched with calcium permeable AMPA receptors, and activity within this circuit is necessary and sufficient for the suppression of sucrose seeking regardless of the behavioral suppressor administered. Furthermore, systemic or intra-accumbal opioid injections rapidly dysregulate thalamostriatal ensemble dynamics, weaken thalamostriatal synaptic innervation of downstream neurons, and unleash reward-seeking behaviors in a manner that is reversed by genetic deletion of thalamic µ-opioid receptors. Overall, our findings reveal a thalamostriatal to parvalbumin interneuron circuit that is both required for the suppression of reward seeking and rapidly disengaged by opioids.

Recent Advances at the Interface of Neuroscience and Artificial Neural Networks

Biological neural networks adapt and learn in diverse behavioral contexts. Artificial neural networks (ANNs) have exploited biological properties to solve complex problems. However, despite their effectiveness for specific tasks, ANNs are yet to realize the flexibility and adaptability of biological cognition. This review highlights recent advances in computational and experimental research to advance our understanding of biological and artificial intelligence. In particular, we discuss critical mechanisms from the cellular, systems, and cognitive neuroscience fields that have contributed to refining the architecture and training algorithms of ANNs. Additionally, we discuss how recent work used ANNs to understand complex neuronal correlates of cognition and to process high throughput behavioral data.

Orbitofrontal cortex populations are differentially recruited to support actions

The ability to use information from one's prior actions is necessary for decision-making. While orbitofrontal cortex (OFC) has been hypothesized as key for inferences made using cue and value-related information, whether OFC populations contribute to the use of information from volitional actions to guide behavior is not clear. Here, we used a self-paced lever-press hold-down task in which mice infer prior lever-press durations to guide subsequent action performance. We show that the activity of genetically identified lateral OFC (lOFC) subpopulations differentially instantiate current and prior action information during ongoing action execution. Transient state-dependent lOFC circuit disruptions of specified subpopulations reduced the encoding of ongoing press durations but did not disrupt the use of prior action information to guide future action performance. In contrast, a chronic functional loss of lOFC circuit activity resulted in increased reliance on recently executed lever-press durations and impaired contingency reversal, suggesting the recruitment of compensatory mechanisms that resulted in repetitive action control. Our results identify a novel role for lOFC in the integration of action information to guide adaptive behavior.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

A molecularly integrated amygdalo-fronto-striatal network coordinates flexible learning and memory

Behavioral flexibility-that is, the ability to deviate from established behavioral sequences-is critical for navigating dynamic environments and requires the durable encoding and retrieval of new memories to guide future choice. The orbitofrontal cortex (OFC) supports outcome-guided behaviors. However, the coordinated neural circuitry and cellular mechanisms by which OFC connections sustain flexible learning and memory remain elusive. Here we demonstrate in mice that basolateral amygdala (BLA)→OFC projections bidirectionally control memory formation when familiar behaviors are unexpectedly not rewarded, whereas OFC→dorsomedial striatum (DMS) projections facilitate memory retrieval. OFC neuronal ensembles store a memory trace for newly learned information, which appears to be facilitated by circuit-specific dendritic spine plasticity and neurotrophin signaling within defined BLA-OFC-DMS connections and obstructed by cocaine. Thus, we describe the directional transmission of information within an integrated amygdalo-fronto-striatal circuit across time, whereby novel memories are encoded by BLA→OFC inputs, represented within OFC ensembles and retrieved via OFC→DMS outputs during future choice.

Looking to the stars for answers: Strategies for determining how astrocytes influence neuronal activity

Astrocytes are critical components of neural circuits positioned in close proximity to the synapse, allowing them to rapidly sense and respond to neuronal activity. One repeatedly observed biomarker of astroglial activation is an increase in intracellular Ca²⁺ levels. These astroglial Ca²⁺ signals are often observed spreading throughout various cellular compartments from perisynaptic astroglial processes, to major astrocytic branches and on to the soma or cell body. Here we review recent evidence demonstrating that astrocytic Ca²⁺ events are remarkably heterogeneous in both form and function, propagate through the astroglial syncytia, and are directly linked to the ability of astroglia to influence local neuronal activity. As many of the cellular functions of astroglia can be linked to intracellular Ca²⁺ signaling, and the diversity and heterogeneity of these events becomes more apparent, there is an increasing need for novel experimental strategies designed to better understand the how these signals evolve in parallel with neuronal activity. Here we review the recent advances that enable the characterization of both subcellular and population-wide astrocytic Ca²⁺ dynamics. Additionally, we also outline the experimental design required for simultaneous in vivo Ca²⁺ imaging in the context of neuronal or astroglial manipulation, highlighting new experimental strategies made possible by recent advances in viral vector, imaging, and quantification technologies. Through combined usage of these reagents and methodologies, we provide a conceptual framework to study how astrocytes functionally integrate into neural circuits and to what extent they influence and direct the synaptic activity underlying behavioral responses.

Drug versus non-drug behaviors: A dual-reward model of sex differences and neurobiological mechanisms in rats

Substance Use Disorders (SUDs) are an impactful problem characterized by chronic relapse and engagement in drug-related behaviors at the expense of non-drug behaviors. Brain regions implicated in drug and non-drug-related behaviors often overlap, complicating investigations of neurobiological mechanisms underlying SUDs. Here we presented a within-subject model for studying self-administration, reinforcer competition, extinction, and cued reinstatement of cocaine- and food-seeking in rats. Due to differences in cocaine- and food-reinforced behavior, we transformed data to proportions of baseline, revealing increased resistance to extinction and disproportionately greater cued reinstatement of cocaine seeking relative to food seeking. Consistent with previous reports, females showed greater preference for cocaine reinforcement than males, though these findings failed to reach statistical significance. To demonstrate the model's utility for investigating neurobiological mechanisms, we included proof-of-concept calcium imaging data demonstrating the utility of the behavioral model for detecting cellular activity patterns associated with cocaine- and food-seeking behaviors. Future studies utilizing this model should improve understanding of the development and expression of pathological behaviors characteristic of SUDs in humans, sex differences in these behaviors, and their neurobiological correlates. Thus, the model has utility for improving understanding of SUDs, leading to novel treatments to reduce the pathological behaviors associated with SUDs.

Developments from Bulk Optogenetics to Single-Cell Strategies to Dissect the Neural Circuits that Underlie Aberrant Motivational States

Motivational states are regulated by complex networks across brain regions that are composed of genetically and functionally distinct neuronal populations. Disruption within these neural circuits leads to aberrant motivational states and are thought to be the root cause of psychiatric disorders related to reward processing and addiction. Critical technological advances in the field have revolutionized the study of neural systems by allowing the use of optical strategies to precisely control and visualize neural activity within genetically identified neural populations in the brain. This review will provide a brief introduction into the history of how technological advances in single-cell strategies have been applied to elucidate the neural circuits that underlie aberrant motivational states that often lead to dysfunction in reward processing and addiction.

Visual association cortex links cues with conjunctions of reward and locomotor contexts

Postrhinal cortex (POR) and neighboring lateral visual association areas are necessary for identifying objects and interpreting them in specific contexts, but how POR neurons encode the same object across contexts remains unclear. Here, we imaged excitatory neurons in mouse POR across tens of days prior to and throughout initial cue-reward learning and reversal learning. We assessed responses to the same cue when it was rewarded or unrewarded, during both locomotor and stationary contexts. Surprisingly, a large class of POR neurons were minimally cue-driven prior to learning. After learning, distinct clusters within this class responded selectively to a given cue when presented in a specific conjunction of reward and locomotion contexts. In addition, another class contained clusters of neurons whose cue responses were more transient, insensitive to reward learning, and adapted over thousands of presentations. These two classes of POR neurons may support context-dependent interpretation and context-independent identification of sensory cues.

Comparison of prefrontal cortex sucrose seeking ensembles engaged in multiple seeking sessions: Context is key

Encoding of memories, including those associated with prior drug or reward, is thought to take place within distinct populations of neurons, termed ensembles. Neuronal ensembles for drug- and reward-seeking have been identified in regions of the medial prefrontal cortex, but much of our understanding of these ensembles is based on experiments that take place in a single reward-associated environment and measure ensemble encoding over short durations of time. In contrast, reward seeking behavior is evident across different reward-associated environments and persists over time. Using TetTag mice and Fos immunohistochemistry, we examined the relationship between persistent sucrose-seeking and ensemble encoding in mice that undergo seeking sessions in the same or different sucrose self-administration contexts 2 weeks apart. We found that prelimbic (PrL) and anterior cingulate cortex ensembles tagged in the first seeking session were highly sensitive to the context in which a second seeking session took place: reactivation of these ensembles was reduced in the same context but elevated in a distinct sucrose self-administration context. Correlational analyses revealed that ensemble reactivation in the PrL was proportional to the persistence of sucrose seeking behavior across sessions in differing ways in female mice. In the same context, reactivation was proportional to the persistence of non-reinforced operant responses, whereas in a distinct context, reactivation was proportional to the persistence of non-reinforced head entries into the sucrose receptacle. This study underlines the importance of the medial prefrontal cortex importance in maintaining a reward-seeking ensemble over time and identifies context-dependent changes in behavioral correlates of ensemble reactivation.

Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate

Learning to predict rewards is essential for the sustained fitness of animals. Contemporary views suggest that such learning is driven by a reward prediction error (RPE)-the difference between received and predicted rewards. The magnitude of learning induced by an RPE is proportional to the product of the RPE and a learning rate. Here we demonstrate using two-photon calcium imaging and optogenetics in mice that certain functionally distinct subpopulations of ventral/medial orbitofrontal cortex (vmOFC) neurons signal learning rate control. Consistent with learning rate control, trial-by-trial fluctuations in vmOFC activity positively correlate with behavioral updating when the RPE is positive, and negatively correlates with behavioral updating when the RPE is negative. Learning rate is affected by many variables including the salience of a reward. We found that the average reward response of these neurons signals the relative salience of a reward, because it decreases after reward prediction learning or the introduction of another highly salient aversive stimulus. The relative salience signaling in vmOFC is sculpted by medial thalamic inputs. These results support emerging theoretical views that prefrontal cortex encodes and controls learning parameters.