Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate

Whole-brain Mapping of Inputs and Outputs of Specific Orbitofrontal Cortical Neurons in Mice

The orbitofrontal cortex (ORB), a region crucial for stimulus-reward association, decision-making, and flexible behaviors, extensively connects with other brain areas. However, brain-wide inputs to projection-defined ORB neurons and the distribution of inhibitory neurons postsynaptic to neurons in specific ORB subregions remain poorly characterized. Here we mapped the inputs of five types of projection-specific ORB neurons and ORB outputs to two types of inhibitory neurons. We found that different projection-defined ORB neurons received inputs from similar cortical and thalamic regions, albeit with quantitative variations, particularly in somatomotor areas and medial groups of the dorsal thalamus. By counting parvalbumin (PV) or somatostatin (SST) interneurons innervated by neurons in specific ORB subregions, we found a higher fraction of PV neurons in sensory cortices and a higher fraction of SST neurons in subcortical regions targeted by medial ORB neurons. These results provide insights into understanding and investigating the function of specific ORB neurons.

An open-source behavior controller for associative learning and memory (B-CALM)

Associative learning and memory, i.e., learning and remembering the associations between environmental stimuli, self-generated actions, and outcomes such as rewards or punishments, are critical for the well-being of animals. Hence, the neural mechanisms underlying these processes are extensively studied using behavioral tasks in laboratory animals. Traditionally, these tasks have been controlled using commercial hardware and software, which limits scalability and accessibility due to their cost. More recently, due to the revolution in microcontrollers or microcomputers, several general-purpose and open-source solutions have been advanced for controlling neuroscientific behavioral tasks. While these solutions have great strength due to their flexibility and general-purpose nature, for the same reasons, they suffer from some disadvantages including the need for considerable programming expertise, limited online visualization, or slower than optimal response latencies for any specific task. Here, to mitigate these concerns, we present an open-source behavior controller for associative learning and memory (B-CALM). B-CALM provides an integrated suite that can control a host of associative learning and memory behaviors. As proof of principle for its applicability, we show data from head-fixed mice learning Pavlovian conditioning, operant conditioning, discrimination learning, as well as a timing task and a choice task. These can be run directly from a user-friendly graphical user interface (GUI) written in MATLAB that controls many independently running Arduino Mega microcontrollers in parallel (one per behavior box). In sum, B-CALM will enable researchers to execute a wide variety of associative learning and memory tasks in a scalable, accurate, and user-friendly manner.

Holographic stimulation of opposing amygdala ensembles bidirectionally modulates valence-specific behavior via mutual inhibition

The basolateral amygdala (BLA) is an evolutionarily conserved brain region, well known for valence processing. Despite this central role, the relationship between activity of BLA neuronal ensembles in response to appetitive and aversive stimuli and the subsequent expression of valence-specific behavior has remained elusive. Here, we leverage two-photon calcium imaging combined with single-cell holographic photostimulation through an endoscopic lens to demonstrate a direct causal role for opposing ensembles of BLA neurons in the control of oppositely valenced behavior in mice. We report that targeted photostimulation of either appetitive or aversive BLA ensembles results in mutual inhibition and shifts behavioral responses to promote consumption of an aversive tastant or reduce consumption of an appetitive tastant, respectively. Here, we identify that neuronal encoding of valence in the BLA is graded and relies on the relative proportion of individual BLA neurons recruited in a stable appetitive or quinine ensemble.

Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including the highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, the unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory designer receptors exclusively activated by designer drugs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-/post-reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of the ventrolateral orbitofrontal cortex (vlOFC), but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Postnatal Phencyclidine-Induced Deficits in Decision Making Are Ameliorated by Optogenetic Inhibition of Ventromedial Orbitofrontal Cortical Glutamate Neurons

Background

The orbitofrontal cortex (OFC) is essential for decision making, and functional disruptions within the OFC are evident in schizophrenia. Postnatal phencyclidine (PCP) administration in rats is a neurodevelopmental manipulation that induces schizophrenia-relevant cognitive impairments. We aimed to determine whether manipulating OFC glutamate cell activity could ameliorate postnatal PCP-induced deficits in decision making.

Methods

Male and female Wistar rats (n = 110) were administered saline or PCP on postnatal days 7, 9, and 11. In adulthood, we expressed YFP (yellow fluorescent protein) (control), ChR2 (channelrhodopsin-2) (activation), or eNpHR 3.0 (enhanced halorhodopsin) (inhibition) in glutamate neurons within the ventromedial OFC (vmOFC). Rats were tested on the probabilistic reversal learning task once daily for 20 days while we manipulated the activity of vmOFC glutamate cells. Behavioral performance was analyzed using a Q-learning computational model of reinforcement learning.

Results

Compared with saline-treated rats expressing YFP, PCP-treated rats expressing YFP completed fewer reversals, made fewer win-stay responses, and had lower learning rates. We induced similar performance impairments in saline-treated rats by activating vmOFC glutamate cells (ChR2). Strikingly, PCP-induced performance deficits were ameliorated when the activity of vmOFC glutamate cells was inhibited (halorhodopsin).

Conclusions

Postnatal PCP-induced deficits in decision making are associated with hyperactivity of vmOFC glutamate cells. Thus, normalizing vmOFC activity may represent a potential therapeutic target for decision-making deficits in patients with schizophrenia.

Neurocircuits for motivation

The nervous system coordinates various motivated behaviors such as feeding, drinking, and escape to promote survival and evolutionary fitness. Although the precise behavioral repertoires required for distinct motivated behaviors are diverse, common features such as approach or avoidance suggest that common brain substrates are required for a wide range of motivated behaviors. In this Review, I describe a framework by which neural circuits specified for some innate drives regulate the activity of ventral tegmental area (VTA) dopamine neurons to reinforce ongoing or planned actions to fulfill motivational demands. This framework may explain why signaling from VTA dopamine neurons is ubiquitously involved in many types of diverse volitional motivated actions, as well as how sensory and interoceptive cues can initiate specific goal-directed actions.

Dissociable contributions of basolateral amygdala and ventrolateral orbitofrontal cortex to flexible learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Orbitofrontal cortex conveys stimulus and task information to the auditory cortex

Auditory cortical neurons modify their response profiles in response to numerous external factors. During task performance, changes in primary auditory cortex (A1) responses are thought to be driven by top-down inputs from the orbitofrontal cortex (OFC), which may lead to response modification on a trial-by-trial basis. While OFC neurons respond to auditory stimuli and project to A1, the function of OFC projections to A1 during auditory tasks is unknown. Here, we observed the activity of putative OFC terminals in A1 in mice by using in vivo two-photon calcium imaging of OFC terminals under passive conditions and during a tone detection task. We found that behavioral activity modulates but is not necessary to evoke OFC terminal responses in A1. OFC terminals in A1 form distinct populations that exclusively respond to either the tone, reward, or error. Using tones against a background of white noise, we found that OFC terminal activity was modulated by the signal-to-noise ratio (SNR) in both the passive and active conditions and that OFC terminal activity varied with SNR, and thus task difficulty in the active condition. Therefore, OFC projections in A1 are heterogeneous in their modulation of auditory encoding and likely contribute to auditory processing under various auditory conditions.

Valence and salience encoding by parallel circuits from the paraventricular thalamus to the nucleus accumbens

The anterior and posterior subregions of the paraventricular thalamus (aPVT and pPVT, respectively) play unique roles in learned behaviors, from fear conditioning to alcohol/drug intake, potentially through differentially organized projections to limbic brain regions including the nucleus accumbens medial shell (mNAcSh). Here, we found that the aPVT projects broadly to the mNAcSh and that the aPVT-mNAcSh circuit encodes positive valence, such that in vivo manipulations of the circuit modulated both innately programmed and learned behavioral responses to positively and negatively valenced stimuli, particularly in females. Further, the endogenous activity of aPVT presynaptic terminals in the mNAcSh was greater in response to positively than negatively valenced stimuli, and the probability of synaptic glutamate release from aPVT neurons in the mNAcSh was higher in females than males. In contrast, we found that the pPVT-mNAcSh circuit encodes stimulus salience regardless of valence. While pPVT-mNAcSh circuit inhibition suppressed behavioral responses in both sexes, circuit activation increased behavioral responses to stimuli only in males. Our results point to circuit-specific stimulus feature encoding by parallel PVT-mNAcSh circuits that have sex-dependent biases in organization and function.

Mesolimbic dopamine release conveys causal associations

Learning to predict rewards based on environmental cues is essential for survival. It is believed that animals learn to predict rewards by updating predictions whenever the outcome deviates from expectations, and that such reward prediction errors (RPEs) are signaled by the mesolimbic dopamine system-a key controller of learning. However, instead of learning prospective predictions from RPEs, animals can infer predictions by learning the retrospective cause of rewards. Hence, whether mesolimbic dopamine instead conveys a causal associative signal that sometimes resembles RPE remains unknown. We developed an algorithm for retrospective causal learning and found that mesolimbic dopamine release conveys causal associations but not RPE, thereby challenging the dominant theory of reward learning. Our results reshape the conceptual and biological framework for associative learning.

How do real animals account for the passage of time during associative learning?

Animals routinely learn to associate environmental stimuli and self-generated actions with their outcomes such as rewards. One of the most popular theoretical models of such learning is the reinforcement learning (RL) framework. The simplest form of RL, model-free RL, is widely applied to explain animal behavior in numerous neuroscientific studies. More complex RL versions assume that animals build and store an explicit model of the world in memory. To apply these approaches to explain animal behavior, typical neuroscientific RL models make implicit assumptions about how real animals represent the passage of time. In this perspective, I explicitly list these assumptions and show that they have several problematic implications. I hope that the explicit discussion of these problems encourages the field to seriously examine the assumptions underlying timing and reinforcement learning. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A zona incerta-basomedial amygdala circuit modulates aversive expectation in emotional stress-induced aversive learning deficits

A previously published study showed that stress may interfere with associative aversive learning and facilitate mood-related disorders. However, whether emotional stress alone affects aversive learning is unknown. Using three chamber-vicarious social defeat stress (3C-VSDS) model mice, we investigated the effect of emotional stress on aversive learning. An important origin of dopamine (DA) neurons, the zona incerta (ZI), is expected to be a novel target for the modulation of aversive learning. However, less is known about the circuit mechanism of ZI^DA neurons in aversive learning. Here, we subjected mice to a fear-conditioning system (FCS) and observed an increased calcium activity of ZI TH⁺ neurons in aversive expectation during the conditioning phase, especially during the late stage of the conditional stimulus (CS) when CS and unconditional stimulus (US) pairings were used. Optogenetic inhibition of ZI TH⁺ neurons at the late stage of CS disrupted conditioned fear learning in mice. We further identified a TH⁺ projection from the ZI to the basomedial amygdala (BMA) and found that optogenetic inhibition of the ZI-BMA circuit could also block aversive learning. Finally, we used 3C-VSDS mice as a model of emotional stress. We found that the 3C-VSDS model mice demonstrated reduced aversive expectation associated with ZI TH⁺ neurons in the late stage of CS and impaired aversive learning in FCS. Optogenetic activation of ZI-BMA TH⁺ projections in the late stage of CS significantly reversed the aversive FCS learning disability of 3C-VSDS model mice. These data suggest that a TH⁺ circuit from the ZI to the BMA is required for aversive expectation, both at baseline and in 3C-VSDS-induced aversive learning deficits and that this circuit is a potential target for the modulation of aversive learning. Low activity of ZI-BMA TH⁺ projections is one reason for 3C-VSDS-induced aversive learning deficits.

The learning of prospective and retrospective cognitive maps within neural circuits

Brain circuits are thought to form a "cognitive map" to process and store statistical relationships in the environment. A cognitive map is commonly defined as a mental representation that describes environmental states (i.e., variables or events) and the relationship between these states. This process is commonly conceptualized as a prospective process, as it is based on the relationships between states in chronological order (e.g., does reward follow a given state?). In this perspective, we expand this concept on the basis of recent findings to postulate that in addition to a prospective map, the brain forms and uses a retrospective cognitive map (e.g., does a given state precede reward?). In doing so, we demonstrate that many neural signals and behaviors (e.g., habits) that seem inflexible and non-cognitive can result from retrospective cognitive maps. Together, we present a significant conceptual reframing of the neurobiological study of associative learning, memory, and decision making.