Differential coding of uncertain reward in rat insular and orbitofrontal cortex

Insular and prelimbic cortices control behavioral accuracy and precision in a temporal decision-making task in rats

The anterior insular cortex (AIC) comprises a region of sensory integration. It appears to detect salient events in order to guide goal-directed behavior, code tracking errors, and estimate the passage of time. Temporal processing in the AIC may be instantiated by the integration of representations of interoception. Projections between the AIC and the medial prefrontal cortex (mPFC) - found both in rats and humans - also suggest a possible role for these structures in the integration of autonomic responses during ongoing behavior. Few studies, however, have investigated the role of AIC and mPFC in decision-making and time estimation tasks. Moreover, their findings are not consistent, so the relationship between temporal decision-making and those areas remains unclear. The present study employed bilateral inactivations to explore the role of AIC and prelimbic cortex (PL) in rats during a temporal decision-making task. In this task, two levers are available simultaneously (but only one is active), one predicting reinforcement after a short, and the other after a long-fixed interval. Optimal performance requires a switch from the short to the long lever after the short-fixed interval elapsed and no reinforcement was delivered. Switch behavior from the short to the long lever was dependent on AIC and PL. During AIC inactivation, switch latencies became more variable, while during PL inactivation switch latencies became both more variable and less accurate. These findings point to a dissociation between AIC and PL in temporal decision-making, suggesting that the AIC is important for temporal precision, and PL is important for both temporal accuracy and precision.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects are often willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No Info) vs. a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%) and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) in the anterior cingulate cortex, orbitofrontal cortex, basolateral amygdala, or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of the anterior cingulate cortex destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that previous choice predicted current choice in all conditions, however previously rewarded Info trials strongly predicted preference in all conditions except in female rats following anterior cingulate cortex inhibition. The results reveal a causal, sex-dependent role for the anterior cingulate cortex in decisions involving information.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information

Subjects often are willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No-info) vs a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%), and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory DREADDs in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), basolateral amygdala (BLA), or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of ACC destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that the previous choice strongly predicted preference in control animals, but not in female rats following ACC inhibition. The results reveal a causal, sex-dependent role for ACC in decisions involving information.

Emotion dynamics as hierarchical Bayesian inference in time

What fundamental property of our environment would be most valuable and optimal in characterizing the emotional dynamics we experience in daily life? Empirical work has shown that an accurate estimation of uncertainty is necessary for our optimal perception, learning, and decision-making. However, the role of this uncertainty in governing our affective dynamics remains unexplored. Using Bayesian encoding, decoding and computational modeling, on a large-scale neuroimaging and behavioral data on a passive movie-watching task, we showed that emotions naturally arise due to ongoing uncertainty estimations about future outcomes in a hierarchical neural architecture. Several prefrontal subregions hierarchically encoded a lower-dimensional signal that highly correlated with the evolving uncertainty. Crucially, the lateral orbitofrontal cortex (lOFC) tracked the temporal fluctuations of this uncertainty and was predictive of the participants' predisposition to anxiety. Furthermore, we observed a distinct functional double-dissociation within OFC with increased connectivity between medial OFC and DMN, while with that of lOFC and FPN in response to the evolving affect. Finally, we uncovered a temporally predictive code updating an individual's beliefs spontaneously with fluctuating outcome uncertainty in the lOFC. A biologically relevant and computationally crucial parameter in the theories of brain function, we propose uncertainty to be central to the definition of complex emotions.

Better living through understanding the insula: Why subregions can make all the difference

Insula function is considered critical for many motivated behaviors, with proposed functions ranging from attention, behavioral control, emotional regulation, goal-directed and aversion-resistant responding. Further, the insula is implicated in many neuropsychiatric conditions including substance abuse. More recently, multiple insula subregions have been distinguished based on anatomy, connectivity, and functional contributions. Generally, posterior insula is thought to encode more somatosensory inputs, which integrate with limbic/emotional information in middle insula, that in turn integrate with cognitive processes in anterior insula. Together, these regions provide rapid interoceptive information about the current or predicted situation, facilitating autonomic recruitment and quick, flexible action. Here, we seek to create a robust foundation from which to understand potential subregion differences, and provide direction for future studies. We address subregion differences across humans and rodents, so that the latter's mechanistic interventions can best mesh with clinical relevance of human conditions. We first consider the insula's suggested roles in humans, then compare subregional studies, and finally describe rodent work. One primary goal is to encourage precision in describing insula subregions, since imprecision (e.g. including both posterior and anterior studies when describing insula work) does a disservice to a larger understanding of insula contributions. Additionally, we note that specific task details can greatly impact recruitment of various subregions, requiring care and nuance in design and interpretation of studies. Nonetheless, the central ethological importance of the insula makes continued research to uncover mechanistic, mood, and behavioral contributions of paramount importance and interest. This article is part of the special Issue on 'Neurocircuitry Modulating Drug and Alcohol Abuse'.

Shared and unique neural circuitry underlying temporally unpredictable threat and reward processing

Temporally unpredictable stimuli influence behavior across species, as previously demonstrated for sequences of simple threats and rewards with fixed or variable onset. Neuroimaging studies have identified a specific frontolimbic circuit that may become engaged during the anticipation of temporally unpredictable threat (U-threat). However, the neural mechanisms underlying processing of temporally unpredictable reward (U-reward) are incompletely understood. It is also unclear whether these processes are mediated by overlapping or distinct neural systems. These knowledge gaps are noteworthy given that disruptions within these neural systems may lead to maladaptive response to uncertainty. Here, using functional magnetic resonance imaging data from a sample of 159 young adults, we showed that anticipation of both U-threat and U-reward elicited activation in the right anterior insula, right ventral anterior nucleus of the thalamus and right inferior frontal gyrus. U-threat also activated the right posterior insula and dorsal anterior cingulate cortex, relative to U-reward. In contrast, U-reward elicited activation in the right fusiform and left middle occipital gyrus, relative to U-threat. Although there is some overlap in the neural circuitry underlying anticipation of U-threat and U-reward, these processes appear to be largely mediated by distinct circuits. Future studies are needed to corroborate and extend these preliminary findings.

Prior Cocaine Exposure Increases Firing to Immediate Reward While Attenuating Cue and Context Signals Related to Reward Value in the Insula

The insula contributes to behavioral control and is disrupted by substance abuse, yet we know little about the neural signals underlying these functions or how they are disrupted after chronic drug self-administration. Here, male and female rats self-administered either cocaine (experimental group) or sucrose (control) for 12 consecutive days. After a 1 month withdrawal period, we recorded from insula while rats performed a previously learned reward-guided decision-making task. Cocaine-exposed rats were more sensitive to value manipulations and were faster to respond. These behavioral changes were accompanied by elevated counts of neurons in the insula that increased firing to reward. These neurons also fired more strongly at the start of long-delay trials, when a more immediate reward would be expected, and fired less strongly in anticipation of the actual delivery of delayed rewards. Although reward-related firing to immediate reward was enhanced after cocaine self-administration, reward-predicting cue and context signals were attenuated. In addition to revealing novel firing patterns unique to insula, our data suggest changes in such neural activity likely contribute to impaired decision making observed after drug use.SIGNIFICANCE STATEMENT The insula plays a clear role in drug addiction and drug-induced impairments of decision making, yet there is little understanding of its underlying neural signals. We found that chronic cocaine self-administration reduces cue and context encoding in insula while enhancing signals related to immediate reward. These changes in neural activity likely contribute to impaired decision making and impulsivity observed after drug use.

Modulation of Dopamine for Adaptive Learning: A Neurocomputational Model

There have been many proposals that learning rates in the brain are adaptive, in the sense that they increase or decrease depending on environmental conditions. The majority of these models are abstract and make no attempt to describe the neural circuitry that implements the proposed computations. This article describes a biologically detailed computational model that overcomes this shortcoming. Specifically, we propose a neural circuit that implements adaptive learning rates by modulating the gain on the dopamine response to reward prediction errors, and we model activity within this circuit at the level of spiking neurons. The model generates a dopamine signal that depends on the size of the tonically active dopamine neuron population and the phasic spike rate. The model was tested successfully against results from two single-neuron recording studies and a fast-scan cyclic voltammetry study. We conclude by discussing the general applicability of the model to dopamine mediated tasks that transcend the experimental phenomena it was initially designed to address.

Prefrontal regulation of behavioural control: Evidence from learning theory and translational approaches in rodents

Everyday activities require adaptive decision-making and control over our actions to achieve our goals. Sub-regions within the cortex are widely reported to regulate these choices. Here we review rodent studies from two disparate fields of instrumental action control - goal-directed and habitual responding, and impulsive and compulsive behaviour. Our aim was to compare findings across the spectrum, from precision associative learning to translational studies of action control. The evidence suggests that each cortical sub-region performs different roles depending on task requirements and, within tasks, clear dissociations exist between regions. Rather than synthesizing a single role or function for a given region, we should consider regions to be capable of many different functions. Further investigation of cortico-cortical connections and the pattern of input and output circuitry within each region may be needed to identify unique process-specific pathways. Despite differences in the scope and purpose of these two fields, integrating evidence across tasks provides a broader context for testing hypotheses about the role of cortical regions in adaptive actions and decision-making.

Adaptive learning under expected and unexpected uncertainty

The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.

An insular view of the social decision-making network

Social animals must detect, evaluate and respond to the emotional states of other individuals in their group. A constellation of gestures, vocalizations, and chemosignals enable animals to convey affect and arousal to others in nuanced, multisensory ways. Observers integrate social information with environmental and internal factors to select behavioral responses to others via a process call social decision-making. The Social Decision Making Network (SDMN) is a system of brain structures and neurochemicals that are conserved across species (mammals, reptiles, amphibians, birds) that are the proximal mediators of most social behaviors. However, how sensory information reaches the SDMN to shape behavioral responses during a social encounter is not well known. Here we review the empirical data that demonstrate the necessity of sensory systems in detecting social stimuli, as well as the anatomical connectivity of sensory systems with each node of the SDMN. We conclude that the insular cortex is positioned to link integrated social sensory cues to this network to produce flexible and appropriate behavioral responses to socioemotional cues.

Sucrose intensity coding and decision-making in rat gustatory cortices

Sucrose's sweet intensity is one attribute contributing to the overconsumption of high-energy palatable foods. However, it is not known how sucrose intensity is encoded and used to make perceptual decisions by neurons in taste-sensitive cortices. We trained rats in a sucrose intensity discrimination task and found that sucrose evoked a widespread response in neurons recorded in posterior-Insula (pIC), anterior-Insula (aIC), and Orbitofrontal cortex (OFC). Remarkably, only a few Intensity-selective neurons conveyed the most information about sucrose's intensity, indicating that for sweetness the gustatory system uses a compact and distributed code. Sucrose intensity was encoded in both firing-rates and spike-timing. The pIC, aIC, and OFC neurons tracked movement direction, with OFC neurons yielding the most robust response. aIC and OFC neurons encoded the subject's choices, whereas all three regions tracked reward omission. Overall, these multimodal areas provide a neural representation of perceived sucrose intensity, and of task-related information underlying perceptual decision-making.

Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making

Rat orbitofrontal cortex (OFC) is located in the dorsal bank of the rhinal sulcus, and is divided into the medial orbital area, ventral orbital area, ventrolateral orbital area, lateral orbital area, dorsolateral orbital area, and agranular insular areas. Over the past 20 years, there has been a marked increase in the number of publications focused on the functions of rat OFC. While collectively this extensive body of work has provided great insight into the functions of OFC, leading to theoretical and computational models of its functions, one issue that has emerged relates to what is defined as OFC because targeting of this region can be quite variable between studies of appetitive behavior, even within the same species. Also apparent is that there is an oversampling and undersampling of certain subregions of rat OFC for study, and this will be demonstrated here. The intent of the Viewpoint is to summarize studies in rat OFC, given the diversity of what groups refer to as "OFC," and to integrate these with the findings of recent anatomical studies. The primary aim is to help discern functions in reward learning and decision-making, clearing the course for future empirical work.

Lateral Orbitofrontal Cortical Modulation on the Medial Prefrontal Cortex-Amygdala Pathway: Differential Regulation of Intra-Amygdala GABAA and GABAB Receptors

BACKGROUND

The basolateral complex of the amygdala receives inputs from neocortical areas, including the medial prefrontal cortex and lateral orbitofrontal cortex. Earlier studies have shown that lateral orbitofrontal cortex activation exerts an inhibitory gating on medial prefrontal cortex-amygdala information flow. Here we examined the individual role of GABAA and GABAB receptors in this process.

METHODS

In vivo extracellular single-unit recordings were done in anesthetized rats. We searched amygdala neurons that fire in response to medial prefrontal cortex activation, tested lateral orbitofrontal cortex gating at different delays (lateral orbitofrontal cortex-medial prefrontal cortex delays: 25, 50, 100, 250, 500, and 1000 milliseconds), and examined differential contribution of GABAA and GABAB receptors with iontophoresis.

RESULTS

Relative to baseline, lateral orbitofrontal cortex stimulation exerted an inhibitory modulatory gating on the medial prefrontal cortex-amygdala pathway and was effective up to a long delay of 500 ms (long-delay latencies at 100, 250, and 500 milliseconds). Moreover, blockade of intra-amygdala GABAA receptors with bicuculline abolished the lateral orbitofrontal cortex inhibitory gating at both short- (25 milliseconds) and long-delay (100 milliseconds) intervals, while blockade of GABAB receptors with saclofen reversed the inhibitory gating at long delay (100 milliseconds) only. Among the majority of the neurons examined (8 of 9), inactivation of either GABAA or GABAB receptors during baseline did not change evoked probability per se, suggesting that local feed-forward inhibitory mechanism is pathway specific.

CONCLUSIONS

Our results suggest that the effect of lateral orbitofrontal cortex inhibitory modulatory gating was effective up to 500 milliseconds and that intra-amygdala GABAA and GABAB receptors differentially modulate the short- and long-delay lateral orbitofrontal cortex inhibitory gating on the medial prefrontal cortex-amygdala pathway.