Persistent Valence Representations by Ensembles of Anterior Cingulate Cortex Neurons

Anterior cingulate cortex provides the neural substrates for feedback-driven iteration of decision and value representation

Adjusting decision-making under uncertain and dynamic situations is the hallmark of intelligence. It requires a system capable of converting feedback information to renew the internal value. The anterior cingulate cortex (ACC) involves in error and reward events that prompt switching or maintenance of current decision strategies. However, it is unclear whether and how the changes of stimulus-action mapping during behavioral adaptation are encoded, nor how such computation drives decision adaptation. Here, we tracked ACC activity in male mice performing go/no-go auditory discrimination tasks with manipulated stimulus-reward contingencies. Individual ACC neurons integrate the outcome information to the value representation in the next-run trials. Dynamic recruitment of them determines the learning rate of error-guided value iteration and decision adaptation, forming a non-linear feedback-driven updating system to secure the appropriate decision switch. Optogenetically suppressing ACC significantly slowed down feedback-driven decision switching without interfering with the execution of the established strategy.

Reconfiguration of Behavioral Signals in the Anterior Cingulate Cortex Based on Emotional State

Behaviors and their execution depend on the context and emotional state in which they are performed. The contextual modulation of behavior likely relies on regions such as the anterior cingulate cortex (ACC) that multiplex information about emotional/autonomic states and behaviors. The objective of the present study was to understand how the representations of behaviors by ACC neurons become modified when performed in different emotional states. A pipeline of machine learning techniques was developed to categorize and classify complex, spontaneous behaviors in male rats from the video. This pipeline, termed Hierarchical Unsupervised Behavioural Discovery Tool (HUB-DT), discovered a range of statistically separable behaviors during a task in which motivationally significant outcomes were delivered in blocks of trials that created three unique "emotional contexts." HUB-DT was capable of detecting behaviors specific to each emotional context and was able to identify and segregate the portions of a neural signal related to a behavior and to emotional context. Overall, ∼10× as many neurons responded to behaviors in a contextually dependent versus a fixed manner, highlighting the extreme impact of emotional state on representations of behaviors that were precisely defined based on detailed analyses of limb kinematics. This type of modulation may be a key mechanism that allows the ACC to modify the behavioral output based on emotional states and contextual demands.

Examining a punishment-related brain circuit with miniature fluorescence microscopes and deep learning

In humans experiencing substance use disorder (SUD), abstinence from drug use is often motivated by a desire to avoid some undesirable consequence of further use: health effects, legal ramifications, etc. This process can be experimentally modeled in rodents by training and subsequently punishing an operant response in a context-induced reinstatement procedure. Understanding the biobehavioral mechanisms underlying punishment learning is critical to understanding both abstinence and relapse in individuals with SUD. To date, most investigations into the neural mechanisms of context-induced reinstatement following punishment have utilized discrete loss-of-function manipulations that do not capture ongoing changes in neural circuitry related to punishment-induced behavior change. Here, we describe a two-pronged approach to analyzing the biobehavioral mechanisms of punishment learning using miniature fluorescence microscopes and deep learning algorithms. We review recent advancements in both techniques and consider a target neural circuit.

Observational activation of anterior cingulate cortical neurons coordinates hippocampal replay in social learning

Social learning enables a subject to make decisions by observing the actions of another. How neural circuits acquire relevant information during observation to guide subsequent behavior is unknown. Utilizing an observational spatial working memory task, we show that neurons in the rat anterior cingulate cortex (ACC) associated with spatial trajectories during self-running in a maze are activated when observing another rat running the same maze. The observation-induced ACC activities are reduced in error trials and are correlated with activities of hippocampal place cells representing the same trajectories. The ACC activities during observation also predict subsequent hippocampal place cell activities during sharp-wave ripples and spatial contents of hippocampal replay prior to self-running. The results support that ACC neurons involved in decisions during self-running are reactivated during observation and coordinate hippocampal replay to guide subsequent spatial navigation.

ACC neural ensemble dynamics are structured by strategy prevalence

Medial frontal cortical areas are thought to play a critical role in the brain's ability to flexibly deploy strategies that are effective in complex settings, yet the underlying circuit computations remain unclear. Here, by examining neural ensemble activity in male rats that sample different strategies in a self-guided search for latent task structure, we observe robust tracking during strategy execution of a summary statistic for that strategy in recent behavioral history by the anterior cingulate cortex (ACC), especially by an area homologous to primate area 32D. Using the simplest summary statistic - strategy prevalence in the last 20 choices - we find that its encoding in the ACC during strategy execution is wide-scale, independent of reward delivery, and persists through a substantial ensemble reorganization that accompanies changes in global context. We further demonstrate that the tracking of reward by the ACC ensemble is also strategy-specific, but that reward prevalence is insufficient to explain the observed activity modulation during strategy execution. Our findings argue that ACC ensemble dynamics is structured by a summary statistic of recent behavioral choices, raising the possibility that ACC plays a role in estimating - through statistical learning - which actions promote the occurrence of events in the environment.

Abstraction of Reward Context Facilitates Relative Reward Coding in Neural Populations of the Macaque Anterior Cingulate Cortex

The anterior cingulate cortex (ACC) is believed to be involved in many cognitive processes, including linking goals to actions and tracking decision-relevant contextual information. ACC neurons robustly encode expected outcomes, but how this relates to putative functions of ACC remains unknown. Here, we approach this question from the perspective of population codes by analyzing neural spiking data in the ventral and dorsal banks of the ACC in two male monkeys trained to perform a stimulus-motor mapping task to earn rewards or avoid losses. We found that neural populations favor a low dimensional representational geometry that emphasizes the valence of potential outcomes while also facilitating the independent, abstract representation of multiple task-relevant variables. Valence encoding persisted throughout the trial, and realized outcomes were primarily encoded in a relative sense, such that cue valence acted as a context for outcome encoding. This suggests that the population coding we observe could be a mechanism that allows feedback to be interpreted in a context-dependent manner. Together, our results point to a prominent role for ACC in context setting and relative interpretation of outcomes, facilitated by abstract, or untangled, representations of task variables.SIGNIFICANCE STATEMENT The ability to interpret events in light of the current context is a critical facet of higher-order cognition. The ACC is suggested to be important for tracking contextual information, whereas alternate views hold that its function is more related to the motor system and linking goals to appropriate actions. We evaluated these possibilities by analyzing geometric properties of neural population activity in monkey ACC when contexts were determined by the valence of potential outcomes and found that this information was represented as a dominant, abstract concept. Ensuing outcomes were then coded relative to these contexts, suggesting an important role for these representations in context-dependent evaluation. Such mechanisms may be critical for the abstract reasoning and generalization characteristic of biological intelligence.

Adult hippocampal neurogenesis shapes adaptation and improves stress response: a mechanistic and integrative perspective

Adult hippocampal neurogenesis (AHN) represents a remarkable form of neuroplasticity that has increasingly been linked to the stress response in recent years. However, the hippocampus does not itself support the expression of the different dimensions of the stress response. Moreover, the main hippocampal functions are essentially preserved under AHN depletion and adult-born immature neurons (abGNs) have no extrahippocampal projections, which questions the mechanisms by which abGNs influence functions supported by brain areas far from the hippocampus. Within this framework, we propose that through its computational influences AHN is pivotal in shaping adaption to environmental demands, underlying its role in stress response. The hippocampus with its high input convergence and output divergence represents a computational hub, ideally positioned in the brain (1) to detect cues and contexts linked to past, current and predicted stressful experiences, and (2) to supervise the expression of the stress response at the cognitive, affective, behavioral, and physiological levels. AHN appears to bias hippocampal computations toward enhanced conjunctive encoding and pattern separation, promoting contextual discrimination and cognitive flexibility, reducing proactive interference and generalization of stressful experiences to safe contexts. These effects result in gating downstream brain areas with more accurate and contextualized information, enabling the different dimensions of the stress response to be more appropriately set with specific contexts. Here, we first provide an integrative perspective of the functional involvement of AHN in the hippocampus and a phenomenological overview of the stress response. We then examine the mechanistic underpinning of the role of AHN in the stress response and describe its potential implications in the different dimensions accompanying this response.

Event-based control of autonomic and emotional states by the anterior cingulate cortex

Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.

The anterior cingulate cortex and event-based modulation of autonomic states

In spite of being an intensive area of research focus, the anterior cingulate cortex (ACC) remains somewhat of an enigma. Many theories have focused on its role in various aspects of cognition yet surgically precise lesions of the ACC, used to treat severe emotional disorders in human patients, typically have no lasting effects on cognition. An alternative view is that the ACC has a prominent role in regulating autonomic states. This view is consistent with anatomical data showing that a main target of the ACC are regions involved in autonomic control and with the observation that stimulation of the ACC evokes changes in autonomic states in both animals and humans. From an electrophysiological perspective, ACC neurons appear able to represent virtually any event or internal state, even though there is not always a strong link between these representations and behavior. Ensembles of neurons form robust contextual representations that strongly influence how specific events are encoded. The activity patterns associated with these contextually-based event representations presumably impact activity in downstream regions that control autonomic state. As a result, the ACC may regulate the autonomic and perhaps emotional reactions to events it is representing. This event-based control of autonomic tone by the ACC would likely arise during all types of cognitive and affective processes, without necessarily being critical for any of them.

Potential roles of the rodent medial prefrontal cortex in conflict resolution between multiple decision-making systems

Mammalian decision-making is mediated by the interaction of multiple, neurally and computationally separable decision systems. Having multiple systems requires a mechanism to manage conflict and converge onto the selection of singular actions. A long history of evidence has pointed to the prefrontal cortex as a central component in processing the interactions between distinct decision systems and resolving conflicts among them. In this chapter we review four theories of how that interaction might occur and identify how the medial prefrontal cortex in the rodent may be involved in each theory. We then present experimental predictions implied by the neurobiological data in the context of each theory as a starting point for future investigation of medial prefrontal cortex and decision-making.

Integration of value and action in medial prefrontal neural systems

The rodent medial prefrontal cortex (mPFC) plays a key role in regulating cognition, emotion, and behavior. mPFC neurons are activated in diverse experimental paradigms, raising the questions of whether there are specific task elements or dimensions encoded by mPFC neurons, and whether these encoded parameters are selective to neurons in particular mPFC subregions or networks. Here, we consider the role of mPFC neurons in processing appetitive and aversive cues, outcomes, and related behaviors. mPFC neurons are strongly activated in tasks probing value and outcome-associated actions, but these responses vary across experimental paradigms. Can we identify specific categories of responses (e.g., positive or negative value), or do mPFC neurons exhibit response properties that are too heterogeneous/complex to cluster into distinct conceptual groups? Based on a review of relevant studies, we consider what has been done and what needs to be further explored in order to address these questions.

Abrupt, Asynchronous Changes in Action Representations by Anterior Cingulate Cortex Neurons during Trial and Error Learning

The ability to act on knowledge about the value of stimuli or actions factors into simple foraging behaviors as well as complex forms of decision-making. In striatal regions, action representations are thought to acquire value through a gradual (reinforcement-learning based) process. It is unclear whether this is also true for anterior cingulate cortex (ACC) where neuronal representations tend to change abruptly. We recorded from ensembles of ACC neurons as rats deduced which of 3 levers was rewarded each day. The rat's lever preferences changed gradually throughout the sessions as they eventually came to focus on the rewarded lever. Most individual neurons changed their responses to both rewarded and nonrewarded lever presses abruptly (<2 trials). These transitions occurred asynchronously across the population but peaked near the point where the rats began to focus on the rewarded lever. Because the individual transitions were asynchronous, the overall change at the population level appeared gradual. Abrupt transitions in action representations of ACC neurons may be part of a mechanism that alters choice strategies as new information is acquired.

Unanticipated Stressful and Rewarding Experiences Engage the Same Prefrontal Cortex and Ventral Tegmental Area Neuronal Populations

Brain networks that mediate motivated behavior in the context of aversive and rewarding experiences involve the prefrontal cortex (PFC) and ventral tegmental area (VTA). Neurons in both regions are activated by stress and reward, and by learned cues that predict aversive or appetitive outcomes. Recent studies have proposed that separate neuronal populations and circuits in these regions encode learned aversive versus appetitive contexts. But how about the actual experience? Do the same or different PFC and VTA neurons encode unanticipated aversive and appetitive experiences? To address this, we recorded unit activity and local field potentials (LFPs) in the dorsomedial PFC (dmPFC) and VTA of male rats as they were exposed, in the same recording session, to reward (sucrose) or stress (tail pinch) spaced 1 h apart. As expected, experience-specific neuronal responses were observed. Approximately 15–25% of single units in each region responded by excitation or inhibition to either stress or reward, and only stress increased LFP theta oscillation power in both regions and coherence between regions. But the largest number of responses (29% dmPFC and 30% VTA units) involved dual-valence neurons that responded to both stress and reward exposure. Moreover, the temporal profile of neuronal population activity in dmPFC and VTA as assessed by principal component analysis (PCA) were similar during both types of experiences. These results reveal that aversive and rewarding experiences engage overlapping neuronal populations in the dmPFC and the VTA. These populations may provide a locus of vulnerability for stress-related disorders, which are often associated with anhedonia.

Dopamine Modulation of Prefrontal Cortex Activity Is Manifold and Operates at Multiple Temporal and Spatial Scales

Although the function of dopamine in subcortical structures is largely limited to reward and movement, dopamine neurotransmission in the prefrontal cortex (PFC) is critical to a multitude of temporally and functionally diverse processes, such as attention, working memory, behavioral flexibility, action planning, and sustained motivational and affective states. How does dopamine influence computation of these temporally complex functions? We find causative links between sustained and burst patterns of phasic dopamine neuron activation and modulation of medial PFC neuronal activity at multiple spatiotemporal scales. These include a multidirectional and weak impact on individual neuron rate activity but a robust influence on coordinated ensemble activity, gamma oscillations, and gamma-theta coupling that persisted for minutes. In addition, PFC network responses to burst pattern of dopamine firing were selectively strengthened in behaviorally active states. This multiplex mode of modulation by dopamine input may enable PFC to compute and generate spatiotemporally diverse and specialized outputs.