Differential encoding of factors influencing predicted reward value in monkey rostral anterior cingulate cortex

Habit formation viewed as structural change in the behavioral network

Habit formation is a process in which an action becomes involuntary. While goal-directed behavior is driven by its consequences, habits are elicited by a situation rather than its consequences. Existing theories have proposed that actions are controlled by corresponding two distinct systems. Although canonical theories based on such distinctions are starting to be challenged, there are a few theoretical frameworks that implement goal-directed behavior and habits within a single system. Here, we propose a novel theoretical framework by hypothesizing that behavior is a network composed of several responses. With this framework, we have shown that the transition of goal-directed actions to habits is caused by a change in a single network structure. Furthermore, we confirmed that the proposed network model behaves in a manner consistent with the existing experimental results reported in animal behavioral studies. Our results revealed that habit could be formed under the control of a single system rather than two distinct systems. By capturing the behavior as a single network change, this framework provides a new perspective on studying the structure of the behavior for experimental and theoretical research.

Dorsomedial prefrontal cortex activation disrupts Pavlovian incentive motivation

The dorsomedial prefrontal cortex (dmPFC) is known to make important contributions to flexible, reward-motivated behavior. However, it remains unclear if the dmPFC is involved in regulating the expression of Pavlovian incentive motivation, the process through which reward-paired cues promote instrumental reward-seeking behavior, which is modeled in rats using the Pavlovian-instrumental transfer (PIT) task. The current study examined this question using a bidirectional chemogenetic strategy in which inhibitory (hM4Di) or excitatory (hM3Dq) designer G-protein coupled receptors were virally expressed in dmPFC neurons, allowing us to later stimulate or inhibit this region by administering CNO prior to PIT testing. We found that dmPFC inhibition did not alter the tendency for a reward-paired cue to instigate instrumental reward-seeking behavior, whereas dmPFC stimulation disrupted the expression of this motivational influence. Neither treatment altered cue-elicited anticipatory activity at the reward-delivery port, indicating that dmPFC stimulation did not lead to more widespread motor suppression. A reporter-only control experiment indicated that our CNO treatment did not have non-specific behavioral effects. Thus, the dmPFC does not mediate the expression of Pavlovian incentive motivation but instead has the capacity to exert pronounced inhibitory control over this process, suggesting that it is involved in adaptively regulating cue-motivated behavior.

Event-based control of autonomic and emotional states by the anterior cingulate cortex

Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.

Abnormal brain activity in nonsuicidal self-injury: a coordinate-based activation likelihood meta-analysis of functional neuroimaging studies

Background

The high prevalence of nonsuicidal self-injury (NSSI) in youths demonstrates a substantial population-level burden on society. NSSI is often associated with emotional and social skill deficits. To date, several studies have aimed to identify the underlying neural mechanism of those deficits in NSSI by using functional magnetic resonance imaging (fMRI). However, their conclusions display poor consistency.

Objective

We aimed to conduct a meta-analysis using activation likelihood estimation (ALE) for fMRI data based on emotional and cognitive tasks to clarify the underlying neural processing deficits of NSSI.

Methods

We searched for MRI studies of NSSI in the PubMed, Cochrane, and Embase databases. We identified significant foci for the included studies and conducted two ALE meta-analyses as follows: (i) activation for the NSSI contrast healthy control group and (ii) deactivation for the NSSI contrast healthy controls. Considering the diverse sex composition of study participants and possible bias from one large sample study, we conducted sensitivity analyses for the meta-analysis.

Results

Nine studies comprising 359 participants were included, and the results demonstrated substantial activation in NSSI patients compared with healthy controls in two clusters located in the right medial frontal gyrus extending to the rostral anterior cingulate and the left inferior frontal gyrus extending to the insula.

Conclusions

The results suggest that individuals with NSSI show brain activity alterations that underpin their core symptoms, including poor emotional regulation and reward processing deficits. Our findings provide new insights into the neural mechanism of NSSI, which may serve as functional biomarkers for developing effective diagnosis and therapeutic interventions for these patients.

Distinct Spiking Patterns of Excitatory and Inhibitory Neurons and LFP Oscillations in Prefrontal Cortex During Sensory Discrimination

Prefrontal cortex (PFC) are broadly linked to various aspects of behavior. During sensory discrimination, PFC neurons can encode a range of task related information, including the identity of sensory stimuli and related behavioral outcome. However, it remains largely unclear how different neuron subtypes and local field potential (LFP) oscillation features in the mouse PFC are modulated during sensory discrimination. To understand how excitatory and inhibitory PFC neurons are selectively engaged during sensory discrimination and how their activity relates to LFP oscillations, we used tetrode recordings to probe well-isolated individual neurons, and LFP oscillations, in mice performing a three-choice auditory discrimination task. We found that a majority of PFC neurons, 78% of the 711 recorded individual neurons, exhibited sensory discrimination related responses that are context and task dependent. Using spike waveforms, we classified these responsive neurons into putative excitatory neurons with broad waveforms or putative inhibitory neurons with narrow waveforms, and found that both neuron subtypes were transiently modulated, with individual neurons' responses peaking throughout the entire duration of the trial. While the number of responsive excitatory neurons remain largely constant throughout the trial, an increasing fraction of inhibitory neurons were gradually recruited as the trial progressed. Further examination of the coherence between individual neurons and LFPs revealed that inhibitory neurons exhibit higher spike-field coherence with LFP oscillations than excitatory neurons during all aspects of the trial and across multiple frequency bands. Together, our results demonstrate that PFC excitatory neurons are continuously engaged during sensory discrimination, whereas PFC inhibitory neurons are increasingly recruited as the trial progresses and preferentially coordinated with LFP oscillations. These results demonstrate increasing involvement of inhibitory neurons in shaping the overall PFC dynamics toward the completion of the sensory discrimination task.

The anterior cingulate cortex and event-based modulation of autonomic states

In spite of being an intensive area of research focus, the anterior cingulate cortex (ACC) remains somewhat of an enigma. Many theories have focused on its role in various aspects of cognition yet surgically precise lesions of the ACC, used to treat severe emotional disorders in human patients, typically have no lasting effects on cognition. An alternative view is that the ACC has a prominent role in regulating autonomic states. This view is consistent with anatomical data showing that a main target of the ACC are regions involved in autonomic control and with the observation that stimulation of the ACC evokes changes in autonomic states in both animals and humans. From an electrophysiological perspective, ACC neurons appear able to represent virtually any event or internal state, even though there is not always a strong link between these representations and behavior. Ensembles of neurons form robust contextual representations that strongly influence how specific events are encoded. The activity patterns associated with these contextually-based event representations presumably impact activity in downstream regions that control autonomic state. As a result, the ACC may regulate the autonomic and perhaps emotional reactions to events it is representing. This event-based control of autonomic tone by the ACC would likely arise during all types of cognitive and affective processes, without necessarily being critical for any of them.

Noise-Correlation Is Modulated by Reward Expectation in the Primary Motor Cortex Bilaterally During Manual and Observational Tasks in Primates

Reward modulation is represented in the motor cortex (M1) and could be used to implement more accurate decoding models to improve brain-computer interfaces (BCIs; Zhao et al., 2018). Analyzing trial-to-trial noise-correlations between neural units in the presence of rewarding (R) and non-rewarding (NR) stimuli adds to our understanding of cortical network dynamics. We utilized Pearson's correlation coefficient to measure shared variability between simultaneously recorded units (32-112) and found significantly higher noise-correlation and positive correlation between the populations' signal- and noise-correlation during NR trials as compared to R trials. This pattern is evident in data from two non-human primates (NHPs) during single-target center out reaching tasks, both manual and action observation versions. We conducted a mean matched noise-correlation analysis to decouple known interactions between event-triggered firing rate changes and neural correlations. Isolated reward discriminatory units demonstrated stronger correlational changes than units unresponsive to reward firing rate modulation, however, the qualitative response was similar, indicating correlational changes within the network as a whole can serve as another information channel to be exploited by BCIs that track the underlying cortical state, such as reward expectation, or attentional modulation. Reward expectation and attention in return can be utilized with reinforcement learning (RL) towards autonomous BCI updating.

Neurons in the monkey orbitofrontal cortex mediate reward value computation and decision-making

Choice reflects the values of available alternatives; more valuable options are chosen more often than less valuable ones. Here we studied whether neuronal responses in orbitofrontal cortex (OFC) reflect the value difference between options, and whether there is a causal link between OFC neuronal activity and choice. Using a decision-making task where two visual stimuli were presented sequentially, each signifying a value, we showed that when the second stimulus appears many neurons encode the value difference between alternatives. Later when the choice occurs, that difference signal disappears and a signal indicating the chosen value emerges. Pharmacological inactivation of OFC neurons coding for choice-related values increases the monkey's latency to make a choice and the likelihood that it will choose the less valuable alternative, when the value difference is small. Thus, OFC neurons code for value information that could be used to directly influence choice.

Persistent Valence Representations by Ensembles of Anterior Cingulate Cortex Neurons

The anterior cingulate cortex (ACC) responds to outcomes of a positive or negative valence, but past studies typically focus on one valence or the other, making it difficult to know how opposing valences are disambiguated. We recorded from ACC neurons as rats received tones followed by aversive, appetitive or null outcomes. The responses to the different tones/outcomes were highly inter-mixed at the single neuron level but combined to produce robust valence-specific representations at the ensemble level. The valence-specific patterns far outlasted the tones and outcomes, persisting throughout the long inter-trial intervals (ITIs) and even throughout trial blocks. When the trials were interleaved, the valence-specific patterns abruptly shifted at the start of each new trial. Overall the aversive trials had the greatest impact on the neurons. Thus within the ACC, valence-specificity is largely an emergent property of ensembles and valence-specific representations can appear quickly and persist long after the initiating event.

Ramping ensemble activity in dorsal anterior cingulate neurons during persistent commitment to a decision

We frequently need to commit to a choice to achieve our goals; however, the neural processes that keep us motivated in pursuit of delayed goals remain obscure. We examined ensemble responses of neurons in macaque dorsal anterior cingulate cortex (dACC), an area previously implicated in self-control and persistence, in a task that requires commitment to a choice to obtain a reward. After reward receipt, dACC neurons signaled reward amount with characteristic ensemble firing rate patterns; during the delay in anticipation of the reward, ensemble activity smoothly and gradually came to resemble the postreward pattern. On the subset of risky trials, in which a reward was anticipated with 50% certainty, ramping ensemble activity evolved to the pattern associated with the anticipated reward (and not with the anticipated loss) and then, on loss trials, took on an inverted form anticorrelated with the form associated with a win. These findings enrich our knowledge of reward processing in dACC and may have broader implications for our understanding of persistence and self-control.

Toward an autonomous brain machine interface: integrating sensorimotor reward modulation and reinforcement learning

For decades, neurophysiologists have worked on elucidating the function of the cortical sensorimotor control system from the standpoint of kinematics or dynamics. Recently, computational neuroscientists have developed models that can emulate changes seen in the primary motor cortex during learning. However, these simulations rely on the existence of a reward-like signal in the primary sensorimotor cortex. Reward modulation of the primary sensorimotor cortex has yet to be characterized at the level of neural units. Here we demonstrate that single units/multiunits and local field potentials in the primary motor (M1) cortex of nonhuman primates (Macaca radiata) are modulated by reward expectation during reaching movements and that this modulation is present even while subjects passively view cursor motions that are predictive of either reward or nonreward. After establishing this reward modulation, we set out to determine whether we could correctly classify rewarding versus nonrewarding trials, on a moment-to-moment basis. This reward information could then be used in collaboration with reinforcement learning principles toward an autonomous brain-machine interface. The autonomous brain-machine interface would use M1 for both decoding movement intention and extraction of reward expectation information as evaluative feedback, which would then update the decoding algorithm as necessary. In the work presented here, we show that this, in theory, is possible.

Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal

Whether driving a car, shopping for food, or paying attention in a classroom of boisterous teenagers, it's often hard to maintain focus on goals in the face of distraction. Brain imaging studies in humans implicate the dorsal anterior cingulate cortex (dACC) in regulating the conflict between goals and distractors. Here we show that single dACC neurons signal conflict between task goals and distractors in the rhesus macaque, particularly for biologically relevant social stimuli. For some neurons, task conflict signals predicted subsequent changes in pupil size-a peripheral index of arousal linked to noradrenergic tone-associated with reduced distractor interference. dACC neurons also responded to errors, and these signals predicted adjustments in pupil size. These findings provide the first neurophysiological endorsement of the hypothesis that dACC regulates conflict, in part, via modulation of pupil-linked processes such as arousal.

Tracking progress toward a goal in corticostriatal ensembles

When performing sequences of actions, we constantly keep track of our current position in the sequence relative to the overall goal. The present study searched for neural representations of sequence progression in corticostriatal circuits. Neurons within the anterior cingulate cortex (ACC) and its target region in the dorsal striatum (DS) were recorded from simultaneously as rats performed different sequences of lever presses. We analyzed the responses of the neurons to presses occurring in the "first," "second," or "third" serial position regardless of the particular sequence or physical levers. Principal component analysis revealed that the main source of firing rate variance in the ACC was a smooth ramp-like change as the animal progressed through the sequence toward the reward. No such smooth-ramping activity was observed in DS ensembles as firing tended to be tightly linked to each action. In the ACC, the progression in firing was observed only for correct choices and not errors, whereas in the DS, firing associated with each action in a sequence was similar regardless of whether the action was correct or not. Therefore, different forms of a signal exist within corticostriatal circuits that evolve across a sequence of actions, with DS ensembles tracking every action and ACC ensembles tracking actual progress toward the goal.

Self-choice enhances value in reward-seeking in primates

When an individual chooses one item from two or more alternatives, they compare the values of the expected outcomes. The outcome value can be determined by the associated reward amount, the probability of reward, and the workload required to earn the reward. Rational choice theory states that choices are made to maximize rewards over time, and that the same outcome values lead to an equal likelihood of choices. However, the theory does not distinguish between conditions with the same reward value, even when acquired under different circumstances, and does not always accurately describe real behavior. We have found that allowing a monkey to choose a reward schedule endows the schedule with extra value when compared to performance in an identical schedule that is chosen by another agent (a computer here). This behavior is not consistent with pure rational choice theory. Theoretical analysis using a modified temporal-difference learning model showed an enhanced schedule state value by self-choice. These results suggest that an increased reward value underlies the improved performances by self-choice during reward-seeking behavior.

Single-neuron mechanisms underlying cost-benefit analysis in frontal cortex

Effective decision-making requires consideration of costs and benefits. Previous studies have implicated orbitofrontal cortex (OFC), dorsolateral prefrontal cortex (DLPFC), and anterior cingulate cortex (ACC) in cost-benefit decision-making. Yet controversy remains about whether different decision costs are encoded by different brain areas, and whether single neurons integrate costs and benefits to derive a subjective value estimate for each choice alternative. To address these issues, we trained four subjects to perform delay- and effort-based cost-benefit decisions and recorded neuronal activity in OFC, ACC, DLPFC, and the cingulate motor area (CMA). Although some neurons, mainly in ACC, did exhibit integrated value signals as if performing cost-benefit computations, they were relatively few in number. Instead, the majority of neurons in all areas encoded the decision type; that is whether the subject was required to perform a delay- or effort-based decision. OFC and DLPFC neurons tended to show the largest changes in firing rate for delay- but not effort-based decisions; whereas, the reverse was true for CMA neurons. Only ACC contained neurons modulated by both effort- and delay-based decisions. These findings challenge the idea that OFC calculates an abstract value signal to guide decision-making. Instead, our results suggest that an important function of single PFC neurons is to categorize sensory stimuli based on the consequences predicted by those stimuli.

Neurons in monkey dorsal raphe nucleus code beginning and progress of step-by-step schedule, reward expectation, and amount of reward outcome in the reward schedule task

The dorsal raphe nucleus is the major source of serotonin in the brain. It is connected to brain regions related to reward processing, and the neurons show activity related to predicted reward outcome. Clinical observations also suggest that it is important in maintaining alertness and its apparent role in addiction seems to be related to reward processing. Here, we examined whether the neurons in dorsal raphe carry signals about reward outcome and task progress during multitrial schedules. We recorded from 98 single neurons in dorsal raphe of two monkeys. The monkeys perform one, two, or three visual discrimination trials (schedule), obtaining one, two, or three drops of liquid. In the valid cue condition, the length and brightness of a visual cue indicated schedule progress and reward amount, respectively. In the random cue condition, the visual cue was randomly presented with respect to schedule length and reward amount. We found information encoded about (1) schedule onset, (2) reward expectation, (3) reward outcome, and (4) reward amount in the mean firing rates. Information theoretic analysis showed that the temporal variation of the neuronal responses contained additional information related to the progress of the schedule toward the reward rather than only discriminating schedule onset or reward/no reward. When considered in light of all that is known about the raphe in anatomy, physiology, and behavior, the rich encoding about both task progress and predicted reward outcome makes the raphe a strong candidate for providing signals throughout the brain to coordinate persistent goal-seeking behavior.