Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior

Influence of surprise on reinforcement learning in younger and older adults

Surprise is a key component of many learning experiences, and yet its precise computational role, and how it changes with age, remain debated. One major challenge is that surprise often occurs jointly with other variables, such as uncertainty and outcome probability. To assess how humans learn from surprising events, and whether aging affects this process, we studied choices while participants learned from bandits with either Gaussian or bi-modal outcome distributions, which decoupled outcome probability, uncertainty, and surprise. A total of 102 participants (51 older, aged 50-73; 51 younger, 19-30 years) chose between three bandits, one of which had a bimodal outcome distribution. Behavioral analyses showed that both age-groups learned the average of the bimodal bandit less well. A trial-by-trial analysis indicated that participants performed choice reversals immediately following large absolute prediction errors, consistent with heightened sensitivity to surprise. This effect was stronger in older adults. Computational models indicated that learning rates in younger as well as older adults were influenced by surprise, rather than uncertainty, but also suggested large interindividual variability in the process underlying learning in our task. Our work bridges between behavioral economics research that has focused on how outcomes with low probability affect choice in older adults, and reinforcement learning work that has investigated age differences in the effects of uncertainty and suggests that older adults overly adapt to surprising events, even when accounting for probability and uncertainty effects.

The Projection-Specific Noradrenergic Modulation of Perseverative Spatial Behavior in Adult Male Rats

Adaptive behavior relies on efficient cognitive control. The anterior cingulate cortex (ACC) is a key node within the executive prefrontal network. The reciprocal connectivity between the locus ceruleus (LC) and ACC is thought to support behavioral reorganization triggered by the detection of an unexpected change. We transduced LC neurons with either excitatory or inhibitory chemogenetic receptors in adult male rats and trained rats on a spatial task. Subsequently, we altered LC activity and confronted rats with an unexpected change of reward locations. In a new spatial context, rats with decreased noradrenaline (NA) in the ACC entered unbaited maze arms more persistently which was indicative of perseveration. In contrast, the suppression of the global NA transmission reduced perseveration. Neither chemogenetic manipulation nor inactivation of the ACC by muscimol affected the rate of learning, possibly due to partial virus transduction of the LC neurons and/or the compensatory engagement of other prefrontal regions. Importantly, we observed behavioral deficits in rats with LC damage caused by virus injection. The latter finding highlights the importance of careful histological assessment of virus-transduced brain tissue as inadvertent damage of the targeted cell population due to virus neurotoxicity or other factors might cause unwanted side effects. Although the specific role of ACC in the flexibility of spatial behavior has not been convincingly demonstrated, our results support the beneficial role of noradrenergic transmission for an optimal function of the ACC. Overall, our findings suggest the LC exerts the projection-specific modulation of neural circuits mediating the flexibility of spatial behavior.

Anterior cingulate cortex lesions impair multiple facets of task engagement not mediated by dorsomedial striatum neuron firing

The anterior cingulate cortex (ACC) has been implicated across multiple highly specialized cognitive functions-including task engagement, motivation, error detection, attention allocation, value processing, and action selection. Here, we ask if ACC lesions disrupt task performance and firing in dorsomedial striatum (DMS) during the performance of a reward-guided decision-making task that engages many of these cognitive functions. We found that ACC lesions impacted several facets of task performance-including decreasing the initiation and completion of trials, slowing reaction times, and resulting in suboptimal and inaccurate action selection. Reductions in movement times towards the end of behavioral sessions further suggested attenuations in motivation, which paralleled reductions in directional action selection signals in the DMS that were observed later in recording sessions. Surprisingly, however, beyond altered action signals late in sessions-neural correlates in the DMS were largely unaffected, even though behavior was disrupted at multiple levels. We conclude that ACC lesions result in overall deficits in task engagement that impact multiple facets of task performance during our reward-guided decision-making task, which-beyond impacting motivated action signals-arise from dysregulated attentional signals in the ACC and are mediated via downstream targets other than DMS.

Shifts in attention drive context-dependent subspace encoding in anterior cingulate cortex in mice during decision making

Attention supports decision making by selecting the features that are relevant for decisions. Selective enhancement of the relevant features and inhibition of distractors has been proposed as potential neural mechanisms driving this selection process. Yet, how attention operates when relevance cannot be directly determined, and the attention signal needs to be internally constructed is less understood. Here we recorded from populations of neurons in the anterior cingulate cortex (ACC) of mice in an attention-shifting task where relevance of stimulus modalities changed across blocks of trials. In contrast with V1 recordings, decoding of the irrelevant modality gradually declined in ACC after an initial transient. Our analytical proof and a recurrent neural network model of the task revealed mutually inhibiting connections that produced context-gated suppression as observed in mice. Using this RNN model we predicted a correlation between contextual modulation of individual neurons and their stimulus drive, which we confirmed in ACC but not in V1.

Beyond peaks and troughs: Multiplexed performance monitoring signals in the EEG

With the discovery of event-related potentials elicited by errors more than 30 years ago, a new avenue of research on performance monitoring, cognitive control, and decision making emerged. Since then, the field has developed and expanded fulminantly. After a brief overview on the EEG correlates of performance monitoring, this article reviews recent advancements based on single-trial analyses using independent component analysis, multiple regression, and multivariate pattern classification. Given the close interconnection between performance monitoring and reinforcement learning, computational modeling and model-based EEG analyses have made a particularly strong impact. The reviewed findings demonstrate that error- and feedback-related EEG dynamics represent variables reflecting how performance-monitoring signals are weighted and transformed into an adaptation signal that guides future decisions and actions. The model-based single-trial analysis approach goes far beyond conventional peak-and-trough analyses of event-related potentials and enables testing mechanistic theories of performance monitoring, cognitive control, and decision making.

Give it a second try? The influence of feedback and performance in the decision of reattempting

Feedback evaluation can affect behavioural continuation or discontinuation, and is essential for cognitive and motor skill learning. One critical factor that influences feedback evaluation is participants' internal estimation of self-performance. Previous research has shown that two event-related potential components, the Feedback-Related Negativity (FRN) and the P3, are related to feedback evaluation. In the present study, we used a time estimation task and EEG recordings to test the influence of feedback and performance on participants' decisions, and the sensitivity of the FRN and P3 components to those factors. In the experiment, participants were asked to reproduce the total duration of an intermittently presented visual stimulus. Feedback was given after every response, and participants had then to decide whether to retry the same trial and try to earn reward points, or to move on to the next trial. Results showed that both performance and feedback influenced participants' decision on whether to retry the ongoing trial. In line with previous studies, the FRN showed larger amplitude in response to negative than to positive feedback. Moreover, our results were also in agreement with previous works showing the relationship between the amplitude of the FRN and the size of feedback-related prediction error (PE), and provide further insight in how PE size influences participants' decisions on whether or not to retry a task. Specifically, we found that the larger the FRN, the more likely participants were to base their decision on their performance - choosing to retry the current trial after good performance or to move on to the next trial after poor performance, regardless of the feedback received. Conversely, the smaller the FRN, the more likely participants were to base their decision on the feedback received.

Anterior Cingulate Cortex Causally Supports Meta-Learning

In dynamic environments with volatile rewards the anterior cingulate cortex (ACC) is believed to determine whether a visual object is relevant and should be chosen. The ACC may achieve this by integrating reward information over time to estimate which objects are worth to explore and which objects should be avoided. Such a higher-order meta-awareness about which objects should be explored predicts that the ACC causally contributes to choices when the reward values of objects are unknown and must be inferred from ongoing exploration. We tested this suggestion in nonhuman primates using a learning task that varied the number of object features that could be relevant, and by controlling the motivational value of choosing objects. During learning the ACC was transiently micro-stimulated when subjects foveated the to-be-chosen stimulus. We found that stimulation selectively impaired learning when feature uncertainty and motivational value of choices were high, which was linked to a deficit in using reward outcomes for feature-specific credit assignment. Application of an adaptive reinforcement learning model confirmed a primary deficit in weighting prediction errors that led to a meta-learning impairment to adaptively increase exploration during learning and to an impaired use of working memory to support learning. These findings provide causal evidence that the reward history traces in ACC are essential for meta-adjusting the exploration-exploitation balance and the strength of working memory of object values during adaptive behavior.

Optogenetic Inhibition of Rat Anterior Cingulate Cortex Impairs the Ability to Initiate and Stay on Task

Our prior research has identified neural correlates of cognitive control in the anterior cingulate cortex (ACC), leading us to hypothesize that the ACC is necessary for increasing attention as rats flexibly learn new contingencies during a complex reward-guided decision-making task. Here, we tested this hypothesis by using optogenetics to transiently inhibit the ACC, while rats of either sex performed the same two-choice task. ACC inhibition had a profound impact on behavior that extended beyond deficits in attention during learning when expected outcomes were uncertain. We found that ACC inactivation slowed and reduced the number of trials rats initiated and impaired both their accuracy and their ability to complete sessions. Furthermore, drift-diffusion model analysis suggested that free-choice performance and evidence accumulation (i.e., reduced drift rates) were degraded during initial learning-leading to weaker associations that were more easily overridden in later trial blocks (i.e., stronger bias). Together, these results suggest that in addition to attention-related functions, the ACC contributes to the ability to initiate trials and generally stay on task.

Hierarchical control over foraging behavior by anterior cingulate cortex

Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).

Mesolimbic dopamine encodes reward prediction errors independent of learning rates

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented in synaptic weights in the striatum, and updated by dopamine-dependent plasticity, suggesting that dopamine release might reflect the product of the learning rate and RPE. Here, we leveraged the fact that animals learn faster in volatile environments to characterize dopamine encoding of learning rates. We trained rats on a task with semi-observable states offering different rewards, and rats adjusted how quickly they initiated trials across states using RPEs. Computational modeling and behavioral analyses showed that learning rates were higher following state transitions, and scaled with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the nucleus accumbens encoded RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.

Synthetic surprise as the foundation of the psychedelic experience

Psychedelic agents, such as LSD and psilocybin, induce marked alterations in consciousness via activation of the 5-HT2A receptor (5-HT2ARs). We hypothesize that psychedelics enforce a state of synthetic surprise through the biased activation of the 5-HTRs system. This idea is informed by recent insights into the role of 5-HT in signaling surprise. The effects on consciousness, explained by the cognitive penetrability of perception, can be described within the predictive coding framework where surprise corresponds to prediction error, the mismatch between predictions and actual sensory input. Crucially, the precision afforded to the prediction error determines its effect on priors, enabling a dynamic interaction between top-down expectations and incoming sensory data. By integrating recent findings on predictive coding circuitry and 5-HT2ARs transcriptomic data, we propose a biological implementation with emphasis on the role of inhibitory interneurons. Implications arise for the clinical use of psychedelics, which may rely primarily on their inherent capacity to induce surprise in order to disrupt maladaptive patterns.

The locus coeruleus as a global model failure system

Predictive processing models posit that brains constantly attempt to predict their sensory inputs. Prediction errors signal when these predictions are incorrect and are thought to be instructive signals that drive corrective plasticity. Recent findings support the idea that the locus coeruleus (LC) - a brain-wide neuromodulatory system - signals several types of prediction error. I discuss how these findings support models proposing that the LC signals global model failures: instances where predictions about the world are strongly violated. Focusing on the cortex, I explore the utility of this signal in learning rate control, how the LC circuit may compute the signal, and how this view may aid our understanding of neurodivergence.

Perceptual oddities: assessing the relationship between film editing and prediction processes

During film viewing, humans parse sequences of individual shots into larger narrative structures, often weaving transitions at edit points into an apparently seamless and continuous flow. Editing helps filmmakers manipulate visual transitions to induce feelings of fluency/disfluency, tension/relief, curiosity, expectation and several emotional responses. We propose that the perceptual dynamics induced by film editing can be captured by a predictive processing (PP) framework. We hypothesise that visual discontinuities at edit points produce discrepancies between anticipated and actual sensory input, leading to prediction error. Further, we propose that the magnitude of prediction error depends on the predictability of each shot within the narrative flow, and lay out an account based on conflict monitoring. We test this hypothesis in two empirical studies measuring electroencephalography (EEG) during passive viewing of film excerpts, as well as behavioural responses during an active edit detection task. We report the neural and behavioural modulations at editing boundaries across three levels of narrative depth, showing greater modulations for edits spanning less predictable, deeper narrative transitions. Overall, our contribution lays the groundwork for understanding film editing from a PP perspective. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectivess'.

Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

Debates on the dorsomedial prefrontal/dorsal anterior cingulate cortex: insights for future research

The dorsomedial prefrontal cortex/dorsal anterior cingulate cortex (dmPFC/dACC) is a brain area subject to many theories and debates over its function(s). Even its precise anatomical borders are subject to much controversy. In the past decades, the dmPFC/dACC has been associated with more than 15 different cognitive processes, which sometimes appear quite unrelated (e.g. body perception, cognitive conflict). As a result, understanding what the dmPFC/dACC does has become a real challenge for many neuroscientists. Several theories of this brain area's function(s) have been developed, leading to successive and competitive publications bearing different models, which sometimes contradict each other. During the last two decades, the lively scientific exchanges around the dmPFC/dACC have promoted fruitful research in cognitive neuroscience. In this review, we provide an overview of the anatomy of the dmPFC/dACC, summarize the state of the art of functions that have been associated with this brain area and present the main theories aiming at explaining the dmPFC/dACC function(s). We explore the commonalities and the arguments between the different theories. Finally, we explain what can be learned from these debates for future investigations of the dmPFC/dACC and other brain regions' functions.

Neurocomputations on dual-brain signals underlie interpersonal prediction during a natural conversation

Prediction on the partner's speech plays a key role in a smooth conversation. However, previous studies on this issue have been majorly conducted at the single-brain rather than dual-brain level, leaving the interpersonal prediction hypothesis untested. To fill this gap, this study combined a neurocomputational modeling approach with a natural conversation paradigm in which two salespersons persuaded a customer to buy their product with their haemodynamic signals being collected using functional near-infrared spectroscopy hyperscanning. First, the results showed a cognitive hierarchy in a natural conversation, with the lower-level process (i.e., pragmatic representation of the persuasion) in the salesperson interacting with the higher-level process (i.e., value representation of the product) in the customer. Next, we found that the right dorsal lateral prefrontal cortex (rdlPFC) and temporoparietal junction (rTPJ) were associated with the representation of the product's value in the customer, while the right inferior frontal cortex (rIFC) was associated with the representation of the pragmatic processes in the salesperson. Finally, neurocomputational modeling results supported the prediction of the salesperson's lower-level brain activity based on the customer's higher-level brain activity. Moreover, the updating weight of the prediction model based on the neural computation between the rIFC of the salesperson and the rTPJ of the customer was closely associated with the interaction context, whereas that based on the rIFC-rdlPFC was not. In summary, these findings provide initial support for the interpersonal prediction hypothesis at the dual-brain level and reveal a hierarchy for the interpersonal prediction process.

Only cortical prediction error signals are involved in visual learning, despite availability of subcortical prediction error signals

Both the midbrain systems, encompassing the ventral striatum (VS), and the cortical systems, including the dorsal anterior cingulate cortex (dACC), play roles in reinforcing and enhancing learning. However, the specific contributions of signals from these regions in learning remains unclear. To investigate this, we examined how VS and dACC are involved in visual perceptual learning (VPL) through an orientation discrimination task. In the primary experiment, subjects fasted for 5 hours before each of 14 days of training sessions and 3 days of test sessions. Subjects were rewarded with water for accurate trial responses. During the test sessions, BOLD signals were recorded from regions including VS and dACC. Although BOLD signals in both areas were associated with positive and negative RPEs, only those in dACC associated with negative RPE showed a significant correlation with performance improvement. Additionally, no significant correlation was observed between BOLD signals associated with RPEs in VS and dACC. These results suggest that although signals associated with positive and negative RPEs from both midbrain and cortical systems are readily accessible, only RPE signals in the prefrontal system, generated without linking to RPE signals in VS, are utilized for the enhancement of VPL.

Prediction during language comprehension: what is next?

Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.

Anterior cingulate cortex regulates pain catastrophizing-like behaviors in rats

Negative pain expectation including pain catastrophizing is a well-known clinical phenomenon whereby patients amplify the aversive value of a painful or oftentimes even a similar, non-painful stimulus. Mechanisms of pain catastrophizing, however, remain elusive. Here, we modeled pain catastrophizing behavior in rats, and found that rats subjected to repeated noxious pin pricks on one paw demonstrated an aversive response to similar but non-noxious mechanical stimuli delivered to the contralateral paw. Optogenetic inhibition of pyramidal neuron activity in the anterior cingulate cortex (ACC) during the application of repetitive noxious pin pricks eliminated this catastrophizing behavior. Time-lapse calcium (Ca2+) imaging in the ACC further revealed an increase in spontaneous neural activity after the delivery of noxious stimuli. Together these results suggest that the experience of repeated noxious stimuli may drive hyperactivity in the ACC, causing increased avoidance of subthreshold stimuli, and that reducing this hyperactivity may play a role in treating pain catastrophizing.

Shifts in attention drive context-dependent subspace encoding in anterior cingulate cortex during decision making

Attention is a cognitive faculty that selects part of a larger set of percepts, driven by cues such as stimulus saliency, internal goals or priors. The enhancement of the attended representation and inhibition of distractors have been proposed as potential neural mechanisms driving this selection process. Yet, how attention operates when the cue has to be internally constructed from conflicting stimuli, decision rules, and reward contingencies, is less understood. Here we recorded from populations of neurons in the anterior cingulate cortex (ACC), an area implicated in ongoing error monitoring and correction during decision conflicts, in a challenging attention-shifting task. In this task, mice had to attend to the rewarded modality when presented identical auditory and visual stimuli in two contexts without direct external cues. In the ACC, the irrelevant stimulus continuously became less decodable than the relevant stimulus as the trial progressed to the decision point. This contrasted strongly with our previous findings in V1 where both relevant and irrelevant stimuli were equally decodable throughout the trial. Using analytical tools and a recurrent neural network (RNN) model, we found that the linearly independent representation of stimulus modalities in ACC was well suited to context-gated suppression of a stimulus modality. We demonstrated that the feedback structure of lateral connections in the RNN consisted of excitatory interactions between cell ensembles representing the same modality and mutual inhibition between cell ensembles representing distinct stimulus modalities. Using this RNN model showing signatures of context-gated suppression, we predicted that the level of contextual modulation of individual neurons should be correlated with their relative responsiveness to the two stimulus modalities used in the task. We verified this prediction in recordings from ACC neurons but not from recordings from V1 neurons. Therefore, ACC effectively operates on low-dimensional neuronal subspaces to combine stimulus related information with internal cues to drive actions under conflict.

Surprise and novelty in the brain

Notions of surprise and novelty have been used in various experimental and theoretical studies across multiple brain areas and species. However, 'surprise' and 'novelty' refer to different quantities in different studies, which raises concerns about whether these studies indeed relate to the same functionalities and mechanisms in the brain. Here, we address these concerns through a systematic investigation of how different aspects of surprise and novelty relate to different brain functions and physiological signals. We review recent classifications of definitions proposed for surprise and novelty along with links to experimental observations. We show that computational modeling and quantifiable definitions enable novel interpretations of previous findings and form a foundation for future theoretical and experimental studies.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

Abstraction of Reward Context Facilitates Relative Reward Coding in Neural Populations of the Macaque Anterior Cingulate Cortex

The anterior cingulate cortex (ACC) is believed to be involved in many cognitive processes, including linking goals to actions and tracking decision-relevant contextual information. ACC neurons robustly encode expected outcomes, but how this relates to putative functions of ACC remains unknown. Here, we approach this question from the perspective of population codes by analyzing neural spiking data in the ventral and dorsal banks of the ACC in two male monkeys trained to perform a stimulus-motor mapping task to earn rewards or avoid losses. We found that neural populations favor a low dimensional representational geometry that emphasizes the valence of potential outcomes while also facilitating the independent, abstract representation of multiple task-relevant variables. Valence encoding persisted throughout the trial, and realized outcomes were primarily encoded in a relative sense, such that cue valence acted as a context for outcome encoding. This suggests that the population coding we observe could be a mechanism that allows feedback to be interpreted in a context-dependent manner. Together, our results point to a prominent role for ACC in context setting and relative interpretation of outcomes, facilitated by abstract, or untangled, representations of task variables.SIGNIFICANCE STATEMENT The ability to interpret events in light of the current context is a critical facet of higher-order cognition. The ACC is suggested to be important for tracking contextual information, whereas alternate views hold that its function is more related to the motor system and linking goals to appropriate actions. We evaluated these possibilities by analyzing geometric properties of neural population activity in monkey ACC when contexts were determined by the valence of potential outcomes and found that this information was represented as a dominant, abstract concept. Ensuing outcomes were then coded relative to these contexts, suggesting an important role for these representations in context-dependent evaluation. Such mechanisms may be critical for the abstract reasoning and generalization characteristic of biological intelligence.

Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy

In cognitive-behavioral conceptualizations of anxiety, exaggerated threat expectancies underlie maladaptive anxiety. This view has led to successful treatments, notably exposure therapy, but is not consistent with the empirical literature on learning and choice alterations in anxiety. Empirically, anxiety is better described as a disorder of uncertainty learning. How disruptions in uncertainty lead to impairing avoidance and are treated with exposure-based methods, however, is unclear. Here, we integrate concepts from neurocomputational learning models with clinical literature on exposure therapy to propose a new framework for understanding maladaptive uncertainty functioning in anxiety. Specifically, we propose that anxiety disorders are fundamentally disorders of uncertainty learning and that successful treatments, particularly exposure therapy, work by remediating maladaptive avoidance from dysfunctional explore/exploit decisions in uncertain, potentially aversive situations. This framework reconciles several inconsistencies in the literature and provides a path forward to better understand and treat anxiety.

Unsigned surprise but not reward magnitude modulates the integration of motor elements during actions

It seems natural that motor responses unfold smoothly and that we are able to easily concatenate different components of movements to achieve goal-directed actions. Theoretical frameworks suggest that different motor features have to be bound to each other to achieve a coherent action. Yet, the nature of the "glue" (i.e., bindings) between elements constituting a motor sequence and enabling a smooth unfolding of motor acts is not well understood. We examined in how far motor feature bindings are affected by reward magnitude or the effects of an unsigned surprise signal. We show that the consistency of action file binding strength is modulated by unsigned surprise, but not by reward magnitude. On a conceptual and theoretical level, the results provide links between frameworks, which have until now not been brought into connection. In particular, theoretical accounts stating that only the unexpectedness (surprisingness) is essential for action control are connected to meta-control accounts of human action control.

Dynamic Feedback Between Antidepressant Placebo Expectancies and Mood

Importance

Despite high antidepressant placebo response rates, the mechanisms underlying the persistence of antidepressant placebo effects are still poorly understood.

Objective

To investigate the neurobehavioral mechanisms underlying the evolution of antidepressant placebo effects using a reinforcement learning (RL) framework.

Design, Setting, and Participants

In this acute within-patient cross-sectional study of antidepressant placebos, patients aged 18 to 55 years not receiving medication for major depressive disorder (MDD) were recruited at the University of Pittsburgh between February 21, 2017, to March 1, 2021.

Interventions

The antidepressant placebo functional magnetic resonance imaging task manipulates placebo-associated expectancies using visually cued fast-acting antidepressant infusions and controls their reinforcement with sham visual neurofeedback while assessing expected and experienced mood improvement.

Main Outcomes and Measures

The trial-by-trial evolution of expectancies and mood was examined using multilevel modeling and RL, relating model-predicted signals to spatiotemporal dynamics of blood oxygenation level-dependent (BOLD) response.

Results

A bayesian RL model comparison in 60 individuals (mean [SE] age, 24.5 [0.8] years; 51 females [85%]) with MDD revealed that antidepressant placebo trial-wise expectancies were updated by composite learning signals multiplexing sensory evidence (neurofeedback) and trial-wise mood (bayesian omnibus risk <0.001; exceedance probability = 97%). Placebo expectancy, neurofeedback manipulations, and composite learning signals modulated the visual cortex and dorsal attention network (threshold-free cluster enhancement [TFCE] = 1 - P >.95). As participants anticipated antidepressant infusions, learned placebo expectancies modulated the salience network (SN, TFCE = 1 - P >.95), positively scaling with depression severity.

Conclusions and Relevance

Results of this cross-sectional study suggest that on a timescale of minutes, antidepressant placebo effects were maintained by positive feedback loops between expectancies and mood improvement. During learning, representations of placebos and their perceived effects were enhanced in primary and secondary sensory cortices. Latent learned placebo expectancies were encoded in the SN.

From pupil to the brain: New insights for studying cortical plasticity through pupillometry

Pupil size variations have been associated with changes in brain activity patterns related with specific cognitive factors, such as arousal, attention, and mental effort. The locus coeruleus (LC), a key hub in the noradrenergic system of the brain, is considered to be a key regulator of cognitive control on pupil size, with changes in pupil diameter corresponding to the release of norepinephrine (NE). Advances in eye-tracking technology and open-source software have facilitated accurate pupil size measurement in various experimental settings, leading to increased interest in using pupillometry to track the nervous system activation state and as a potential biomarker for brain disorders. This review explores pupillometry as a non-invasive and fully translational tool for studying cortical plasticity starting from recent literature suggesting that pupillometry could be a promising technique for estimating the degree of residual plasticity in human subjects. Given that NE is known to be a critical mediator of cortical plasticity and arousal, the review includes data revealing the importance of the LC-NE system in modulating brain plasticity and pupil size. Finally, we will review data suggesting that pupillometry could provide a quantitative and complementary measure of cortical plasticity also in pre-clinical studies.

Integrating Nutrient Biomarkers, Cognitive Function, and Structural MRI Data to Build Multivariate Phenotypes of Healthy Aging

BACKGROUND

Research in the emerging field of Nutritional Cognitive Neuroscience demonstrates that many aspects of nutrition - from entire diets to specific nutrients - affect cognitive performance and brain health.

OBJECTIVE

While prior research has primarily examined the bivariate relationship between nutrition and cognition, or nutrition and brain health, the present study sought to investigate the joint relationship between these essential and interactive elements of human health.

METHODS

We applied a state-of-the-art data fusion method, Coupled Matrix Tensor Factorization, to characterize the joint association between measures of nutrition (52 nutrient biomarkers), cognition (Wechsler Abbreviated Test of Intelligence and Wechsler Memory Scale), and brain health (high-resolution Magnetic Resonance Imaging measures of structural brain volume) within a cross-sectional sample of 111 healthy older adults that had an average age of 69.1 years, were 62% female and had an average Body Mass Index of 26.0.

RESULTS

Data fusion uncovered 3 latent factors that capture the joint association between specific nutrient profiles, cognitive measures, and cortical volumes, demonstrating the respects in which these health domains are coupled. Hierarchical cluster analysis further revealed systematic differences between the observed latent factors, providing evidence for multivariate phenotypes that represent high versus low levels of performance across multiple health domains. The primary features that distinguish between each phenotype were: (i) nutrient biomarkers for monounsaturated and polyunsaturated fatty acids; (ii) cognitive measures of immediate, auditory, and delayed memory; and (iii) brain volumes within frontal, temporal, and parietal cortex.

CONCLUSIONS

By incorporating innovations in nutritional epidemiology (nutrient biomarker analysis), cognitive neuroscience (high-resolution structural brain imaging), and statistics (data fusion), the present study provides an interdisciplinary synthesis of methods that elucidate how nutrition, cognition, and brain health are integrated through lifestyle choices that affect healthy aging.

Anterior cingulate and medial prefrontal cortex oscillations underlie learning alterations in trait anxiety in humans

Anxiety has been linked to altered belief formation and uncertainty estimation, impacting learning. Identifying the neural processes underlying these changes is important for understanding brain pathology. Here, we show that oscillatory activity in the medial prefrontal, anterior cingulate and orbitofrontal cortex (mPFC, ACC, OFC) explains anxiety-related learning alterations. In a magnetoencephalography experiment, two groups of human participants pre-screened with high and low trait anxiety (HTA, LTA: 39) performed a probabilistic reward-based learning task. HTA undermined learning through an overestimation of volatility, leading to faster belief updating, more stochastic decisions and pronounced lose-shift tendencies. On a neural level, we observed increased gamma activity in the ACC, dmPFC, and OFC during encoding of precision-weighted prediction errors in HTA, accompanied by suppressed ACC alpha/beta activity. Our findings support the association between altered learning and belief updating in anxiety and changes in gamma and alpha/beta activity in the ACC, dmPFC, and OFC.

Modulation of Motor Vigor by Expectation of Reward Probability Trial-by-Trial Is Preserved in Healthy Ageing and Parkinson's Disease Patients

Motor improvements, such as faster movement times or increased velocity, have been associated with reward magnitude in deterministic contexts. Yet whether individual inferences on reward probability influence motor vigor dynamically remains undetermined. We investigated how dynamically inferring volatile action-reward contingencies modulated motor performance trial-by-trial. We conducted three studies that coupled a reversal learning paradigm with a motor sequence task and used a validated hierarchical Bayesian model to fit trial-by-trial data. In Study 1, we tested healthy younger [HYA; 37 (24 females)] and older adults [HOA; 37 (17 females)], and medicated Parkinson's disease (PD) patients [20 (7 females)]. We showed that stronger predictions about the tendency of the action-reward contingency led to faster performance tempo, commensurate with movement time, on a trial-by-trial basis without robustly modulating reaction time (RT). Using Bayesian linear mixed models, we demonstrated a similar invigoration effect on performance tempo in HYA, HOA, and PD, despite HOA and PD being slower than HYA. In Study 2 [HYA, 39 (29 females)], we additionally showed that retrospective subjective inference about credit assignment did not contribute to differences in motor vigor effects. Last, Study 3 [HYA, 33 (27 females)] revealed that explicit beliefs about the reward tendency (confidence ratings) modulated performance tempo trial-by-trial. Our study is the first to reveal that the dynamic updating of beliefs about volatile action-reward contingencies positively biases motor performance through faster tempo. We also provide robust evidence for a preserved sensitivity of motor vigor to inferences about the action-reward mapping in aging and medicated PD.SIGNIFICANCE STATEMENT Navigating a world rich in uncertainty relies on updating beliefs about the probability that our actions lead to reward. Here, we investigated how inferring the action-reward contingencies in a volatile environment modulated motor vigor trial-by-trial in healthy younger and older adults, and in Parkinson's disease (PD) patients on medication. We found an association between trial-by-trial predictions about the tendency of the action-reward contingency and performance tempo, with stronger expectations speeding the movement. We additionally provided evidence for a similar sensitivity of performance tempo to the strength of these predictions in all groups. Thus, dynamic beliefs about the changing relationship between actions and their outcome enhanced motor vigor. This positive bias was not compromised by age or Parkinson's disease.

Task-evoked pupillary responses track precision-weighted prediction errors and learning rate during interceptive visuomotor actions

In this study, we examined the relationship between physiological encoding of surprise and the learning of anticipatory eye movements. Active inference portrays perception and action as interconnected inference processes, driven by the imperative to minimise the surprise of sensory observations. To examine this characterisation of oculomotor learning during a hand-eye coordination task, we tested whether anticipatory eye movements were updated in accordance with Bayesian principles and whether trial-by-trial learning rates tracked pupil dilation as a marker of 'surprise'. Forty-four participants completed an interception task in immersive virtual reality that required them to hit bouncing balls that had either expected or unexpected bounce profiles. We recorded anticipatory eye movements known to index participants' beliefs about likely ball bounce trajectories. By fitting a hierarchical Bayesian inference model to the trial-wise trajectories of these predictive eye movements, we were able to estimate each individual's expectations about bounce trajectories, rates of belief updating, and precision-weighted prediction errors. We found that the task-evoked pupil response tracked prediction errors and learning rates but not beliefs about ball bounciness or environmental volatility. These findings are partially consistent with active inference accounts and shed light on how encoding of surprise may shape the control of action.

Neural implementation of computational mechanisms underlying the continuous trade-off between cooperation and competition

Social interactions evolve continuously. Sometimes we cooperate, sometimes we compete, while at other times we strategically position ourselves somewhere in between to account for the ever-changing social contexts around us. Research on social interactions often focuses on a binary dichotomy between competition and cooperation, ignoring people's evolving shifts along a continuum. Here, we develop an economic game - the Space Dilemma - where two players change their degree of cooperativeness over time in cooperative and competitive contexts. Using computational modelling we show how social contexts bias choices and characterise how inferences about others' intentions modulate cooperativeness. Consistent with the modelling predictions, brain regions previously linked to social cognition, including the temporo-parietal junction, dorso-medial prefrontal cortex and the anterior cingulate gyrus, encode social prediction errors and context-dependent signals, correlating with shifts along a cooperation-competition continuum. These results provide a comprehensive account of the computational and neural mechanisms underlying the continuous trade-off between cooperation and competition.

The PRO model accounts for the anterior cingulate cortex role in risky decision-making and monitoring

The anterior cingulate cortex (ACC) has been implicated in a number of functions, including performance monitoring and decision-making involving effort. The prediction of responses and outcomes (PRO) model has provided a unified account of much human and monkey ACC data involving anatomy, neurophysiology, EEG, fMRI, and behavior. We explored the computational nature of ACC with the PRO model, extending it to account specifically for both human and macaque monkey decision-making under risk, including both behavioral and neural data. We show that the PRO model can account for a number of additional effects related to outcome prediction, decision-making under risk, gambling behavior. In particular, we show that the ACC represents the variance of uncertain outcomes, suggesting a link between ACC function and mean-variance theories of decision making. The PRO model provides a unified account of a large set of data regarding the ACC.

Neural signals implicated in the processing of appetitive and aversive events in social and non-social contexts

In 2014, we participated in a special issue of Frontiers examining the neural processing of appetitive and aversive events. Specifically, we reviewed brain areas that contribute to the encoding of prediction errors and value versus salience, attention and motivation. Further, we described how we disambiguated these cognitive processes and their neural substrates by using paradigms that incorporate both appetitive and aversive stimuli. We described a circuit in which the orbitofrontal cortex (OFC) signals expected value and the basolateral amygdala (BLA) encodes the salience and valence of both appetitive and aversive events. This information is integrated by the nucleus accumbens (NAc) and dopaminergic (DA) signaling in order to generate prediction and prediction error signals, which guide decision-making and learning via the dorsal striatum (DS). Lastly, the anterior cingulate cortex (ACC) is monitoring actions and outcomes, and signals the need to engage attentional control in order to optimize behavioral output. Here, we expand upon this framework, and review our recent work in which within-task manipulations of both appetitive and aversive stimuli allow us to uncover the neural processes that contribute to the detection of outcomes delivered to a conspecific and behaviors in social contexts. Specifically, we discuss the involvement of single-unit firing in the ACC and DA signals in the NAc during the processing of appetitive and aversive events in both social and non-social contexts.

Role of anterior cingulate cortex inputs to periaqueductal gray for pain avoidance

Although pain-related excessive fear is known to be a key factor in chronic pain disability, which involves the anterior cingulate cortex (ACC), little is known about the downstream circuits of the ACC for fear avoidance in pain processing. Using behavioral experiments and functional magnetic resonance imaging with optogenetics at 15.2 T, we demonstrate that the ACC is a part of the abnormal circuit changes in chronic pain and its downstream circuits are closely related to modulating sensorimotor integration and generating active movement rather than carrying sensory information. The projection from the ACC to the dorsolateral and lateral parts of the periaqueductal gray (dl/lPAG) especially enhances both reflexive and active avoidance behavior toward pain. Collectively, our results indicate that increased signals from the ACC to the dl/lPAG might be critical for excessive fear avoidance in chronic pain disability.

Highly unstable heterogeneous representations in VIP interneurons of the anterior cingulate cortex

A hallmark of the anterior cingulate cortex (ACC) is its functional heterogeneity. Functional and imaging studies revealed its importance in the encoding of anxiety-related and social stimuli, but it is unknown how microcircuits within the ACC encode these distinct stimuli. One type of inhibitory interneuron, which is positive for vasoactive intestinal peptide (VIP), is known to modulate the activity of pyramidal cells in local microcircuits, but it is unknown whether VIP cells in the ACC (VIPACC) are engaged by particular contexts or stimuli. Additionally, recent studies demonstrated that neuronal representations in other cortical areas can change over time at the level of the individual neuron. However, it is not known whether stimulus representations in the ACC remain stable over time. Using in vivo Ca2+ imaging and miniscopes in freely behaving mice to monitor neuronal activity with cellular resolution, we identified individual VIPACC that preferentially activated to distinct stimuli across diverse tasks. Importantly, although the population-level activity of the VIPACC remained stable across trials, the stimulus-selectivity of individual interneurons changed rapidly. These findings demonstrate marked functional heterogeneity and instability within interneuron populations in the ACC. This work contributes to our understanding of how the cortex encodes information across diverse contexts and provides insight into the complexity of neural processes involved in anxiety and social behavior.

Reward signals in the cerebellum: origins, targets, and functional implications

The cerebellum has long been proposed to play a role in cognitive function, although this has remained controversial. This idea has received renewed support with the recent discovery that signals associated with reward can be observed in the cerebellar circuitry, particularly in goal-directed learning tasks involving an interplay between the cerebellar cortex, basal ganglia, and cerebral cortex. Remarkably, a wide range of reward contingencies-including reward expectation, delivery, size, and omission-can be encoded by specific circuit elements in a manner that reflects the microzonal organization of the cerebellar cortex. The facts that reward signals have been observed in both the mossy fiber and climbing fiber input pathways to the cerebellar cortex and that their convergence may trigger plasticity in Purkinje cells suggest that these interactions may be crucial for the role of the cerebellar cortex in learned behavior. These findings strengthen the emerging consensus that the cerebellum plays a pivotal role in shaping cognitive processing and suggest that the cerebellum may combine both supervised learning and reinforcement learning to optimize goal-directed action. We make specific predictions about how cerebellar circuits can work in concert with the basal ganglia to guide different stages of learning.

Context-dependent relationships between locus coeruleus firing patterns and coordinated neural activity in the anterior cingulate cortex

Ascending neuromodulatory projections from the locus coeruleus (LC) affect cortical neural networks via the release of norepinephrine (NE). However, the exact nature of these neuromodulatory effects on neural activity patterns in vivo is not well understood. Here, we show that in awake monkeys, LC activation is associated with changes in coordinated activity patterns in the anterior cingulate cortex (ACC). These relationships, which are largely independent of changes in firing rates of individual ACC neurons, depend on the type of LC activation: ACC pairwise correlations tend to be reduced when ongoing (baseline) LC activity increases but enhanced when external events evoke transient LC responses. Both relationships covary with pupil changes that reflect LC activation and arousal. These results suggest that modulations of information processing that reflect changes in coordinated activity patterns in cortical networks can result partly from ongoing, context-dependent, arousal-related changes in activation of the LC-NE system.

Interactions between ventrolateral prefrontal and anterior cingulate cortex during learning and behavioural change

Hypotheses and beliefs guide credit assignment - the process of determining which previous events or actions caused an outcome. Adaptive hypothesis formation and testing are crucial in uncertain and changing environments in which associations and meanings are volatile. Despite primates' abilities to form and test hypotheses, establishing what is causally responsible for the occurrence of particular outcomes remains a fundamental challenge for credit assignment and learning. Hypotheses about what surprises are due to stochasticity inherent in an environment as opposed to real, systematic changes are necessary for identifying the environment's predictive features, but are often hard to test. We review evidence that two highly interconnected frontal cortical regions, anterior cingulate cortex and ventrolateral prefrontal area 47/12o, provide a biological substrate for linking two crucial components of hypothesis-formation and testing: the control of information seeking and credit assignment. Neuroimaging, targeted disruptions, and neurophysiological studies link an anterior cingulate - 47/12o circuit to generation of exploratory behaviour, non-instrumental information seeking, and interpretation of subsequent feedback in the service of credit assignment. Our observations support the idea that information seeking and credit assignment are linked at the level of neural circuits and explain why this circuit is important for ensuring behaviour is flexible and adaptive.

Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior

Efficient foraging is essential to survival and depends on frontal cortex in mammals. Because of its role in psychiatric disorders, frontal cortex and its contributions to reward procurement have been studied extensively in both rodents and non-human primates. How frontal cortex of these animal models compares is a source of intense debate. Here we argue that translating findings from rodents to non-human primates requires an appreciation of both the niche in which each animal forages as well as the similarities in frontal cortex anatomy and function. Consequently, we highlight similarities and differences in behavior and anatomy, before focusing on points of convergence in how parts of frontal cortex contribute to distinct aspects of foraging in rats and macaques, more specifically. In doing so, our aim is to emphasize where translation of frontal cortex function between species is clearer, where there is divergence, and where future work should focus. We finish by highlighting aspects of foraging for which have received less attention but we believe are critical to uncovering how frontal cortex promotes survival in each species.

Computational models of adaptive behavior and prefrontal cortex

The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus-action associations through rewards; (2) predictive models learning stimulus- and/or action-outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.

Event-based control of autonomic and emotional states by the anterior cingulate cortex

Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.

Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate

Learning to predict rewards is essential for the sustained fitness of animals. Contemporary views suggest that such learning is driven by a reward prediction error (RPE)-the difference between received and predicted rewards. The magnitude of learning induced by an RPE is proportional to the product of the RPE and a learning rate. Here we demonstrate using two-photon calcium imaging and optogenetics in mice that certain functionally distinct subpopulations of ventral/medial orbitofrontal cortex (vmOFC) neurons signal learning rate control. Consistent with learning rate control, trial-by-trial fluctuations in vmOFC activity positively correlate with behavioral updating when the RPE is positive, and negatively correlates with behavioral updating when the RPE is negative. Learning rate is affected by many variables including the salience of a reward. We found that the average reward response of these neurons signals the relative salience of a reward, because it decreases after reward prediction learning or the introduction of another highly salient aversive stimulus. The relative salience signaling in vmOFC is sculpted by medial thalamic inputs. These results support emerging theoretical views that prefrontal cortex encodes and controls learning parameters.

Anterior cingulate cortex is necessary for spontaneous opioid withdrawal and withdrawal-induced hyperalgesia in male mice

The anterior cingulate cortex (ACC) is implicated in many pathologies, including depression, anxiety, substance-use disorders, and pain. There is also evidence from brain imaging that the ACC is hyperactive during periods of opioid withdrawal. However, there are limited data contributing to our understanding of ACC function at the cellular level during opioid withdrawal. Here, we address this issue by performing ex vivo electrophysiological analysis of thick-tufted, putative dopamine D2 receptor expressing, layer V pyramidal neurons in the ACC (ACC L5 PyNs) in a mouse model of spontaneous opioid withdrawal. We found that escalating doses of morphine (20, 40, 60, 80, and 100 mg/kg, i.p. on days 1-5, respectively) injected twice daily into male C57BL/6 mice evoked withdrawal behaviors and an associated withdrawal-induced mechanical hypersensitivity. Brain slices prepared 24 h following the last morphine injection showed increases in ACC L5 thick-tufted PyN-intrinsic membrane excitability, increases in membrane resistance, reductions in the rheobase, and reductions in HCN channel-mediated currents (I_H). We did not observe changes in intrinsic or synaptic properties on thin-tufted, dopamine D1-receptor-expressing ACC L5 PyNs recorded from male Drd1a-tdTomato transgenic mice. In addition, we found that chemogenetic inhibition of the ACC blocked opioid-induced withdrawal and withdrawal-induced mechanical hypersensitivity. These results demonstrate that spontaneous opioid withdrawal alters neuronal properties within the ACC and that ACC activity is necessary to control behaviors associated with opioid withdrawal and withdrawal-induced mechanical hypersensitivity. The ability of the ACC to regulate both withdrawal behaviors and withdrawal-induced mechanical hypersensitivity suggests overlapping mechanisms between two seemingly distinguishable behaviors. This commonality potentially suggests that the ACC is a locus for multiple withdrawal symptoms.

Characterizing anhedonia: A systematic review of neuroimaging across the subtypes of reward processing deficits in depression

Anhedonia is a key symptom of major depressive disorder (MDD) and comprises behavioural deficits in three reward processing subtypes: reward liking, reward wanting, and reward learning. However, neuroimaging findings regarding the neural abnormalities underpinning these deficits are complex. We have conducted a systematic review to update, reframe and summarize neuroimaging findings across the three subtypes of anhedonia in MDD. Using PubMed, The Cochrane Library, PsycINFO, and Web of Science databases, we identified 59 fMRI studies comparing participants with current or remitted MDD with controls, using reward processing tasks. For reward liking and wanting, striatal hypoactivation was observed, alongside hypoactivation and hyperactivation across frontal regions. For reward learning, blunted frontostriatal sensitivity to positive feedback was observed. These findings highlight the importance of studying anhedonia not only as a clinical manifestation but also as a neurobiological mechanism underlying depressive disorder and other broader psychiatric conditions.

Brain activations associated with anticipation and delivery of monetary reward: A systematic review and meta-analysis of fMRI studies

BACKGROUND

While multiple studies have examined the brain functional correlates of reward, meta-analyses have either focused on studies using the monetary incentive delay (MID) task, or have adopted a broad strategy, combining data from studies using both monetary and non-monetary reward, as probed using a wide range of tasks.

OBJECTIVE

To meta-analyze fMRI studies that used monetary reward and in which there was a definable cue-reward contingency. Studies were limited to those using monetary reward in order to avoid potential heterogeneity from use of other rewards, especially social rewards. Studies using gambling or delay discounting tasks were excluded on the grounds that reward anticipation is not easily quantifiable.

STUDY ELIGIBILITY

English-language fMRI studies (i) that reported fMRI findings on healthy adults; (ii) that used monetary reward; and (iii) in which a cue that was predictive of reward was compared to a no win (or lesser win) condition. Only voxel-based studies were included; those where brain coverage was incomplete were excluded.

DATA SOURCES

Ovid, Medline and PsycInfo, from 2000 to 2020, plus checking of review articles and meta-analyses.

DATA SYNTHESIS

Data were pooled using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI). Heterogeneity among studies was examined using the I2 statistic. Publication bias was examined using funnel plots and statistical examination of asymmetries. Moderator variables including whether the task was pre-learnt, sex distribution, amount of money won and width of smoothing kernel were examined.

RESULTS

Pooled data from 45 studies of reward anticipation revealed activations in the ventral striatum, the middle cingulate cortex/supplementary motor area and the insula. Pooled data from 28 studies of reward delivery again revealed ventral striatal activation, plus cortical activations in the anterior and posterior cingulate cortex. There was relatively little evidence of publication bias. Among moderating variables, only whether the task was pre-learnt exerted an influence.

CONCLUSIONS

According to this meta-analysis monetary reward anticipation and delivery both activate the ventral but not the dorsal striatum, and are associated with different patterns of cortical activation.

Anterior cingulate gyrus acts as a moderator of the relationship between problematic mobile phone use and depressive symptoms in college students

This study aimed to investigate the brain grey matter volume (GMV) related to problematic mobile phone use (PMPU), and whether these regions of GMV play a potential moderating role in the relationship between PMPU and depressive symptoms. We recruited 266 students who underwent magnetic resonance imaging (MRI) scanning. PMPU and depressive symptoms were assessed by a self-rating questionnaire for adolescent PMPU and patient health questionnaire-9, respectively. A multiple regression model was performed to detect GMV and white matter (WM) integrity associated with PMPU by voxel-based morphometry (VBM) and tract-based spatial statistics (TBSS) methods, and the moderating analysis was conducted by PROCESS using SPSS software. VBM analysis found an inverse correlation between the GMV of the anterior cingulate gyrus (ACC) and right fusiform gyrus (FFG) with PMPU (P_FDR < 0.05), and TBSS analysis revealed that fractional anisotropy (FA) in the body of the corpus callosum was negatively correlated with PMPU. The correlation between PMPU and depressive symptoms was moderated by the GMV of the ACC. These results suggest that the GMV of the ACC and right FFG, as well as FA in the body of the corpus callosum, was related to PMPU, and we further found that increased GMV of the ACC could reduce the relationship between PMPU and depressive symptoms in college students.

Controlling one's world: Identification of sub-regions of primate PFC underlying goal-directed behavior

Impaired detection of causal relationships between actions and their outcomes can lead to maladaptive behavior. However, causal roles of specific prefrontal cortex (PFC) sub-regions and the caudate nucleus in mediating such relationships in primates are unclear. We inactivated and overactivated five PFC sub-regions, reversibly and pharmacologically: areas 24 (perigenual anterior cingulate cortex), 32 (medial PFC), 11 (anterior orbitofrontal cortex, OFC), 14 (rostral ventromedial PFC/medial OFC), and 14-25 (caudal ventromedial PFC) and the anteromedial caudate to examine their role in expressing learned action-outcome contingencies using a contingency degradation paradigm in marmoset monkeys. Area 24 or caudate inactivation impaired the response to contingency change, while area 11 inactivation enhanced it, and inactivation of areas 14, 32, or 14-25 had no effect. Overactivation of areas 11 and 24 impaired this response. These findings demonstrate the distinct roles of PFC sub-regions in goal-directed behavior and illuminate the candidate neurobehavioral substrates of psychiatric disorders, including obsessive-compulsive disorder.

•

Monkey pgACC-24 is necessary for detecting causal control of actions over outcomes

•

Its projection target in the caudate nucleus is also implicated

•

Three other subregions of the ventromedial prefrontal cortex are not necessary

•

Anterior OFC-11 may mediate Pavlovian influences on goal-directed behavior

Anterior Cingulate Cortex Lesions Abolish Budget Effects on Effort-Based Decision-Making in Rat Consumers

Demand theory can be applied to analyze how animal consumers change their selection of commodities in response to changes in commodity prices, given budget constraints. Previous work has shown that demand elasticities in rats differed between uncompensated budget conditions in which the budget available to be spent on the commodities (e.g., the finite number of discrete operants to "purchase" rewards in two-alternative fixed-ratio schedules) was kept constant, and compensated budget conditions in which the budget was adjusted so that consumers could potentially continue to obtain the original reward bundles. Here, we hypothesized that rat anterior cingulate cortex (ACC) was necessary to produce this budget effect on demand elasticities. We applied excitotoxic or sham lesions to ACC in rats performing an effort task in which the prices of liquid vanilla or chocolate rewards (the effort required to obtain rewards) and the budget (the total number of operants) was manipulated. When reward prices changed, and the budget was compensated, all rats adjusted their demand for chocolate and vanilla accordingly. In sham-lesioned rats, changes in demand were even more pronounced when the budget was not compensated for the price changes. By contrast, ACC-lesioned animals did not show this additional budget effect. An in-depth comparison of the rats' choice patterns showed that, unlike sham rats, ACC-lesioned animals failed to maximize session-bundle utility after price/budget changes, revealing deficits in higher-order choice-strategy adaptations. Our results suggest a novel role of ACC in considering purchasing power during complex cost-benefit value computations.SIGNIFICANCE STATEMENT Anterior cingulate cortex (ACC) is important for allocating effort in cost-benefit calculations in animals and humans. Economic theory prescribes that the value of the costs in cost-benefit analyses not only depends on the net nominal costs required to purchase a reward, but also on the available budget resources, i.e., on the budget's "purchasing value." We asked whether ACC, a region implicated in effort-based decision-making and reward comparisons, is required for computing the value of effort relative to a budget constraint. Applying demand theory to describe rat choices in a rodent effort allocation task with varying effort prices and budgets, we show that ACC integrity was necessary for computing purchasing power, a core variable in economic choice theory.