Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner

The impact of subclinical psychotic symptoms on delay and effort discounting: Insights from behavioral, computational, and electrophysiological methods

BACKGROUND

The ability to value rewards is crucial for adaptive behavior and is influenced by the time and effort required to obtain them. Impairments in these computations have been observed in patients with schizophrenia and may be present in individuals with subclinical psychotic symptoms (PS).

METHODS

In this study, we employed delay and effort-discounting tasks with food rewards in thirty-nine participants divided into high and low levels of PS. We investigated the underlying mechanisms of effort-discounting through computational modelling of dopamine prefrontal and subcortical circuits and the electrophysiological biomarker of both delay and effort-discounting alterations through resting-state frontal alpha asymmetry (FAA).

RESULTS

Results revealed greater delay discounting in the High PS group compared to the Low PS group but no differences in the effort discounting task. However, in this task, the same levels of estimated dopamine release were associated with a lower willingness to exert effort for high-calorie food rewards in High PS participants compared to Low PS participants. Although there were no significant differences in FAA between the High PS and Low PS groups, FAA was significantly associated with the severity of participants' negative symptoms.

CONCLUSIONS

Our study suggests that the dysfunction in temporal and effort cost computations, seen in patients with schizophrenia, may be present in individuals with subclinical PS. These findings provide valuable insight into the early vulnerability markers (behavioral, computational, and electrophysiological) for psychosis, which may aid in the development of preventive interventions. These findings are preliminary and warrant further investigation.

Mind the instructions: Reward cues are liked first, wanted later

Current theories propose that mental effort is invested only when the anticipated benefits, such as rewards, outweigh the associated costs, like task difficulty. Yet, it remains unclear whether this motivational and mitigating aspect of reward processing is reflected in the evaluation of reward/difficulty cues as such, and to what extent it depends on task experience. In a pre-registered experiment (N = 84), we used the affect misattribution procedure (AMP) to gauge affective evaluations of nonword cues predicting reward and task difficulty levels. Contrary to previous studies, the AMP was administered at the outset, after cue instructions, and after the cues were used in a random dot motion (RDM) task. Compared to baseline, cues predicting a larger reward were evaluated more positively after RDM task experience, and most importantly, already after cue instructions, with no difference between the two phases. This evaluative effect manifested in increased performance after larger reward cues in the RDM task. Our results suggest that AMP effects may generally capture performance expectations which are independent of task experience. Importantly, these instructed expectations of reward and difficulty play a crucial role in the evaluation and subsequent investment of mental effort.

Methamphetamine-induced adaptation of learning rate dynamics depend on baseline performance

The ability to calibrate learning according to new information is a fundamental component of an organism's ability to adapt to changing conditions. Yet, the exact neural mechanisms guiding dynamic learning rate adjustments remain unclear. Catecholamines appear to play a critical role in adjusting the degree to which we use new information over time, but individuals vary widely in the manner in which they adjust to changes. Here, we studied the effects of a low dose of methamphetamine (MA), and individual differences in these effects, on probabilistic reversal learning dynamics in a within-subject, double-blind, randomized design. Participants first completed a reversal learning task during a drug-free baseline session to provide a measure of baseline performance. Then they completed the task during two sessions, one with MA (20 mg oral) and one with placebo (PL). First, we showed that, relative to PL, MA modulates the ability to dynamically adjust learning from prediction errors. Second, this effect was more pronounced in participants who performed poorly at baseline. These results present novel evidence for the involvement of catecholaminergic transmission on learning flexibility and highlights that baseline performance modulates the effect of the drug.

Anterior Cingulate Cortex Causally Supports Meta-Learning

In dynamic environments with volatile rewards the anterior cingulate cortex (ACC) is believed to determine whether a visual object is relevant and should be chosen. The ACC may achieve this by integrating reward information over time to estimate which objects are worth to explore and which objects should be avoided. Such a higher-order meta-awareness about which objects should be explored predicts that the ACC causally contributes to choices when the reward values of objects are unknown and must be inferred from ongoing exploration. We tested this suggestion in nonhuman primates using a learning task that varied the number of object features that could be relevant, and by controlling the motivational value of choosing objects. During learning the ACC was transiently micro-stimulated when subjects foveated the to-be-chosen stimulus. We found that stimulation selectively impaired learning when feature uncertainty and motivational value of choices were high, which was linked to a deficit in using reward outcomes for feature-specific credit assignment. Application of an adaptive reinforcement learning model confirmed a primary deficit in weighting prediction errors that led to a meta-learning impairment to adaptively increase exploration during learning and to an impaired use of working memory to support learning. These findings provide causal evidence that the reward history traces in ACC are essential for meta-adjusting the exploration-exploitation balance and the strength of working memory of object values during adaptive behavior.

Task performance errors and rewards affect voluntary task choices

Humans are remarkably flexible in adapting their behavior to current demands. It has been suggested that the decision which of multiple tasks to perform is based on a variety of factors pertaining to the rewards associated with each task as well as task performance (e.g., error rates associated with each task and/or error commission on the previous trial). However, further empirical investigation is needed to examine whether task performance still influences task choices if task choices are rewarded but task performance is not. Accordingly, we exposed participants to a novel reward-varying voluntary task switching paradigm where the reward for the performed task gradually decreased while the reward associated for the alternative task was unchanged. Importantly, we rewarded participants' task choices before participants performed the task to investigate the effect of rewards independent from task performance. We examined the effect of (i) reward, (ii) error rates associated with each of the two tasks, and (iii) error commission in the previous trial on voluntary task choices. As expected, we found that participants' task selection was influenced by reward differences between task choices. In addition, error rates associated with a task also influenced task selection, with participants requiring larger reward differences to switch to a task associated with relatively higher error rates, compared to switching to a task with relatively lower error rates. However, errors in n - 1 did not influence participants' probability to switch to the alternative task. These findings contribute to an ongoing discussion on the influence of task performance on task selection.

Learning environment-specific learning rates

People often have to switch back and forth between different environments that come with different problems and volatilities. While volatile environments require fast learning (i.e., high learning rates), stable environments call for lower learning rates. Previous studies have shown that people adapt their learning rates, but it remains unclear whether they can also learn about environment-specific learning rates, and instantaneously retrieve them when revisiting environments. Here, using optimality simulations and hierarchical Bayesian analyses across three experiments, we show that people can learn to use different learning rates when switching back and forth between two different environments. We even observe a signature of these environment-specific learning rates when the volatility of both environments is suddenly the same. We conclude that humans can flexibly adapt and learn to associate different learning rates to different environments, offering important insights for developing theories of meta-learning and context-specific control.

Temperament and probabilistic predictive coding in visual-spatial attention

Cholinergic (Ach), Noradrenergic (NE), and Dopaminergic (DA) pathways play an important role in the regulation of spatial attention. The same neurotransmitters are also responsible for inter-individual differences in temperamental traits. Here we explored whether biologically defined temperamental traits determine differences in the ability to orient spatial attention as a function of the probabilistic association between cues and targets. To this aim, we administered the Structure of Temperament Questionnaire (STQ-77) to a sample of 151 participants who also performed a Posner task with central endogenous predictive (80 % valid/20 % invalid) or non-predictive cues (50 % valid/50 % invalid). We found that only participants with high scores in Plasticity and Intellectual Endurance showed a selective abatement of attentional costs with non-predictive cues. In addition, stepwise regression showed that costs in the non-predictive condition were negatively predicted by scores in Plasticity and positively predicted by scores in Probabilistic Thinking. These results show that stable temperamental characteristics play an important role in defining the inter-individual differences in attentional behaviour, especially in the presence of different probabilistic organisations of the sensory environment. These findings emphasize the importance of considering temperamental and personality traits in social and professional environments where the ability to control one's attention is a crucial functional skill.

Policy abstraction as a predictor of cognitive effort avoidance

Consistent evidence has established that people avoid cognitively effortful tasks. However, the features that make a task cognitively effortful are still not well understood. Multiple hypotheses have been proposed regarding which task demands underlie cognitive effort costs, such as time-on-task, error likelihood, and the general engagement of cognitive control. In this study, we test the novel hypothesis that tasks requiring behavior according to higher degrees of policy abstraction are experienced as more effortful. Accordingly, policy abstraction, operationalized as the levels of contextual contingency required by task rules, drives task avoidance over and above the effects of task performance, such as time-on-task or error likelihood. To test this hypothesis, we combined two previously established cognitive control tasks that parametrically manipulated policy abstraction with the demand selection task procedure. The design of these tasks allowed us to test whether people avoided tasks with higher order policy abstraction while controlling for the contribution of factors such as time-on-task and expected error rate (ER). Consistent with our hypothesis, we observed that policy abstraction was the strongest predictor of cognitive effort choices, followed by ER. This was evident across both studies and in a within-subject cross-study analysis. These results establish at least one task feature independent of performance, which is predictive of task avoidance behavior. We interpret these results within an opportunity cost framework for understanding aversive experiences of cognitive effort while performing a task. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Neural activation of regions involved in food reward and cognitive control in young females with anorexia nervosa and atypical anorexia nervosa versus healthy controls

Anorexia nervosa (AN) and atypical AN (AtypAN) are complex neurobiological illnesses that typically onset in adolescence with an often treatment-refractory and chronic illness trajectory. Aberrant eating behaviors in this population have been linked to abnormalities in food reward and cognitive control, but prior studies have not examined respective contributions of clinical characteristics and metabolic state. Research is needed to identify specific disruptions and inform novel intervention targets to improve outcomes. Fifty-nine females with AN (n = 34) or AtypAN (n = 25), ages 10-22 years, all ≤90% expected body weight, and 34 age-matched healthy controls (HC) completed a well-established neuroimaging food cue paradigm fasting and after a standardized meal, and we used ANCOVA models to investigate main and interaction effects of Group and Appetitive State on blood oxygenation level-dependent (BOLD) activation for the contrast of exposure to high-calorie food images minus objects. We found main effects of Group with greater BOLD activation in the dorsal anterior cingulate cortex (dACC), dorsolateral prefrontal cortex (DLPFC), hippocampus, caudate, and putamen for AN/AtypAN versus HC groups, and in the three-group model including AN, AtypAN, and HC (sub-)groups, where differences were primarily driven by greater activation in the AtypAN subgroup versus HC group. We found a main effect of Appetitive State with increased premeal BOLD activation in the hypothalamus, amygdala, nucleus accumbens, and caudate for models that included AN/AtypAN and HC groups, and in BOLD activation in the nucleus accumbens for the model that included AN, AtypAN, and HC (sub-)groups. There were no interaction effects of Group with Appetitive State for any of the models. Our findings demonstrate robust feeding-state independent group effects reflecting greater neural activation of specific regions typically associated with reward and cognitive control processing across AN and AtypAN relative to healthy individuals in this food cue paradigm. Differential activation of specific brain regions in response to the passive viewing of high-calorie food images may underlie restrictive eating behavior in this clinical population.

More than a feeling: physiological measures of affect index the integration of effort costs and rewards during anticipatory effort evaluation

The notion that humans avoid effortful action is one of the oldest and most persistent in psychology. Influential theories of effort propose that effort valuations are made according to a cost-benefit trade-off: we tend to invest mental effort only when the benefits outweigh the costs. While these models provide a useful conceptual framework, the affective components of effort valuation remain poorly understood. Here, we examined whether primitive components of affective response-positive and negative valence, captured via facial electromyography (fEMG)-can be used to better understand valuations of cognitive effort. Using an effortful arithmetic task, we find that fEMG activity in the corrugator supercilii-thought to index negative valence-1) tracks the anticipation and exertion of cognitive effort and 2) is attenuated in the presence of high rewards. Together, these results suggest that activity in the corrugator reflects the integration of effort costs and rewards during effortful decision-making.

The Neurodegenerative Elderly Syndrome (NES) hypothesis: Alzheimer and Parkinson are two faces of the same disease

Increasing evidence suggests that Alzheimer's disease (AD) and Parkinson's disease (PD) share monoamine and alpha-synuclein (αSyn) dysfunctions, often beginning years before clinical manifestations onset. The triggers for these impairments and the causes leading these early neurodegenerative processes to become AD or PD remain unclear. We address these issues by proposing a radically new perspective to frame AD and PD: they are different manifestations of one only disease we call "Neurodegenerative Elderly Syndrome (NES)". NES goes through three phases. The seeding stage, which starts years before clinical signs, and where the part of the brain-body affected by the initial αSyn and monoamine dysfunctions, influences the future possible progression of NES towards PD or AD. The compensatory stage, where the clinical symptoms are still silent thanks to compensatory mechanisms keeping monoamine concentrations homeostasis. The bifurcation stage, where NES becomes AD or PD. We present recent literature supporting NES and discuss how this hypothesis could radically change the comprehension of AD and PD comorbidities and the design of novel system-level diagnostic and therapeutic actions.

Uncertainty modulates visual maps during noninstrumental information demand

Animals are intrinsically motivated to obtain information independently of instrumental incentives. This motivation depends on two factors: a desire to resolve uncertainty by gathering accurate information and a desire to obtain positively-valenced observations, which predict favorable rather than unfavorable outcomes. To understand the neural mechanisms, we recorded parietal cortical activity implicated in prioritizing stimuli for spatial attention and gaze, in a task in which monkeys were free (but not trained) to obtain information about probabilistic non-contingent rewards. We show that valence and uncertainty independently modulated parietal neuronal activity, and uncertainty but not reward-related enhancement consistently correlated with behavioral sensitivity. The findings suggest uncertainty-driven and valence-driven information demand depend on partially distinct pathways, with the former being consistently related to parietal responses and the latter depending on additional mechanisms implemented in downstream structures.

Curiosity is motivated by uncertainty and valence, but how uncertainty and valence are encoded in the brain remains poorly understood. Here, the authors show that parietal neurons are enhanced by both factors, but that they specifically predict visual information seeking based on uncertainty.

Distinctive types of aversiveness are represented as the same in a portion of the dorsal anterior cingulate cortex: An fMRI study with the cue paradigm

Some studies have argued that the dorsal anterior cingulate cortex (dACC) is generally activated in response to aversive information, including pain, negative affect, and cognitive conflict. Other studies have claimed that the dACC has subdivisions, and each division has a specific function. By manipulating emotionally and cognitively aversive cues, the present study determined whether the dACC is generally responsive to aversiveness or it has subdivisions for specific forms of aversiveness. Conjunction functional magnetic resonance imaging (fMRI) analysis showed that emotionally and cognitively aversive cues activated the same portion of the dACC. When these cues were contiguously presented, the region demonstrated additive activity, further supporting the overlapping representation of the two different forms of aversiveness in the dACC. Additional effective connectivity analysis showed that the dACC was co-activated with different brain regions depending on the cue type, characterizing its behavioral control mechanism. Complementary multivariate analyses showed that the reaction time was negatively correlated with the activity of the dACC and that the activity of the dACC under the emotional cue was predicted by the individual state anxiety score but not under the cognitive cue. We also found that the superior part of the dACC was uniquely activated in response to cognitively aversive cues, partially supporting the functional segregation account. Collectively, our results provide evidence that the specific locus of the dACC is generally responsive to distinctive motivational information, whereas the other loci may have segregated functions. Discussion includes recent neurocomputational theories that seem to satisfactorily account for the present results.

Left and right temporal-parietal junctions (TPJs) as "match/mismatch" hedonic machines: A unifying account of TPJ function

Experimental and theoretical studies have tried to gain insights into the involvement of the Temporal Parietal Junction (TPJ) in a broad range of cognitive functions like memory, attention, language, self-agency and theory of mind. Recent investigations have demonstrated the partition of the TPJ in discrete subsectors. Nonetheless, whether these subsectors play different roles or implement an overarching function remains debated. Here, based on a review of available evidence, we propose that the left TPJ codes both matches and mismatches between expected and actual sensory, motor, or cognitive events while the right TPJ codes mismatches. These operations help keeping track of statistical contingencies in personal, environmental, and conceptual space. We show that this hypothesis can account for the participation of the TPJ in disparate cognitive functions, including "humour", and explain: a) the higher incidence of spatial neglect in right brain damage; b) the different emotional reactions that follow left and right brain damage; c) the hemispheric lateralisation of optimistic bias mechanisms; d) the lateralisation of mechanisms that regulate routine and novelty behaviours. We propose that match and mismatch operations are aimed at approximating "free energy", in terms of the free energy principle of decision-making. By approximating "free energy", the match/mismatch TPJ system supports both information seeking to update one's own beliefs and the pleasure of being right in one's own' current choices. This renewed view of the TPJ has relevant clinical implications because the misfunctioning of TPJ-related "match" and "mismatch" circuits in unilateral brain damage can produce low-dimensional deficits of active-inference and predictive coding that can be associated with different neuropsychological disorders.

A reference-based theory of motivation and effort allocation

Motivation is key for performance in domains such as work, sport, and learning. Research has established that motivation and the willingness to invest effort generally increase as a function of reward. However, this view struggles to explain some empirical observations-for example, in the domain of sport, athletes sometimes appear to lose motivation when playing against weak opponents-this despite objective rewards being high. This and similar evidence highlight the role of subjective value in motivation and effort allocation. To capture this, here, we advance a novel theory and computational model where motivation and effort allocation arise from reference-based evaluation processes. Our proposal argues that motivation (and the ensuing willingness to exert effort) stems from subjective value, which in turns depends on one's standards about performance and on the confidence about these standards. In a series of simulations, we show that the model explains puzzling motivational dynamics and associated feelings. Crucially, the model identifies realistic standards (i.e., those matching one's own actual ability) as those more beneficial for motivation and performance. On this basis, the model establishes a normative solution to the problem of optimal allocation of effort, analogous to the optimal allocation of neural and computational resources as in efficient coding.

The selective use of punishments on congruent versus incongruent trials in the Stroop task

Conflict adaptation refers to the dynamic modulation of conflict processing across successive trials and reflects improved cognitive control. Interestingly, aversive motivation can increase conflict adaptation, although it remains unclear through which process this modulation occurs because previous studies presented punishment feedback following suboptimal performance on both congruent and incongruent trials. According to integrative accounts of conflict monitoring and aversive motivation in the dorsal anterior cingulate cortex, punishment feedback following slow or erroneous performance on incongruent trials in particular should lead to improved conflict adaptation. Second, selectively increasing motivation on incongruent trials should reduce the overall congruency effect. The current study sought to test both hypotheses. Specifically, we administered the confound-minimized Stroop task to a large group of participants and manipulated the position of feedback (following either congruent or incongruent trials) and aversive motivation (tied to a monetary loss or not) across different blocks. As expected, the congruency effect was found to be smaller when punishment was coupled with incongruent versus congruent trials. However, results showed that conflict adaptation was increased when punishment feedback was selectively coupled with congruent rather than incongruent trials. Together, these results suggest that aversive motivation does not uniformly improve cognitive control but this gain appears to be context dependent.

Theta oscillations shift towards optimal frequency for cognitive control

Cognitive control allows to flexibly guide behaviour in a complex and ever-changing environment. It is supported by theta band (4-7 Hz) neural oscillations that coordinate distant neural populations. However, little is known about the precise neural mechanisms permitting such flexible control. Most research has focused on theta amplitude, showing that it increases when control is needed, but a second essential aspect of theta oscillations, their peak frequency, has mostly been overlooked. Here, using computational modelling and behavioural and electrophysiological recordings, in three independent datasets, we show that theta oscillations adaptively shift towards optimal frequency depending on task demands. We provide evidence that theta frequency balances reliable set-up of task representation and gating of task-relevant sensory and motor information and that this frequency shift predicts behavioural performance. Our study presents a mechanism supporting flexible control and calls for a reevaluation of the mechanistic role of theta oscillations in adaptive behaviour.

Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis

IMPORTANCE

Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent.

OBJECTIVE

To assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals.

DATA SOURCES

Web of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood).

STUDY SELECTION

Studies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models.

DATA EXTRACTION AND SYNTHESIS

Articles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected.

MAIN OUTCOMES AND MEASURES

The primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate).

RESULTS

A total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004).

CONCLUSIONS AND RELEVANCE

The simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders.

Time-Based Binding as a Solution to and a Limitation for Flexible Cognition

Why can't we keep as many items as we want in working memory? It has long been debated whether this resource limitation is a bug (a downside of our fallible biological system) or instead a feature (an optimal response to a computational problem). We propose that the resource limitation is a consequence of a useful feature. Specifically, we propose that flexible cognition requires time-based binding, and time-based binding necessarily limits the number of (bound) memoranda that can be stored simultaneously. Time-based binding is most naturally instantiated via neural oscillations, for which there exists ample experimental evidence. We report simulations that illustrate this theory and that relate it to empirical data. We also compare the theory to several other (feature and bug) resource theories.

Locus coeruleus neurons encode the subjective difficulty of triggering and executing actions

The brain stem noradrenergic nucleus locus coeruleus (LC) is involved in various costly processes: arousal, stress, and attention. Recent work has pointed toward an implication in physical effort, and indirect evidence suggests that the LC could be also involved in cognitive effort. To assess the dynamic relation between LC activity, effort production, and difficulty, we recorded the activity of 193 LC single units in 5 monkeys performing 2 discounting tasks (a delay discounting task and a force discounting task), as well as a simpler target detection task where conditions were matched for difficulty and only differed in terms of sensory-motor processes. First, LC neurons displayed a transient activation both when monkeys initiated an action and when exerting force. Second, the magnitude of the activation scaled with the associated difficulty, and, potentially, the corresponding amount of effort produced, both for decision and force production. Indeed, at action initiation in both discounting tasks, LC activation increased in conditions associated with lower average engagement rate, i.e., those requiring more cognitive control to trigger the response. Decision-related activation also scaled with response time (RT), over and above task parameters, in line with the idea that it reflects the amount of resources (here time) spent on the decision process. During force production, LC activation only scaled with the amount of force produced in the force discounting task, but not in the control target detection task, where subjective difficulty was equivalent across conditions. Our data show that LC neurons dynamically track the amount of effort produced to face both cognitive and physical challenges with a subsecond precision. This works provides key insight into effort processing and the contribution of the noradrenergic system, which is affected in several pathologies where effort is impaired, including Parkinson disease and depression.

A model for learning based on the joint estimation of stochasticity and volatility

Previous research has stressed the importance of uncertainty for controlling the speed of learning, and how such control depends on the learner inferring the noise properties of the environment, especially volatility: the speed of change. However, learning rates are jointly determined by the comparison between volatility and a second factor, moment-to-moment stochasticity. Yet much previous research has focused on simplified cases corresponding to estimation of either factor alone. Here, we introduce a learning model, in which both factors are learned simultaneously from experience, and use the model to simulate human and animal data across many seemingly disparate neuroscientific and behavioral phenomena. By considering the full problem of joint estimation, we highlight a set of previously unappreciated issues, arising from the mutual interdependence of inference about volatility and stochasticity. This interdependence complicates and enriches the interpretation of previous results, such as pathological learning in individuals with anxiety and following amygdala damage.

Computational Modeling of Catecholamines Dysfunction in Alzheimer's Disease at Pre-Plaque Stage

Background:

Alzheimer’s disease (AD) etiopathogenesis remains partially unexplained. The main conceptual framework used to study AD is the Amyloid Cascade Hypothesis, although the failure of recent clinical experimentation seems to reduce its potential in AD research.

Objective:

A possible explanation for the failure of clinical trials is that they are set too late in AD progression. Recent studies suggest that the ventral tegmental area (VTA) degeneration could be one of the first events occurring in AD progression (pre-plaque stage).

Methods:

Here we investigate this hypothesis through a computational model and computer simulations validated with behavioral and neural data from patients.

Results:

We show that VTA degeneration might lead to system-level adjustments of catecholamine release, triggering a sequence of events leading to relevant clinical and pathological signs of AD. These changes consist first in a midfrontal-driven compensatory hyperactivation of both VTA and locus coeruleus (norepinephrine) followed, with the progression of the VTA impairment, by a downregulation of catecholamine release. These processes could then trigger the neural degeneration at the cortical and hippocampal levels, due to the chronic loss of the neuroprotective role of norepinephrine.

Conclusion:

Our novel hypothesis might contribute to the formulation of a wider system-level view of AD which might help to devise early diagnostic and therapeutic interventions.

Internal manipulation of perceptual representations in human flexible cognition: A computational model

Executive functions represent a set of processes in goal-directed cognition that depend on integrated cortical-basal ganglia brain systems and form the basis of flexible human behaviour. Several computational models have been proposed for studying cognitive flexibility as a key executive function and the Wisconsin card sorting test (WCST) that represents an important neuropsychological tool to investigate it. These models clarify important aspects that underlie cognitive flexibility, particularly decision-making, motor response, and feedback-dependent learning processes. However, several studies suggest that the categorisation processes involved in the solution of the WCST include an additional computational stage of category representation that supports the other processes. Surprisingly, all models of the WCST ignore this fundamental stage and they assume that decision making directly triggers actions. Thus, we propose a novel hypothesis where the key mechanisms of cognitive flexibility and goal-directed behaviour rely on the acquisition of suitable representations of percepts and their top-down internal manipulation. Moreover, we propose a neuro-inspired computational model to operationalise this hypothesis. The capacity of the model to support cognitive flexibility was validated by systematically reproducing and interpreting the behaviour exhibited in the WCST by young and old healthy adults, and by frontal and Parkinson patients. The results corroborate and further articulate the hypothesis that the internal manipulation of representations is a core process in goal-directed flexible cognition.

The computational cost of active information sampling before decision-making under uncertainty

Humans often seek information to minimize the pervasive effect of uncertainty on decisions. Current theories explain how much knowledge people should gather before a decision, based on the cost-benefit structure of the problem at hand. Here, we demonstrate that this framework omits a crucial agent-related factor: the cognitive effort expended while collecting information. Using an active sampling model, we unveil a speed-efficiency trade-off whereby more informative samples take longer to find. Crucially, under sufficient time pressure, humans can break this trade-off, sampling both faster and more efficiently. Computational modelling demonstrates the existence of a cost of cognitive effort which, when incorporated into theoretical models, provides a better account of people's behaviour and also relates to self-reported fatigue accumulated during active sampling. Thus, the way people seek knowledge to guide their decisions is shaped not only by task-related costs and benefits, but also crucially by the quantifiable computational costs incurred.

Deficits of hierarchical predictive coding in left spatial neglect

Right brain-damaged patients with unilateral spatial neglect fail to explore the left side of space. Recent EEG and clinical evidence suggests that neglect patients might suffer deficits in predictive coding, i.e. in identifying and exploiting probabilistic associations among sensory stimuli in the environment. To gain direct insights on this issue, we focussed on the hierarchical components of predictive coding. We recorded EEG responses evoked by central, left-side or right-side tones that were presented at the end of sequences of four central tones. Left-side and right-side deviant tones produce a pre-attentive Mismatch Negativity that reflects a lower-order prediction error for the 'Local' deviation of the tone at the end of the sequence. Higher-order prediction errors for the frequency of these deviations in the acoustic environment, i.e. 'Global' deviation, are marked by the P3 response. We show that when neglect patients are immersed in an acoustic environment characterized by frequent left-side deviant tones, they display no pre-attentive Mismatch Negativity both for left-side deviant tones and infrequent omissions of the last tone, while they have Mismatch Negativity for infrequent right-side deviant tones. In the same condition, neglect patients show no P300 response to 'Global' prediction errors for deviant tones, including those in the non-neglected right-side, and omissions. In contrast to this, when right-side deviant tones are predominant in the acoustic environment, neglect patients have pre-attentive Mismatch Negativity both for right-side deviant tones and infrequent omissions, while they display no Mismatch Negativity for infrequent left-side deviant tones. Most importantly, in the same condition neglect patients show enhanced P300 response to infrequent left-side deviant tones, notwithstanding that these tones evoked no pre-attentive Mismatch Negativity. This latter finding indicates that 'Global' predictions are independent of 'Local' error signals provided by the Mismatch Negativity. These results qualify deficits of predictive coding in the spatial neglect syndrome and show that neglect patients base their predictive behaviour only on statistical regularities that are related to the frequent occurrence of sensory events on the right side of space.

The Best Laid Plans: Computational Principles of Anterior Cingulate Cortex

Despite continual debate for the past 30 years about the function of anterior cingulate cortex (ACC), its key contribution to neurocognition remains unknown. However, recent computational modeling work has provided insight into this question. Here we review computational models that illustrate three core principles of ACC function, related to hierarchy, world models, and cost. We also discuss four constraints on the neural implementation of these principles, related to modularity, binding, encoding, and learning and regulation. These observations suggest a role for ACC in hierarchical model-based hierarchical reinforcement learning (HMB-HRL), which instantiates a mechanism motivating the execution of high-level plans.

The midcingulate cortex and temporal integration

The ability to integrate information across time at multiple timescales is a vital element of adaptive behavior, because it provides the capacity to link events separated in time, extract useful information from previous events and actions, and to construct plans for behavior over time. Here we make the argument that this information integration capacity is a central function of the midcingulate cortex (MCC), by reviewing the anatomical, intrinsic network, neurophysiological, and behavioral properties of MCC. The MCC is the region of the medial wall situated dorsal to the corpus callosum and sometimes referred to as dACC. It is positioned within the densely connected core network of the primate brain, with a rich diversity of cognitive, somatomotor and autonomic connections. Furthermore, the MCC shows strong local network inhibition which appears to control the metastability of the region-an established feature of many cortical networks in which the neural dynamics move through a series of quasi-stationary states. We propose that the strong local inhibition in MCC leads to particularly long dynamic state durations, and so less frequent transitions. Apparently as a result of these anatomical features and synaptic and ionic determinants, the MCC cells display the longest neuronal timescales among a range of recorded cortical areas. We conclude that the anatomical position, intrinsic properties, and local network interactions of MCC make it a uniquely positioned cortical area to perform the integration of diverse information over time that is necessary for behavioral adaptation.

Human complex exploration strategies are enriched by noradrenaline-modulated heuristics

An exploration-exploitation trade-off, the arbitration between sampling a lesser-known against a known rich option, is thought to be solved using computationally demanding exploration algorithms. Given known limitations in human cognitive resources, we hypothesised the presence of additional cheaper strategies. We examined for such heuristics in choice behaviour where we show this involves a value-free random exploration, that ignores all prior knowledge, and a novelty exploration that targets novel options alone. In a double-blind, placebo-controlled drug study, assessing contributions of dopamine (400 mg amisulpride) and noradrenaline (40 mg propranolol), we show that value-free random exploration is attenuated under the influence of propranolol, but not under amisulpride. Our findings demonstrate that humans deploy distinct computationally cheap exploration strategies and that value-free random exploration is under noradrenergic control.

Learning to Synchronize: Midfrontal Theta Dynamics during Rule Switching

In recent years, several hierarchical extensions of well-known learning algorithms have been proposed. For example, when stimulus-action mappings vary across time or context, the brain may learn two or more stimulus-action mappings in separate modules, and additionally (at a hierarchically higher level) learn to appropriately switch between those modules. However, how the brain mechanistically coordinates neural communication to implement such hierarchical learning remains unknown. Therefore, the current study tests a recent computational model that proposed how midfrontal theta oscillations implement such hierarchical learning via the principle of binding by synchrony (Sync model). More specifically, the Sync model uses bursts at theta frequency to flexibly bind appropriate task modules by synchrony. The 64-channel EEG signal was recorded while 27 human subjects (female: 21, male: 6) performed a probabilistic reversal learning task. In line with the Sync model, postfeedback theta power showed a linear relationship with negative prediction errors, but not with positive prediction errors. This relationship was especially pronounced for subjects with better behavioral fit (measured via Akaike information criterion) of the Sync model. Also consistent with Sync model simulations, theta phase-coupling between midfrontal electrodes and temporoparietal electrodes was stronger after negative feedback. Our data suggest that the brain uses theta power and synchronization for flexibly switching between task rule modules, as is useful, for example, when multiple stimulus-action mappings must be retained and used.SIGNIFICANCE STATEMENT Everyday life requires flexibility in switching between several rules. A key question in understanding this ability is how the brain mechanistically coordinates such switches. The current study tests a recent computational framework (Sync model) that proposed how midfrontal theta oscillations coordinate activity in hierarchically lower task-related areas. In line with predictions of this Sync model, midfrontal theta power was stronger when rule switches were most likely (strong negative prediction error), especially in subjects who obtained a better model fit. Additionally, also theta phase connectivity between midfrontal and task-related areas was increased after negative feedback. Thus, the data provided support for the hypothesis that the brain uses theta power and synchronization for flexibly switching between rules.

Failure to modulate reward prediction errors in declarative learning with theta (6 Hz) frequency transcranial alternating current stimulation

Recent evidence suggests that reward prediction errors (RPEs) play an important role in declarative learning, but its neurophysiological mechanism remains unclear. Here, we tested the hypothesis that RPEs modulate declarative learning via theta-frequency oscillations, which have been related to memory encoding in prior work. For that purpose, we examined the interaction between RPE and transcranial Alternating Current Stimulation (tACS) in declarative learning. Using a between-subject (real versus sham stimulation group), single-blind stimulation design, 76 participants learned 60 Dutch-Swahili word pairs, while theta-frequency (6 Hz) tACS was administered over the medial frontal cortex (MFC). Previous studies have implicated MFC in memory encoding. We replicated our previous finding of signed RPEs (SRPEs) boosting declarative learning; with larger and more positive RPEs enhancing memory performance. However, tACS failed to modulate the SRPE effect in declarative learning and did not affect memory performance. Bayesian statistics supported evidence for an absence of effect. Our study confirms a role of RPE in declarative learning, but also calls for standardized procedures in transcranial electrical stimulation.

Reward uncertainty asymmetrically affects information transmission within the monkey fronto-parietal network

A central hypothesis in research on executive function is that controlled information processing is costly and is allocated according to the behavioral benefits it brings. However, while computational theories predict that the benefits of new information depend on prior uncertainty, the cellular effects of uncertainty on the executive network are incompletely understood. Using simultaneous recordings in monkeys, we describe several mechanisms by which the fronto-parietal network reacts to uncertainty. We show that the variance of expected rewards, independently of the value of the rewards, was encoded in single neuron and population spiking activity and local field potential (LFP) oscillations, and, importantly, asymmetrically affected fronto-parietal information transmission (measured through the coherence between spikes and LFPs). Higher uncertainty selectively enhanced information transmission from the parietal to the frontal lobe and suppressed it in the opposite direction, consistent with Bayesian principles that prioritize sensory information according to a decision maker’s prior uncertainty.

Bahareh Taghizadeh and Nicholas Foley et al. show that individual neuronal responses, population spiking activity, and local field potential oscillations encode the variance of expected rewards independent of their value. They also demonstrate that reward uncertainty asymmetrically affects neuronal transmission within the monkey fronto-parietal network.

The role of norepinephrine in the pathophysiology of schizophrenia

Several lines of evidence have suggested for decades a role for norepinephrine (NE) in the pathophysiology and treatment of schizophrenia. Recent experimental findings reveal anatomical and physiological properties of the locus coeruleus-norepinephrine (LC-NE) system and its involvement in brain function and cognition. Here, we integrate these two lines of evidence. First, we review the functional and structural properties of the LC-NE system and its impact on functional brain networks, cognition, and stress, with special emphasis on recent experimental and theoretical advances. Subsequently, we present an update about the role of LC-associated functions for the pathophysiology of schizophrenia, focusing on the cognitive and motivational deficits. We propose that schizophrenia phenomenology, in particular cognitive symptoms, may be explained by an abnormal interaction between genetic susceptibility and stress-initiated LC-NE dysfunction. This in turn, leads to imbalance between LC activity modes, dysfunctional regulation of brain network integration and neural gain, and deficits in cognitive functions. Finally, we suggest how recent development of experimental approaches can be used to characterize LC function in schizophrenia.

Neural Dynamics of Cognitive Control in Various Types of Incongruence

Conflict-control is a core function of cognitive control. Although numerous studies have considered cognitive control to be domain-general, the shared and distinct brain responses to different types of incongruence or conflict remain unclear. Using a hybrid flanker task, the present study explored the temporal dynamics of brain activation to three types of incongruence: flanker interference, rule-based response switch (rule-switch), and action-based response switch (response-alternation). The results showed that: (1) all three types of incongruence evoked larger N2 amplitudes than the congruent condition in the frontal region, with the N2 amplitudes and topographical distribution of the N2 effect differing between the different types of incongruence; and (2) in the P300 time window, the flanker interference condition yielded the most delayed P300 latency, whereas the rule-switch and response-alternation conditions yielded smaller P300 amplitudes with a longer interval from P300 peak to a keypress. These findings suggest that different types of incongruence are first monitored similarly by the cognitive control system and then resolved differently.

Interplay between midbrain and dorsal anterior cingulate regions arbitrates lingering reward effects on memory encoding

Rewarding events enhance memory encoding via dopaminergic influences on hippocampal plasticity. Phasic dopamine release depends on immediate reward magnitude, but presumably also on tonic dopamine levels, which may vary as a function of the average accumulation of reward over time. Using model-based fMRI in combination with a novel associative memory task, we show that immediate reward magnitude exerts a monotonically increasing influence on the nucleus accumbens, ventral tegmental area (VTA), and hippocampal activity during encoding, and enhances memory. By contrast, average reward levels modulate feedback-related responses in the VTA and hippocampus in a non-linear (inverted U-shape) fashion, with similar effects on memory performance. Additionally, the dorsal anterior cingulate cortex (dACC) monotonically tracks average reward levels, while VTA-dACC functional connectivity is non-linearly modulated (inverted U-shape) by average reward. We propose that the dACC computes the net behavioral impact of average reward and relays this information to memory circuitry via the VTA.

Reward Prediction Error and Declarative Memory

Learning based on reward prediction error (RPE) was originally proposed in the context of nondeclarative memory. We postulate that RPE may support declarative memory as well. Indeed, recent years have witnessed a number of independent empirical studies reporting effects of RPE on declarative memory. We provide a brief overview of these studies, identify emerging patterns, and discuss open issues such as the role of signed versus unsigned RPEs in declarative learning.

Reward Modulates Affective Priming Effect in Cognitive Conflict Processing: Electrophysiological Evidence

Previous research demonstrated that cognitive conflict could induce an affective priming effect, and the stage (detection/resolution) of conflict processing led to different directions (positive/negative) of the affective priming effect. We suggested that rewards play a critical role in the affective priming effect on conflict resolution. The present study used event-related potentials (ERPs), using the arrow flanker task as primes and choosing specific affective words as targets to investigate the affective priming effect induced by cognitive conflict during the resolution stage. Our question was whether rewards created a modulating effect. Participants were asked to judge the congruency of the prime stimuli and then evaluate the valence of the target words. For behavioral results, the conflict effect was significant, and the reward promoted the behavioral performance of participants. For ERP results, enhanced N2 amplitudes for incongruent primes indicated a significant conflict effect. More importantly, as expected, in the rewarded condition, the enhanced N400 amplitudes for positive targets following incongruent primes were found, indicating a positive priming effect. However, in the unrewarded condition, the reduced N400 amplitudes for positive targets following incongruent primes were found, indicating conflict resolution hindered the processing of positive stimuli. These findings suggested that cognitive conflict-induced the positive priming effect during the resolution stage and that rewards had a moderating effect on the positive priming effect.

Catecholaminergic modulation of meta-learning

The remarkable expedience of human learning is thought to be underpinned by meta-learning, whereby slow accumulative learning processes are rapidly adjusted to the current learning environment. To date, the neurobiological implementation of meta-learning remains unclear. A burgeoning literature argues for an important role for the catecholamines dopamine and noradrenaline in meta-learning. Here, we tested the hypothesis that enhancing catecholamine function modulates the ability to optimise a meta-learning parameter (learning rate) as a function of environmental volatility. 102 participants completed a task which required learning in stable phases, where the probability of reinforcement was constant, and volatile phases, where probabilities changed every 10-30 trials. The catecholamine transporter blocker methylphenidate enhanced participants' ability to adapt learning rate: Under methylphenidate, compared with placebo, participants exhibited higher learning rates in volatile relative to stable phases. Furthermore, this effect was significant only with respect to direct learning based on the participants' own experience, there was no significant effect on inferred-value learning where stimulus values had to be inferred. These data demonstrate a causal link between catecholaminergic modulation and the adjustment of the meta-learning parameter learning rate.

Striatal dopamine D2 binding correlates with locus of control: Preliminary evidence from [11C]raclopride Positron Emission Tomography

The ability to exert control has been widely investigated as a hallmark of adaptive behaviour. Dopamine is recognized as the key neuromodulator mediating various control-related processes. The neural mechanisms underlying the subjective perception of being in control, or Locus of Control (LOC) are however less clear. LOC indicates the subjective tendency to attribute environmental outcomes to one's actions (internal LOC) or instead to external incontrollable factors (external LOC). Here we hypothesized that dopamine levels also relate to LOC. Previous work shows that dopamine signaling mediates learning of action-outcome relationships, outcome predictability, and opportunity cost. Prominent theories propose dopamine dysregulation as the key pathogenetic mechanism in schizophrenia and depression. Critically, external LOC is a risk factor for schizophrenia and depression, and predicts increased vulnerability to stress. However, a direct link between LOC and dopamine levels in healthy control had not been demonstrated. The purpose of our study was to investigate this link. Using [¹¹C]raclopride Positron Emission Tomography we tested the relationship between D2 receptor binding in the striatum and LOC (measured with the Rotter Locus of Control scale) in 15 healthy volunteers. Our results show a large and positive correlation: increased striatal D2 binding was associated with External LOC. This finding opens promising avenues for the study of several psychological impairments that have been associated with both dopamine and LOC, such as addiction, schizophrenia, and depression.

Pupil dilation as an implicit measure of appetitive Pavlovian learning

Appetitive Pavlovian conditioning is a learning mechanism of fundamental biological and pathophysiological significance. Nonetheless, its exploration in humans remains sparse, which is partly attributed to the lack of an established psychophysiological parameter that aptly represents conditioned responding. This study evaluated pupil diameter and other ocular response measures (gaze dwelling time, blink duration and count) as indices of conditioning. Additionally, a learning model was used to infer participants' learning progress on the basis of their pupil dilation. Twenty-nine healthy volunteers completed an appetitive differential delay conditioning paradigm with a primary reward, while the ocular response measures along with other psychophysiological (heart rate, electrodermal activity, postauricular and eyeblink reflex) and behavioral (ratings, contingency awareness) parameters were obtained to examine the relation among different measures. A significantly stronger increase in pupil diameter, longer gaze duration and shorter eyeblink duration was observed in response to the reward-predicting cue compared to the control cue. The Pearce-Hall attention model best predicted the trial-by-trial pupil diameter. This conditioned response was corroborated by a pronounced heart rate deceleration to the reward-predicting cue, while no conditioning effect was observed in the electrodermal activity or startle responses. There was no discernible correlation between the psychophysiological response measures. These results highlight the potential value of ocular response measures as sensitive indices for representing appetitive conditioning.

Metacontrol of decision-making strategies in human aging

Humans employ different strategies when making decisions. Previous research has reported reduced reliance on model-based strategies with aging, but it remains unclear whether this is due to cognitive or motivational factors. Moreover, it is not clear how aging affects the metacontrol of decision making, that is the dynamic adaptation of decision-making strategies to varying situational demands. In this cross-sectional study, we tested younger and older adults in a sequential decision-making task that dissociates model-free and model-based strategies. In contrast to previous research, model-based strategies led to higher payoffs. Moreover, we manipulated the costs and benefits of model-based strategies by varying reward magnitude and the stability of the task structure. Compared to younger adults, older adults showed reduced model-based decision making and less adaptation of decision-making strategies. Our findings suggest that aging affects the metacontrol of decision-making strategies and that reduced model-based strategies in older adults are due to limited cognitive abilities.

Learning to synchronize: How biological agents can couple neural task modules for dealing with the stability-plasticity dilemma

We provide a novel computational framework on how biological and artificial agents can learn to flexibly couple and decouple neural task modules for cognitive processing. In this way, they can address the stability-plasticity dilemma. For this purpose, we combine two prominent computational neuroscience principles, namely Binding by Synchrony and Reinforcement Learning. The model learns to synchronize task-relevant modules, while also learning to desynchronize currently task-irrelevant modules. As a result, old (but currently task-irrelevant) information is protected from overwriting (stability) while new information can be learned quickly in currently task-relevant modules (plasticity). We combine learning to synchronize with task modules that learn via one of several classical learning algorithms (Rescorla-Wagner, backpropagation, Boltzmann machines). The resulting combined model is tested on a reversal learning paradigm where it must learn to switch between three different task rules. We demonstrate that our combined model has significant computational advantages over the original network without synchrony, in terms of both stability and plasticity. Importantly, the resulting models' processing dynamics are also consistent with empirical data and provide empirically testable hypotheses for future MEG/EEG studies.

Affect and cognitive control: Insights from research on effort mobilization

We present theory and research on effort mobilization that is relevant for understanding the role of affect in cognitive control. We posit that cognitive control and effort are closely related and introduce motivational intensity theory and supporting empirical evidence mainly based on cardiovascular measures of effort. Most important, we discuss the role of affect in the context of effort mobilization and cognitive control from different perspectives. We first present theories predicting affective influences on effort, namely the mood-behavior-model and the implicit-affect-primes-effort model, and supporting empirical evidence. Second, we discuss further implications of the resource conservation principle highlighting the aversive aspect of effort and review evidence for the impact of value and its affective component on effort and cognitive control. Finally, we present a recent integration of the neural mechanisms underlying both effort and cognitive control. We conclude that affective processes are necessary and instrumental for both effort mobilization and cognitive control.

Parietal neurons encode information sampling based on decision uncertainty

In natural behavior animals actively gather information that is relevant for learning or actions, but the mechanisms of active sampling are rarely investigated. We tested parietal neurons involved in oculomotor control in a task in which monkeys made saccades to gather visual information before reporting a decision based on the information. We show that the neurons encode, before the saccade, the information gains (reduction in decision uncertainty) that the saccade was expected to bring, correlating with the monkeys’ efficiency in processing the information in the post-saccadic fixation. Informational sensitivity is independent of the neurons’ reward sensitivity, which is unreliable across task contexts, inconsistent with the view that the cells encode economic utility. Instead, we suggest that parietal cells are involved in implementing active sampling policies, showing uncertainty-dependent boosts of neural gain that facilitate the selection of relevant cues and the efficient use of the information delivered by these cues.

Anterior cingulate cortex encoding of effortful behavior

An animal’s ability to assess the value of their behaviors to minimize energy use while maximizing goal achievement is critical to its survival. The anterior cingulate cortex (ACC) has been previously shown to play a critical role in this behavioral optimization process, especially when animals are faced with effortful behaviors. In the present study, we designed a novel task to investigate the role of the ACC in evaluating behaviors that varied in effort but all resulted in the same outcome. We recorded single unit activity from the ACC as rats ran back and forth in a shuttle box that could be tilted to different tilt angles (0, 15, and 25°) to manipulate effort. Overall, a majority of ACC neurons showed selective firing to specific effort conditions. During effort expenditure, ACC units showed a consistent firing rate bias toward the downhill route compared with the more difficult uphill route, regardless of the tilt angle of the apparatus. Once rats completed a run and received their fixed reward, ACC units also showed a clear firing rate preference for the single condition with the highest relative value (25° downhill). To assess effort preferences, we used a choice version of our task and confirmed that rats prefer downhill routes to uphill routes when given the choice. Overall, these results help to elucidate the functional role of the ACC in monitoring and evaluating effortful behaviors that may then bias decision-making toward behaviors with the highest utility.

NEW & NOTEWORTHY We developed a novel effort paradigm to investigate how the anterior cingulate cortex (ACC) responds to behaviors with varied degrees of physical effort and how changes in effort influence the ACC’s evaluation of behavioral outcomes. Our results provide evidence for a wider role of the ACC in its ability to motivate effortful behaviors and evaluate the outcome of multiple behaviors within an environment.