Valuation and decision-making in frontal cortex: one or many serial or parallel systems?

Goal-Directed Decision Making with Spiking Neurons

Behavioral and neuroscientific data on reward-based decision making point to a fundamental distinction between habitual and goal-directed action selection. The formation of habits, which requires simple updating of cached values, has been studied in great detail, and the reward prediction error theory of dopamine function has enjoyed prominent success in accounting for its neural bases. In contrast, the neural circuit mechanisms of goal-directed decision making, requiring extended iterative computations to estimate values online, are still unknown. Here we present a spiking neural network that provably solves the difficult online value estimation problem underlying goal-directed decision making in a near-optimal way and reproduces behavioral as well as neurophysiological experimental data on tasks ranging from simple binary choice to sequential decision making. Our model uses local plasticity rules to learn the synaptic weights of a simple neural network to achieve optimal performance and solves one-step decision-making tasks, commonly considered in neuroeconomics, as well as more challenging sequential decision-making tasks within 1 s. These decision times, and their parametric dependence on task parameters, as well as the final choice probabilities match behavioral data, whereas the evolution of neural activities in the network closely mimics neural responses recorded in frontal cortices during the execution of such tasks. Our theory provides a principled framework to understand the neural underpinning of goal-directed decision making and makes novel predictions for sequential decision-making tasks with multiple rewards.

SIGNIFICANCE STATEMENT Goal-directed actions requiring prospective planning pervade decision making, but their circuit-level mechanisms remain elusive. We show how a model circuit of biologically realistic spiking neurons can solve this computationally challenging problem in a novel way. The synaptic weights of our network can be learned using local plasticity rules such that its dynamics devise a near-optimal plan of action. By systematically comparing our model results to experimental data, we show that it reproduces behavioral decision times and choice probabilities as well as neural responses in a rich set of tasks. Our results thus offer the first biologically realistic account for complex goal-directed decision making at a computational, algorithmic, and implementational level.

Specific frontal neural dynamics contribute to decisions to check

Curiosity and information seeking potently shapes our behaviour and are thought to rely on the frontal cortex. Yet, the frontal regions and neural dynamics that control the drive to check for information remain unknown. Here we trained monkeys in a task where they had the opportunity to gain information about the potential delivery of a large bonus reward or continue with a default instructed decision task. Single-unit recordings in behaving monkeys reveal that decisions to check for additional information first engage midcingulate cortex and then lateral prefrontal cortex. The opposite is true for instructed decisions. Importantly, deciding to check engages neurons also involved in performance monitoring. Further, specific midcingulate activity could be discerned several trials before the monkeys actually choose to check the environment. Our data show that deciding to seek information on the current state of the environment is characterized by specific dynamics of neural activity within the prefrontal cortex.

Information seeking is thought to rely on the brain's frontal cortex but which regions specifically control this drive remains unknown. Here the authors show that monkeys deciding to seek information on the current state of the environment showed specific neural dynamics in the lateral prefrontal cortex and midcingulate cortex.

Neural and neurochemical basis of reinforcement-guided decision making

Decision making is an adaptive behavior that takes into account several internal and external input variables and leads to the choice of a course of action over other available and often competing alternatives. While it has been studied in diverse fields ranging from mathematics, economics, ecology, and ethology to psychology and neuroscience, recent cross talk among perspectives from different fields has yielded novel descriptions of decision processes. Reinforcement-guided decision making models are based on economic and reinforcement learning theories, and their focus is on the maximization of acquired benefit over a defined period of time. Studies based on reinforcement-guided decision making have implicated a large network of neural circuits across the brain. This network includes a wide range of cortical (e.g., orbitofrontal cortex and anterior cingulate cortex) and subcortical (e.g., nucleus accumbens and subthalamic nucleus) brain areas and uses several neurotransmitter systems (e.g., dopaminergic and serotonergic systems) to communicate and process decision-related information. This review discusses distinct as well as overlapping contributions of these networks and neurotransmitter systems to the processing of decision making. We end the review by touching on neural circuitry and neuromodulatory regulation of exploratory decision making.

Neuronal remapping and circuit persistence in economic decisions

The orbitofrontal cortex plays a central role in good-based economic decisions. When subjects make choices, neurons in this region represent the identities and values of offered and chosen goods. Notably, choices in different behavioral contexts may involve a potentially infinite variety of goods. Thus a fundamental question concerns the stability versus flexibility of the decision circuit. Here we show in rhesus monkeys that neurons encoding the identity or the subjective value of particular goods in a given context 'remap' and become associated with different goods when the context changes. At the same time, the overall organization of the decision circuit and the function of individual cells remain stable across contexts. In particular, two neurons supporting the same decision in one context also support the same decision in different contexts. These results demonstrate how the same neural circuit can underlie economic decisions involving a large variety of goods.

Risk-taking and risky decision-making in Internet gaming disorder: Implications regarding online gaming in the setting of negative consequences

Individuals with Internet gaming disorder (IGD) continue gaming despite adverse consequences. However, the precise mechanism underlying this behavior remains unknown. In this study, data from 20 IGD subjects and 16 otherwise comparable healthy control subjects (HCs) were recorded and compared when they were undergoing risk-taking and risky decision-making during functional magnetic resonance imaging (fMRI). During risk-taking and as compared to HCs, IGD subjects selected more risk-disadvantageous trials and demonstrated less activation of the anterior cingulate, posterior cingulate and middle temporal gyrus. During risky decision-making and as compared to HCs, IGD subjects showed shorter response times and less activations of the inferior frontal and superior temporal gyri. Taken together, data suggest that IGD subjects show impaired executive control in selecting risk-disadvantageous choices, and they make risky decisions more hastily and with less recruitment of regions implicated in impulse control. These results suggest a possible neurobiological underpinning for why IGD subjects may exhibit poor control over their game-seeking behaviors even when encountering negative consequences and provide possible therapeutic targets for interventions in this population.

Capturing the temporal evolution of choice across prefrontal cortex

Activity in prefrontal cortex (PFC) has been richly described using economic models of choice. Yet such descriptions fail to capture the dynamics of decision formation. Describing dynamic neural processes has proven challenging due to the problem of indexing the internal state of PFC and its trial-by-trial variation. Using primate neurophysiology and human magnetoencephalography, we here recover a single-trial index of PFC internal states from multiple simultaneously recorded PFC subregions. This index can explain the origins of neural representations of economic variables in PFC. It describes the relationship between neural dynamics and behaviour in both human and monkey PFC, directly bridging between human neuroimaging data and underlying neuronal activity. Moreover, it reveals a functionally dissociable interaction between orbitofrontal cortex, anterior cingulate cortex and dorsolateral PFC in guiding cost-benefit decisions. We cast our observations in terms of a recurrent neural network model of choice, providing formal links to mechanistic dynamical accounts of decision-making.

DOI:http://dx.doi.org/10.7554/eLife.11945.001

In 1848, a railroad worker named Phineas Gage suffered an accident that was to secure him a place in neuroscience lore. While constructing a new railway line, a mistimed explosion propelled an iron bar into the base of his skull, where it passed behind his left eye before exiting through the top of his head. Gage survived the accident, but those who knew him reported significant changes in his personality and behaviour.

Gage’s ability to make decisions was particularly impaired by his injury. Decision-making involves weighing up the costs and benefits associated with alternative courses of action. It entails looking into the future to decide whether an anticipated reward will justify the effort or expense necessary to obtain it. This process is dependent on a region of the brain called the prefrontal cortex, the area that sustained the most damage in Phineas Gage.

While many studies have shown correlations between activity in particular parts of prefrontal cortex and the outcome of decisions, little is known about how this activity evolves over time as a decision is made. To explore this process, Hunt et al. trained macaque monkeys to choose between pairs of images that were associated with specific rewards (quantities of fruit juice) and costs (either amounts of work or fixed delays).

Electrode recordings revealed changes in prefrontal activity that varied over time as the monkeys deliberated over each pair of images, choosing for example between a large reward after a long delay versus a smaller reward immediately. This activity was consistent with a mathematical model of decision-making, which also explains data from brain imaging experiments in humans. This provides an important link between human data and electrode recordings in animals.

However, some of the patterns of activity observed in both macaques and humans appeared to reflect the speed at which decisions were made, rather than the outcome of the decisions themselves. By extracting information about decision speed on each decision from each region, it was shown that communication between regions of prefrontal cortex changes when choices are between two different amounts of work, as opposed to two different delays. Further experiments are needed to explore this phenomenon and to determine how other brain regions interact with the prefrontal cortex to support the decision-making process.

DOI:http://dx.doi.org/10.7554/eLife.11945.002

The Role of High-Level Processes for Oscillatory Phase Entrainment to Speech Sound

Constantly bombarded with input, the brain has the need to filter out relevant information while ignoring the irrelevant rest. A powerful tool may be represented by neural oscillations which entrain their high-excitability phase to important input while their low-excitability phase attenuates irrelevant information. Indeed, the alignment between brain oscillations and speech improves intelligibility and helps dissociating speakers during a “cocktail party”. Although well-investigated, the contribution of low- and high-level processes to phase entrainment to speech sound has only recently begun to be understood. Here, we review those findings, and concentrate on three main results: (1) Phase entrainment to speech sound is modulated by attention or predictions, likely supported by top-down signals and indicating higher-level processes involved in the brain’s adjustment to speech. (2) As phase entrainment to speech can be observed without systematic fluctuations in sound amplitude or spectral content, it does not only reflect a passive steady-state “ringing” of the cochlea, but entails a higher-level process. (3) The role of intelligibility for phase entrainment is debated. Recent results suggest that intelligibility modulates the behavioral consequences of entrainment, rather than directly affecting the strength of entrainment in auditory regions. We conclude that phase entrainment to speech reflects a sophisticated mechanism: several high-level processes interact to optimally align neural oscillations with predicted events of high relevance, even when they are hidden in a continuous stream of background noise.

The Good, the Bad, and the Irrelevant: Neural Mechanisms of Learning Real and Hypothetical Rewards and Effort

Natural environments are complex, and a single choice can lead to multiple outcomes. Agents should learn which outcomes are due to their choices and therefore relevant for future decisions and which are stochastic in ways common to all choices and therefore irrelevant for future decisions between options. We designed an experiment in which human participants learned the varying reward and effort magnitudes of two options and repeatedly chose between them. The reward associated with a choice was randomly real or hypothetical (i.e., participants only sometimes received the reward magnitude associated with the chosen option). The real/hypothetical nature of the reward on any one trial was, however, irrelevant for learning the longer-term values of the choices, and participants ought to have only focused on the informational content of the outcome and disregarded whether it was a real or hypothetical reward. However, we found that participants showed an irrational choice bias, preferring choices that had previously led, by chance, to a real reward in the last trial. Amygdala and ventromedial prefrontal activity was related to the way in which participants' choices were biased by real reward receipt. By contrast, activity in dorsal anterior cingulate cortex, frontal operculum/anterior insula, and especially lateral anterior prefrontal cortex was related to the degree to which participants resisted this bias and chose effectively in a manner guided by aspects of outcomes that had real and more sustained relationships with particular choices, suppressing irrelevant reward information for more optimal learning and decision making.

SIGNIFICANCE STATEMENT

In complex natural environments, a single choice can lead to multiple outcomes. Human agents should only learn from outcomes that are due to their choices, not from outcomes without such a relationship. We designed an experiment to measure learning about reward and effort magnitudes in an environment in which other features of the outcome were random and had no relationship with choice. We found that, although people could learn about reward magnitudes, they nevertheless were irrationally biased toward repeating certain choices as a function of the presence or absence of random reward features. Activity in different brain regions in the prefrontal cortex either reflected the bias or reflected resistance to the bias.

How the risky features of previous selection affect subsequent decision-making: evidence from behavioral and fMRI measures

Human decision making is rarely conducted in temporal isolation. It is often biased and affected by environmental variables, particularly prior selections. In this study, we used a task that simulates a real gambling process to explore the effect of the risky features of a previous selection on subsequent decision making. Compared with decision making after an advantageous risk-taking situation (Risk_Adv), that after a disadvantageous risk-taking situation (Risk_Disadv) is associated with a longer response time (RT, the time spent in making decisions) and higher brain activations in the caudate and the dorsolateral prefrontal cortex (DLPFC). Compared with decisions after Risk_Adv, those after Risk_Disadv in loss trials are associated with higher brain activations in the left superior temporal gyrus (STG) and the precuneus. Brain activity and relevant RTs significantly correlated. Overall, people who experience disadvantageous risk-taking selections tend to focus on current decision making and engage cognitive endeavors in value evaluation and in the regulation of their risk-taking behaviors during decision making.

Morphometric changes in the reward system of Parkinson's disease patients with impulse control disorders

Impulse control disorders (ICDs) occur in a subset of patients with Parkinson's disease (PD) who are receiving dopamine replacement therapy. In this study, we aimed to investigate structural abnormalities within the mesocortical and limbic cortices and subcortical structures in PD patients with ICDs. We studied 18 PD patients with ICDs, 18 PD patients without ICDs and a group of 24 age and sex-matched healthy controls. Cortical thickness (CTh) and subcortical nuclei volume analyses were carried out using the automated surface-based analysis package FreeSurfer (version 5.3.0). We found significant differences in MRI measures between the three groups. There was volume loss in the nucleus accumbens of both PD patients with ICDs and without ICDs compared to the control group. In addition, PD patients with ICDs showed significant atrophy in caudate, hippocampus and amygdala compared to the group of healthy controls. PD patients with ICDs had significant increased cortical thickness in rostral anterior cingulate cortex and frontal pole compared to PD patients without ICDs. Cortical thickness in rostral anterior cingulate and frontal pole was increased in PD patients with ICDs compared to the control group, but the differences failed to reach corrected levels of statistical significance. PD patients with ICDs showed increased cortical thickness in medial prefrontal regions. We speculate that these findings reflect either a pre-existing neural trait vulnerability to impulsivity or the expression of a maladaptive synaptic plasticity under non-physiological dopaminergic stimulation.

Common neural correlates of intertemporal choices and intelligence in adolescents

Converging behavioral evidence indicates that temporal discounting, measured by intertemporal choice tasks, is inversely related to intelligence. At the neural level, the parieto-frontal network is pivotal for complex, higher-order cognitive processes. Relatedly, underrecruitment of the pFC during a working memory task has been found to be associated with steeper temporal discounting. Furthermore, this network has also been shown to be related to the consistency of intertemporal choices. Here we report an fMRI study that directly investigated the association of neural correlates of intertemporal choice behavior with intelligence in an adolescent sample (n = 206; age 13.7-15.5 years). After identifying brain regions where the BOLD response during intertemporal choice was correlated with individual differences in intelligence, we further tested whether BOLD responses in these areas would mediate the associations between intelligence, the discounting rate, and choice consistency. We found positive correlations between BOLD response in a value-independent decision network (i.e., dorsolateral pFC, precuneus, and occipital areas) and intelligence. Furthermore, BOLD response in a value-dependent decision network (i.e., perigenual ACC, inferior frontal gyrus, ventromedial pFC, ventral striatum) was positively correlated with intelligence. The mediation analysis revealed that BOLD responses in the value-independent network mediated the association between intelligence and choice consistency, whereas BOLD responses in the value-dependent network mediated the association between intelligence and the discounting rate. In summary, our findings provide evidence for common neural correlates of intertemporal choice and intelligence, possibly linked by valuation as well as executive functions.

Lateral orbitofrontal cortex links social impressions to political choices

Recent studies of political behavior suggest that voting decisions can be influenced substantially by "first-impression" social attributions based on physical appearance. Separate lines of research have implicated the orbitofrontal cortex (OFC) in the judgment of social traits on the one hand and economic decision-making on the other, making this region a plausible candidate for linking social attributions to voting decisions. Here, we asked whether OFC lesions in humans disrupted the ability to judge traits of political candidates or affected how these judgments influenced voting decisions. Seven patients with lateral OFC damage, 18 patients with frontal damage sparing the lateral OFC, and 53 matched healthy participants took part in a simulated election paradigm, in which they voted for real-life (but unknown) candidates based only on photographs of their faces. Consistent with previous work, attributions of "competence" and "attractiveness" based on candidate appearance predicted voting behavior in the healthy control group. Frontal damage did not affect substantially the ability to make competence or attractiveness judgments, but patients with damage to the lateral OFC differed from other groups in how they applied this information when voting. Only attractiveness ratings had any predictive power for voting choices after lateral OFC damage, whereas other frontal patients and healthy controls relied on information about both competence and attractiveness in making their choice. An intact lateral OFC may not be necessary for judgment of social traits based on physical appearance, but it seems to be crucial in applying this information in political decision-making.

A Neural Mechanism of Strategic Social Choice under Sanction-Induced Norm Compliance

In recent years, much has been learned about the representation of subjective value in simple, nonstrategic choices. However, a large fraction of our daily decisions are embedded in social interactions in which value guided decisions require balancing benefits for self against consequences imposed by others in response to our choices. Yet, despite their ubiquity, much less is known about how value computation takes place in strategic social contexts that include the possibility of retribution for norm violations. Here, we used functional magnetic resonance imaging (fMRI) to show that when human subjects face such a context connectivity increases between the temporoparietal junction (TPJ), implicated in the representation of other peoples’ thoughts and intentions, and regions of ventromedial prefrontal cortex (vmPFC) that are associated with value computation. In contrast, we find no increase in connectivity between these regions in social nonstrategic cases where decision-makers are immune from retributive monetary punishments from a human partner. Moreover, there was also no increase in TPJ-vmPFC connectivity when the potential punishment was performed by a computer programmed to punish fairness norm violations in the same manner as a human would. Thus, TPJ-vmPFC connectivity is not simply a function of the social or norm enforcing nature of the decision, but rather occurs specifically in situations where subjects make decisions in a social context and strategically consider putative consequences imposed by others.

Signatures of Value Comparison in Ventral Striatum Neurons

The ventral striatum (VS), like its cortical afferents, is closely associated with processing of rewards, but the relative contributions of striatal and cortical reward systems remains unclear. Most theories posit distinct roles for these structures, despite their similarities. We compared responses of VS neurons to those of ventromedial prefrontal cortex (vmPFC) Area 14 neurons, recorded in a risky choice task. Five major response patterns observed in vmPFC were also observed in VS: (1) offer value encoding, (2) value difference encoding, (3) preferential encoding of chosen relative to unchosen value, (4) a correlation between residual variance in responses and choices, and (5) prominent encoding of outcomes. We did observe some differences as well; in particular, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, the close match between vmPFC and VS suggests that cortex and its striatal targets make overlapping contributions to economic choice.

A study of single neurons in the ventral striatum reveals signatures of value comparison and selection during a risky choice task, suggesting that the cortex and its striatal targets make overlapping contributions to the choice process. Read the accompanying Primer.

The neural calculations underlying reward-based choice are closely associated with a network of brain areas including the ventral striatum (VS) and ventromedial prefrontal cortex (vmPFC). Most theories ascribe distinct roles to these two structures during choice, but these differences have yet to be confirmed at the level of single neurons. We compared responses of VS neurons to those of vmPFC neurons recorded in rhesus macaques choosing between potential gambles for water rewards. We found widespread similarities in the way that VS and vmPFC neurons fire during the choice process. Neurons in both areas encoded the value of the offered gamble, the difference in value between offered gambles, and the gamble outcome. Additionally, both areas showed stronger coding for the chosen gamble than for the unchosen one and predicted choice even when we controlled for offer value. Interestingly, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, similarities between vmPFC and VS suggest that cortex and its striatal targets make overlapping contributions to reward-based choice.

Neuronal variability in orbitofrontal cortex during economic decisions

Neuroeconomic models assume that economic decisions are based on the activity of offer value cells in the orbitofrontal cortex (OFC), but testing this assertion has proven difficult. In principle, the decision made on a given trial should correlate with the stochastic fluctuations of these cells. However, this correlation, measured as a choice probability (CP), is small. Importantly, a neuron's CP reflects not only its individual contribution to the decision (termed readout weight), but also the intensity and the structure of correlated variability across the neuronal population (termed noise correlation). A precise mathematical relation between CPs, noise correlations, and readout weights was recently derived by Haefner and colleagues (Haefner RM, Gerwinn S, Macke JH, Bethge M. Nat Neurosci 16: 235-242, 2013) for a linear decision model. In this framework, concurrent measurements of noise correlations and CPs can provide quantitative information on how a population of cells contributes to a decision. Here we examined neuronal variability in the OFC of rhesus monkeys during economic decisions. Noise correlations had similar structure but considerably lower strength compared with those typically measured in sensory areas during perceptual decisions. In contrast, variability in the activity of individual cells was high and comparable to that recorded in other cortical regions. Simulation analyses based on Haefner's equation showed that noise correlations measured in the OFC combined with a plausible readout of offer value cells reproduced the experimental measures of CPs. In other words, the results obtained for noise correlations and those obtained for CPs taken together support the hypothesis that economic decisions are primarily based on the activity of offer value cells.

Uncovering a missing link in anterior cingulate research

Research on human anterior cingulate cortex has long indicated a role in detecting conflict. However, efforts to find parallel effects in non-human primates were surprisingly unsuccessful. Here, Ebitz and Platt (2015) break the resulting impasse by uncovering what appear to be conflict-related signals in monkey cingulate cortex.

Medial prefrontal cortical activity reflects dynamic re-evaluation during voluntary persistence

Deciding how long to keep waiting for future rewards is a nontrivial problem, especially when the timing of rewards is uncertain. We carried out an experiment in which human decision makers waited for rewards in two environments in which reward-timing statistics favored either a greater or lesser degree of behavioral persistence. We found that decision makers adaptively calibrated their level of persistence for each environment. Functional neuroimaging revealed signals that evolved differently during physically identical delays in the two environments, consistent with a dynamic and context-sensitive reappraisal of subjective value. This effect was observed in a region of ventromedial prefrontal cortex that is sensitive to subjective value in other contexts, demonstrating continuity between valuation mechanisms involved in discrete choice and in temporally extended decisions analogous to foraging. Our findings support a model in which voluntary persistence emerges from dynamic cost/benefit evaluation rather than from a control process that overrides valuation mechanisms.

Anticipatory anxiety disrupts neural valuation during risky choice

Incidental negative emotions unrelated to the current task, such as background anxiety, can strongly influence decisions. This is most evident in psychiatric disorders associated with generalized emotional disturbances. However, the neural mechanisms by which incidental emotions may affect choices remain poorly understood. Here we study the effects of incidental anxiety on human risky decision making, focusing on both behavioral preferences and their underlying neural processes. Although observable choices remained stable across affective contexts with high and low incidental anxiety, we found a clear change in neural valuation signals: during high incidental anxiety, activity in ventromedial prefrontal cortex and ventral striatum showed a marked reduction in (1) neural coding of the expected subjective value (ESV) of risky options, (2) prediction of observed choices, (3) functional coupling with other areas of the valuation system, and (4) baseline activity. At the same time, activity in the anterior insula showed an increase in coding the negative ESV of risky lotteries, and this neural activity predicted whether the risky lotteries would be rejected. This pattern of results suggests that incidental anxiety can shift the focus of neural valuation from possible positive consequences to anticipated negative consequences of choice options. Moreover, our findings show that these changes in neural value coding can occur in the absence of changes in overt behavior. This suggest a possible pathway by which background anxiety may lead to the development of chronic reward desensitization and a maladaptive focus on negative cognitions, as prevalent in affective and anxiety disorders.

Necessary, yet dissociable contributions of the insular and ventromedial prefrontal cortices to norm adaptation: computational and lesion evidence in humans

Social norms and their enforcement are fundamental to human societies. The ability to detect deviations from norms and to adapt to norms in a changing environment is therefore important to individuals' normal social functioning. Previous neuroimaging studies have highlighted the involvement of the insular and ventromedial prefrontal (vmPFC) cortices in representing norms. However, the necessity and dissociability of their involvement remain unclear. Using model-based computational modeling and neuropsychological lesion approaches, we examined the contributions of the insula and vmPFC to norm adaptation in seven human patients with focal insula lesions and six patients with focal vmPFC lesions, in comparison with forty neurologically intact controls and six brain-damaged controls. There were three computational signals of interest as participants played a fairness game (ultimatum game): sensitivity to the fairness of offers, sensitivity to deviations from expected norms, and the speed at which people adapt to norms. Significant group differences were assessed using bootstrapping methods. Patients with insula lesions displayed abnormally low adaptation speed to norms, yet detected norm violations with greater sensitivity than controls. Patients with vmPFC lesions did not have such abnormalities, but displayed reduced sensitivity to fairness and were more likely to accept the most unfair offers. These findings provide compelling computational and lesion evidence supporting the necessary, yet dissociable roles of the insula and vmPFC in norm adaptation in humans: the insula is critical for learning to adapt when reality deviates from norm expectations, and that the vmPFC is important for valuation of fairness during social exchange.

Social discounting involves modulation of neural value signals by temporoparietal junction

Most people are generous, but not toward everyone alike: generosity usually declines with social distance between individuals, a phenomenon called social discounting. Despite the pervasiveness of social discounting, social distance between actors has been surprisingly neglected in economic theory and neuroscientific research. We used functional magnetic resonance imaging (fMRI) to study the neural basis of this process to understand the neural underpinnings of social decision making. Participants chose between selfish and generous alternatives, yielding either a large reward for the participant alone, or smaller rewards for the participant and another individual at a particular social distance. We found that generous choices engaged the temporoparietal junction (TPJ). In particular, the TPJ activity was scaled to the social-distance-dependent conflict between selfish and generous motives during prosocial choice, consistent with ideas that the TPJ promotes generosity by facilitating overcoming egoism bias. Based on functional coupling data, we propose and provide evidence for a biologically plausible neural model according to which the TPJ supports social discounting by modulating basic neural value signals in the ventromedial prefrontal cortex to incorporate social-distance-dependent other-regarding preferences into an otherwise exclusively own-reward value representation.

Mechanisms of Individual Differences in Impulsive and Risky Choice in Rats

Individual differences in impulsive and risky choice are key risk factors for a variety of maladaptive behaviors such as drug abuse, gambling, and obesity. In our rat model, ordered individual differences are stable across choice parameters, months of testing, and span a broad spectrum, suggesting that rats, like humans, exhibit trait-level impulsive and risky choice behaviors. In addition, impulsive and risky choices are highly correlated, suggesting a degree of correlation between these two traits. An examination of the underlying cognitive mechanisms has suggested an important role for timing processes in impulsive choice. In addition, in an examination of genetic factors in impulsive choice, the Lewis rat strain emerged as a possible animal model for studying disordered impulsive choice, with this strain demonstrating deficient delay processing. Early rearing environment also affected impulsive behaviors, with rearing in an enriched environment promoting adaptable and more self-controlled choices. The combined results with impulsive choice suggest an important role for timing and reward sensitivity in moderating impulsive behaviors. Relative reward valuation also affects risky choice, with manipulation of objective reward value (relative to an alternative reference point) resulting in loss chasing behaviors that predicted overall risky choice behaviors. The combined results are discussed in relation to domain-specific versus domain-general subjective reward valuation processes and the potential neural substrates of impulsive and risky choice.

Abnormal gray matter and white matter volume in 'Internet gaming addicts'

Internet gaming addiction (IGA) is usually defined as the inability of an individual to control his/her use of the Internet with serious negative consequences. It is becoming a prevalent mental health concern around the world. To understand whether Internet gaming addiction contributes to cerebral structural changes, the present study examined the brain gray matter density and white matter density changes in participants suffering IGA using voxel-based morphometric analysis. Compared with the healthy controls (N=36, 22.2 ± 3.13 years), IGA participants (N=35, 22.28 ± 2.54 years) showed significant lower gray matter density in the bilateral inferior frontal gyrus, left cingulate gyrus, insula, right precuneus, and right hippocampus (all p<0.05). IGA participants also showed significant lower white matter density in the inferior frontal gyrus, insula, amygdala, and anterior cingulate than healthy controls (all p<0.05). Previous studies suggest that these brain regions are involved in decision-making, behavioral inhibition and emotional regulation. Current findings might provide insight in understanding the biological underpinnings of IGA.

Neurofeedback of slow cortical potentials: neural mechanisms and feasibility of a placebo-controlled design in healthy adults

To elucidate basic mechanisms underlying neurofeedback we investigated neural mechanisms of training of slow cortical potentials (SCPs) by considering EEG- and fMRI. Additionally, we analyzed the feasibility of a double-blind, placebo-controlled design in NF research based on regulation performance during treatment sessions and self-assessment of the participants. Twenty healthy adults participated in 16 sessions of SCPs training: 9 participants received regular SCP training, 11 participants received sham feedback. At three time points (pre, intermediate, post) fMRI and EEG/ERP-measurements were conducted during a continuous performance test (CPT). Performance-data during the sessions (regulation performance) in the treatment group and the placebo group were analyzed. Analysis of EEG-activity revealed in the SCP group a strong enhancement of the CNV (electrode Cz) at the intermediate assessment, followed by a decrease back to baseline at the post-treatment assessment. In contrast, in the placebo group a continuous but smaller increase of the CNV could be obtained from pre to post assessment. The increase of the CNV in the SCP group at intermediate testing was superior to the enhancement in the placebo group. The changes of the CNV were accompanied by a continuous improvement in the test performance of the CPT from pre to intermediate to post assessment comparable in both groups. The change of the CNV in the SCP group is interpreted as an indicator of neural plasticity and efficiency while an increase of the CNV in the placebo group might reflect learning and improved timing due to the frequent task repetition. In the fMRI analysis evidence was obtained for neuronal plasticity. After regular SCP neurofeedback activation in the posterior parietal cortex decreased from the pre- to the intermediate measurement and increased again in the post measurement, inversely following the U-shaped increase and decrease of the tCNV EEG amplitude in the SCP-trained group. Furthermore, we found a localized increase of activity in the anterior cingulate cortex (ACC). Analyses of the estimation of treatment assignment by the participants indicate feasibility of blinding. Participants could not assess treatment assignment confidently. Participants of the SCP-group improved regulation capability during treatment sessions (in contrast to the participants of the placebo-group), although regulation capability appeared to be instable, presumably due to diminished confidence in the training (SCP- or sham-training). Our results indicate that SCP training in healthy adults might lead to functional changes in neuronal circuits serving cognitive preparation even after a limited number of sessions.

Midcingulate Motor Map and Feedback Detection: Converging Data from Humans and Monkeys

The functional and anatomical organization of the cingulate cortex across primate species is the subject of considerable and often confusing debate. The functions attributed to the midcingulate cortex (MCC) embrace, among others, feedback processing, pain, salience, action-reward association, premotor functions, and conflict monitoring. This multiplicity of functional concepts suggests either unresolved separation of functional contributions or integration and convergence. We here provide evidence from recent experiments in humans and from a meta-analysis of monkey data that MCC feedback-related activity is generated in the rostral cingulate premotor area by specific body maps directly related to the modality of feedback. As such, we argue for an embodied mechanism for adaptation and exploration in MCC. We propose arguments and precise tools to resolve the origins of performance monitoring signals in the medial frontal cortex, and to progress on issues regarding homology between human and nonhuman primate cingulate cortex.

Intelligence moderates reinforcement learning: a mini-review of the neural evidence

Our understanding of the neural basis of reinforcement learning and intelligence, two key factors contributing to human strivings, has progressed significantly recently. However, the overlap of these two lines of research, namely, how intelligence affects neural responses during reinforcement learning, remains uninvestigated. A mini-review of three existing studies suggests that higher IQ (especially fluid IQ) may enhance the neural signal of positive prediction error in dorsolateral prefrontal cortex, dorsal anterior cingulate cortex, and striatum, several brain substrates of reinforcement learning or intelligence.

The neural network underlying incentive-based learning: implications for interpreting circuit disruptions in psychiatric disorders

Coupling stimuli and actions with positive or negative outcomes facilitates the selection of appropriate actions. Several brain regions are involved in the development of goal-directed behaviors and habit formation during incentive-based learning. This Review focuses on higher cognitive control of decision making and the cortical and subcortical structures and connections that attribute value to stimuli, associate that value with choices, and select an action plan. Delineating the connectivity between these areas is fundamental for understanding how brain regions work together to evaluate stimuli, develop actions plans, and modify behavior, as well as for elucidating the pathophysiology of psychiatric diseases.

Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value

Previous theories predict that human dorsal anterior cingulate (dACC) should respond to decision difficulty. An alternative theory has been recently advanced that proposes that dACC evolved to represent the value of 'non-default', foraging behavior, calling into question its role in choice difficulty. However, this new theory does not take into account that choosing whether or not to pursue foraging-like behavior can also be more difficult than simply resorting to a default. The results of two neuroimaging experiments show that dACC is only associated with foraging value when foraging value is confounded with choice difficulty; when the two are dissociated, dACC engagement is only explained by choice difficulty, and not the value of foraging. In addition to refuting this new theory, our studies help to formalize a fundamental connection between choice difficulty and foraging-like decisions, while also prescribing a solution for a common pitfall in studies of reward-based decision making.

Functional Neuroscience of Psychopathic Personality in Adults

Psychopathy is a personality disorder that involves a constellation of traits including callous-unemotionality, manipulativeness, and impulsiveness. Here we review recent advances in the research of functional neural correlates of psychopathic personality traits in adults. We first provide a concise overview of functional neuroimaging findings in clinical samples diagnosed with the PCL-R. We then review studies with community samples that have focused on how individual differences in psychopathic traits (variously measured) relate to individual differences in brain function. Where appropriate, we draw parallels between the findings from these studies and those with clinical samples. Extant data suggest that individuals with high levels of psychopathic traits show lower activity in affect-processing brain areas to emotional/salient stimuli, and that attenuated activity may be dependent on the precise content of the task. They also seem to show higher activity in regions typically associated with reward processing and cognitive control in tasks involving moral processing, decision making, and reward. Furthermore, affective-interpersonal and lifestyle-antisocial facets of psychopathy appear to be associated with different patterns of atypical neural activity. Neuroimaging findings from community samples typically mirror those observed in clinical samples, and largely support the notion that psychopathy is a dimensional construct.

Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders

The ability to learn contingencies between actions and outcomes in a dynamic environment is critical for flexible, adaptive behavior. Goal-directed actions adapt to changes in action-outcome contingencies as well as to changes in the reward-value of the outcome. When networks involved in reward processing and contingency learning are maladaptive, this fundamental ability can be lost, with detrimental consequences for decision-making. Impaired decision-making is a core feature in a number of psychiatric disorders, ranging from depression to schizophrenia. The argument can be developed, therefore, that seemingly disparate symptoms across psychiatric disorders can be explained by dysfunction within common decision-making circuitry. From this perspective, gaining a better understanding of the neural processes involved in goal-directed action, will allow a comparison of deficits observed across traditional diagnostic boundaries within a unified theoretical framework. This review describes the key processes and neural circuits involved in goal-directed decision-making using evidence from animal studies and human neuroimaging. Select studies are discussed to outline what we currently know about causal judgments regarding actions and their consequences, action-related reward evaluation, and, most importantly, how these processes are integrated in goal-directed learning and performance. Finally, we look at how adaptive decision-making is impaired across a range of psychiatric disorders and how deepening our understanding of this circuitry may offer insights into phenotypes and more targeted interventions.

Multiple neural mechanisms of decision making and their competition under changing risk pressure

Sometimes when a choice is made, the outcome is not guaranteed and there is only a probability of its occurrence. Each individual’s attitude to probability, sometimes called risk proneness or aversion, has been assumed to be static. Behavioral ecological studies, however, suggest such attitudes are dynamically modulated by the context an organism finds itself in; in some cases, it may be optimal to pursue actions with a low probability of success but which are associated with potentially large gains. We show that human subjects rapidly adapt their use of probability as a function of current resources, goals, and opportunities for further foraging. We demonstrate that dorsal anterior cingulate cortex (dACC) carries signals indexing the pressure to pursue unlikely choices and signals related to the taking of such choices. We show that dACC exerts this control over behavior when it, rather than ventromedial prefrontal cortex, interacts with posterior cingulate cortex.

•

Against common belief, risk preference is not fixed but is dynamically updated

•

Environments exert varying “risk pressure” tracked by ACC

•

Risk-related values and behavior are then regulated by ACC

•

Two mechanisms regulate competition for control of behavior between ACC and vmPFC

Dissociating contributions of ACC and vmPFC in reward prediction, outcome, and choice

Acting in an uncertain environment requires estimating the probability and the value of potential outcomes. These computations are typically ascribed to various parts of the medial prefrontal cortex (mPFC), but the functional architecture of this region remains debated. The anterior cingulate cortex (ACC) encodes reward prediction and outcome (i.e. win vs lose, Silvetti, Seurinck, & Verguts, 2013. Cortex, 49(6), 1627-35. doi:10.1016/j.cortex.2012.05.008). An outcome-related value signal has also been reported in the ventromedial Prefrontal Cortex (vmPFC, Rangel & Hare, 2010. Current Opinion in Neurobiology, 20(2), 262-70. doi:10.1016/j.conb.2010.03.001). Whether a functional dissociation can be traced in these regions with respect to reward prediction and outcome has been suggested but not rigorously tested. Hence an fMRI study was designed to systematically examine the contribution of ACC and vmPFC to reward prediction and outcome. A striking dissociation was identified, with ACC coding for positive prediction errors and vmPFC responding to outcome, irrespective of probability. Moreover, ACC has been assigned a crucial role in the selection of intentional actions (decision-making) and computing the value associated to these actions (action-based value). Conversely, vmPFC seems to implement stimulus-based value processing (Rudebeck et al., 2008. Journal of Neuroscience, 28(51), 13775-85. doi:10.1523/JNEUROSCI.3541-08.2008; Rushworth, Behrens, Rudebeck, & Walton, 2007. Trends in Cognitive Sciences, 11(4), 168-76. doi:10.1016/j.tics.2007.01.004). Therefore, a decision-making factor (choice vs. no choice condition) was also implemented in the present paradigm to distinguish stimulus-based versus action-based value coding in the mPFC during both decision and outcome phase. We found that vmPFC was more activated during the outcome phase in the no-choice than in the choice condition, potentially confirming the role of this area in stimulus-based (more than action-based) value processing.

Decision-making after continuous wins or losses in a randomized guessing task: implications for how the prior selection results affect subsequent decision-making

Background

Human decision-making is often affected by prior selections and their outcomes, even in situations where decisions are independent and outcomes are unpredictable.

Methods

In this study, we created a task that simulated real-life non-strategic gambling to examine the effect of prior outcomes on subsequent decisions in a group of male college students.

Results

Behavioral performance showed that participants needed more time to react after continuous losses (LOSS) than continuous wins (WIN) and discontinuous outcomes (CONTROL). In addition, participants were more likely to repeat their selections in both WIN and LOSS conditions. Functional MRI data revealed that decisions in WINs were associated with increased activation in the mesolimbic pathway, but decreased activation in the inferior frontal gyrus relative to LOSS. Increased prefrontal cortical activation was observed during LOSS relative to WIN and CONTROL conditions.

Conclusion

Taken together, the behavioral and neuroimaging findings suggest that participants tended to repeat previous selections during LOSS trials, a pattern resembling the gambler’s fallacy. However, during WIN trials, participants tended to follow their previous lucky decisions, like the ‘hot hand’ fallacy.

Using Neural Data to Test A Theory of Investor Behavior: An Application to Realization Utility

We use measures of neural activity provided by functional magnetic resonance imaging (fMRI) to test the "realization utility" theory of investor behavior, which posits that people derive utility directly from the act of realizing gains and losses. Subjects traded stocks in an experimental market while we measured their brain activity. We find that all subjects exhibit a strong disposition effect in their trading, even though it is suboptimal. Consistent with the realization utility explanation for this behavior, we find that activity in the ventromedial prefrontal cortex, an area known to encode the value of options during choices, correlates with the capital gains of potential trades; that the neural measures of realization utility correlate across subjects with their individual tendency to exhibit a disposition effect; and that activity in the ventral striatum, an area known to encode information about changes in the present value of experienced utility, exhibits a positive response when subjects realize capital gains. These results provide support for the realization utility model and, more generally, demonstrate how neural data can be helpful in testing models of investor behavior.

Comparative analysis of the macroscale structural connectivity in the macaque and human brain

The macaque brain serves as a model for the human brain, but its suitability is challenged by unique human features, including connectivity reconfigurations, which emerged during primate evolution. We perform a quantitative comparative analysis of the whole brain macroscale structural connectivity of the two species. Our findings suggest that the human and macaque brain as a whole are similarly wired. A region-wise analysis reveals many interspecies similarities of connectivity patterns, but also lack thereof, primarily involving cingulate regions. We unravel a common structural backbone in both species involving a highly overlapping set of regions. This structural backbone, important for mediating information across the brain, seems to constitute a feature of the primate brain persevering evolution. Our findings illustrate novel evolutionary aspects at the macroscale connectivity level and offer a quantitative translational bridge between macaque and human research.

What are the commonalities and differences of human brains when compared to the brains of other primates? The brain can be conceived as a complex network. Its topological properties constrain its function. Ethical and technical reasons necessitate the use of animal brains, like the macaque monkey, as models for the human brain. However, evolutionary changes, including “brain rewiring”, might result in unique human features. Hence, a detailed and quantitative comparative analysis of the connectivity of the brains of the two species is needed. Here, we undertake this task by adopting techniques analogous to those used in comparative studies in other scientific fields. Our approach reveals converging but also diverging wiring patterns. The brain of the two species as a whole is similarly wired. The majority of the brain regions appear to have evolutionary conserved connectivity patterns while for certain regions this appears not to be the case. We also uncover an evolutionary conserved “structural backbone” in the brain of the two species. Our findings highlight common and unique “wiring properties” of the brains of these two primate species and offer a quantitative basis for translating findings from macaque research to human research.

Neurophysiology of performance monitoring and adaptive behavior

Successful goal-directed behavior requires not only correct action selection, planning, and execution but also the ability to flexibly adapt behavior when performance problems occur or the environment changes. A prerequisite for determining the necessity, type, and magnitude of adjustments is to continuously monitor the course and outcome of one's actions. Feedback-control loops correcting deviations from intended states constitute a basic functional principle of adaptation at all levels of the nervous system. Here, we review the neurophysiology of evaluating action course and outcome with respect to their valence, i.e., reward and punishment, and initiating short- and long-term adaptations, learning, and decisions. Based on studies in humans and other mammals, we outline the physiological principles of performance monitoring and subsequent cognitive, motivational, autonomic, and behavioral adaptation and link them to the underlying neuroanatomy, neurochemistry, psychological theories, and computational models. We provide an overview of invasive and noninvasive systemic measures, such as electrophysiological, neuroimaging, and lesion data. We describe how a wide network of brain areas encompassing frontal cortices, basal ganglia, thalamus, and monoaminergic brain stem nuclei detects and evaluates deviations of actual from predicted states indicating changed action costs or outcomes. This information is used to learn and update stimulus and action values, guide action selection, and recruit adaptive mechanisms that compensate errors and optimize goal achievement.

Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation

Previous work indicates that economic decisions can be made independently of the visuomotor contingencies of the choice task (space of goods). However, the neuronal mechanisms through which the choice outcome (the chosen good) is transformed into a suitable action plan remain poorly understood. Here we show that neurons in lateral prefrontal cortex reflect the early stages of this good-to-action transformation. Monkeys chose between different juices. The experimental design dissociated in space and time the presentation of the offers and the saccade targets associated with them. We recorded from the orbital, ventrolateral, and dorsolateral prefrontal cortices (OFC, LPFCv, and LPFCd, respectively). Prior to target presentation, neurons in both LPFCv and LPFCd encoded the choice outcome in goods space. After target presentation, they gradually came to encode the location of the targets and the upcoming action plan. Consistent with the anatomical connectivity, all spatial and action-related signals emerged in LPFCv before LPFCd.

A novel role for the rat retrosplenial cortex in cognitive control

By virtue of its frontal and hippocampal connections, the retrosplenial cortex is uniquely placed to support cognition. Here, we tested whether the retrosplenial cortex is required for frontal tasks analogous to the Stroop Test, i.e., for the ability to select between conflicting responses and inhibit responding to task-irrelevant cues. Rats first acquired two instrumental conditional discriminations, one auditory and one visual, set in two distinct contexts. As a result, rats were rewarded for pressing either the right or left lever when a particular auditory or visual signal was present. In extinction, rats received compound stimuli that either comprised the auditory and visual elements that signaled the same lever response (congruent) or signaled different lever responses (incongruent) during training. On conflict (incongruent) trials, lever selection by sham-operated animals followed the stimulus element that had previously been trained in that same test context, whereas animals with retrosplenial cortex lesions failed to disambiguate the conflicting response cues. Subsequent experiments demonstrated that this abnormality on conflict trials was not due to a failure in distinguishing the contexts. Rather, these data reveal the selective involvement of the rat retrosplenial cortex in response conflict, and so extend the frontal system underlying cognitive control.

Single-neuron mechanisms underlying cost-benefit analysis in frontal cortex

Effective decision-making requires consideration of costs and benefits. Previous studies have implicated orbitofrontal cortex (OFC), dorsolateral prefrontal cortex (DLPFC), and anterior cingulate cortex (ACC) in cost-benefit decision-making. Yet controversy remains about whether different decision costs are encoded by different brain areas, and whether single neurons integrate costs and benefits to derive a subjective value estimate for each choice alternative. To address these issues, we trained four subjects to perform delay- and effort-based cost-benefit decisions and recorded neuronal activity in OFC, ACC, DLPFC, and the cingulate motor area (CMA). Although some neurons, mainly in ACC, did exhibit integrated value signals as if performing cost-benefit computations, they were relatively few in number. Instead, the majority of neurons in all areas encoded the decision type; that is whether the subject was required to perform a delay- or effort-based decision. OFC and DLPFC neurons tended to show the largest changes in firing rate for delay- but not effort-based decisions; whereas, the reverse was true for CMA neurons. Only ACC contained neurons modulated by both effort- and delay-based decisions. These findings challenge the idea that OFC calculates an abstract value signal to guide decision-making. Instead, our results suggest that an important function of single PFC neurons is to categorize sensory stimuli based on the consequences predicted by those stimuli.

Facets and mechanisms of adaptive pain behavior: predictive regulation and action

Neural mechanisms underlying nociception and pain perception are considered to serve the ultimate goal of limiting tissue damage. However, since pain usually occurs in complex environments and situations that call for elaborate control over behavior, simple avoidance is insufficient to explain a range of mammalian pain responses, especially in the presence of competing goals. In this integrative review we propose a Predictive Regulation and Action (PRA) model of acute pain processing. It emphasizes evidence that the nervous system is organized to anticipate potential pain and to adjust behavior before the risk of tissue damage becomes critical. Regulatory processes occur on many levels, and can be dynamically influenced by local interactions or by modulation from other brain areas in the network. The PRA model centers on neural substrates supporting the predictive nature of pain processing, as well as on finely-calibrated yet versatile regulatory processes that ultimately affect behavior. We outline several operational categories of pain behavior, from spinally-mediated reflexes to adaptive voluntary action, situated at various neural levels. An implication is that neural processes that track potential tissue damage in terms of behavioral consequences are an integral part of pain perception.

Updating expected action outcome in the medial frontal cortex involves an evaluation of error type

Forming expectations about the outcome of an action is an important prerequisite for action control and reinforcement learning in the human brain. The medial frontal cortex (MFC) has been shown to play an important role in the representation of outcome expectations, particularly when an update of expected outcome becomes necessary because an error is detected. However, error detection alone is not always sufficient to compute expected outcome because errors can occur in various ways and different types of errors may be associated with different outcomes. In the present study, we therefore investigate whether updating expected outcome in the human MFC is based on an evaluation of error type. Our approach was to consider an electrophysiological correlate of MFC activity on errors, the error-related negativity (Ne/ERN), in a task in which two types of errors could occur. Because the two error types were associated with different amounts of monetary loss, updating expected outcomes on error trials required an evaluation of error type. Our data revealed a pattern of Ne/ERN amplitudes that closely mirrored the amount of monetary loss associated with each error type, suggesting that outcome expectations are updated based on an evaluation of error type. We propose that this is achieved by a proactive evaluation process that anticipates error types by continuously monitoring error sources or by dynamically representing possible response-outcome relations.

Modulation of feedback-related negativity during trial-and-error exploration and encoding of behavioral shifts

The feedback-related negativity (FRN) is a mid-frontal event-related potential (ERP) recorded in various cognitive tasks and associated with the onset of sensory feedback signaling decision outcome. Some properties of the FRN are still debated, notably its sensitivity to positive and negative reward prediction error (RPE)—i.e., the discrepancy between the expectation and the actual occurrence of a particular feedback,—and its role in triggering the post-feedback adjustment. In the present study we tested whether the FRN is modulated by both positive and negative RPE. We also tested whether an instruction cue indicating the need for behavioral adjustment elicited the FRN. We asked 12 human subjects to perform a problem-solving task where they had to search by trial and error which of five visual targets, presented on a screen, was associated with a correct feedback. After exploration and discovery of the correct target, subjects could repeat their correct choice until the onset of a visual signal to change (SC) indicative of a new search. Analyses showed that the FRN was modulated by both negative and positive prediction error (RPE). Finally, we found that the SC elicited an FRN-like potential on the frontal midline electrodes that was not modulated by the probability of that event. Collectively, these results suggest the FRN may reflect a mechanism that evaluates any event (outcome, instruction cue) signaling the need to engage adaptive actions.

The expected value of control: an integrative theory of anterior cingulate cortex function

The dorsal anterior cingulate cortex (dACC) has a near-ubiquitous presence in the neuroscience of cognitive control. It has been implicated in a diversity of functions, from reward processing and performance monitoring to the execution of control and action selection. Here, we propose that this diversity can be understood in terms of a single underlying function: allocation of control based on an evaluation of the expected value of control (EVC). We present a normative model of EVC that integrates three critical factors: the expected payoff from a controlled process, the amount of control that must be invested to achieve that payoff, and the cost in terms of cognitive effort. We propose that dACC integrates this information, using it to determine whether, where and how much control to allocate. We then consider how the EVC model can explain the diverse array of findings concerning dACC function.

Neural substrates underlying effort computation in schizophrenia

The lack of initiative, drive or effort in patients with schizophrenia is linked to marked functional impairments. However, our assessment of effort and motivation is crude, relying on clinical rating scales based largely on patient recall. In order to better understand the neurobiology of effort in schizophrenia, we need more rigorous measurements of this construct. In the behavioural neuroscience literature, decades of work has been carried out developing various paradigms to examine the neural underpinnings of an animal's willingness to expend effort for a reward. Here, we shall review this literature on the nature of paradigms used in rodents to assess effort, as well as those used in humans. Next, the neurobiology of these effort-based decisions will be discussed. We shall then review what is known about effort in schizophrenia, and what might be inferred from experiments done in other human populations. Lastly, we shall discuss future directions of research that may assist in shedding light on the neurobiology of effort cost computations in schizophrenia.