Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems

Common neural dysfunction of economic decision-making across psychiatric conditions

Adaptive decision-making, which is often impaired in various psychiatric conditions, is essential for well-being. Recent evidence has indicated that decision-making capacity in multiple tasks could be accounted for by latent dimensions, enlightening the question of whether there is a common disruption of brain networks in economic decision-making across psychiatric conditions. Here, we addressed the issue by combining activation/lesion network mapping analyses with a transdiagnostic brain imaging meta-analysis. Our findings indicate that there were transdiagnostic alterations in the thalamus and ventral striatum during the decision or outcome stage of decision-making. The identified regions represent key nodes in a large-scale network, which is composed of multiple heterogeneous brain regions and plays a causal role in motivational functioning. The findings suggest that disturbances in the network associated with emotion- and reward-related processing play a key role in dysfunctions of decision-making observed in various psychiatric conditions. This study provides the first meta-analytic evidence of common neural alterations linked to deficits in economic decision-making.

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

Modulating risk-taking behavior with theta-band tACS

Although risk is prevalent in decision-making, the specific neural processes underlying risk-taking behavior remain unclear. Previous studies have suggested that frontal theta-band activity plays a crucial role in modulating risk-taking behavior. The functional relevance of theta in risk-taking behavior is yet to be clearly established and studies using noninvasive brain stimulation have yielded inconsistent findings. We aimed to investigate this relevance using transcranial alternating current stimulation (tACS) over right or left dorsolateral prefrontal cortex (DLPFC). We also studied the influence of stimulation intensity on risk-taking behavior and electrophysiological effects. We applied theta-band (6.5 Hz) tACS over the left (F3) and right (F4) DLPFC with lower (1.5 mA) and higher (3 mA) tACS intensities. We employed a single-blinded, sham-controlled, within-subject design and combined tACS with electroencephalography (EEG) measurements and the Maastricht Gambling Task (MGT) to elicit and evaluate risk-taking behavior. Our results show an increase in risk-taking behavior after left DLPFC stimulation at both intensities and a reduction of risk-taking behavior after 3 mA (and not 1.5 mA) right DLPFC stimulation compared to sham. Further analyses showed a negative correlation between resting-state frontal theta-power and risk-taking behavior. Overall, frontal theta-power was increased after left, but not right, theta-band tACS independent of stimulation intensity. Our findings confirm the functional relevance of frontal theta-band activity in decision-making under risk and the differential role of left and right DLPFC. We also were able to show that stimulation intensity did have an effect on behavioral responses, namely risk-taking behavior. Significant right hemisphere stimulation effects were observed only after high-intensity stimulation. Nevertheless, electrophysiological effects were only significant after left DLPFC stimulation, regardless of tACS intensity. Furthermore, the results indicate the role of the baseline frontal theta-power in the direction of behavioral effects after theta-band tACS.

Critical tests of fuzzy trace theory in brain and behavior: uncertainty across time, probability, and development

Uncertainty permeates decisions from the trivial to the profound. Integrating brain and behavioral evidence, we discuss how probabilistic (varied outcomes) and temporal (delayed outcomes) uncertainty differ across age and individuals; how critical tests adjudicate between theories of uncertainty (prospect theory and fuzzy-trace theory); and how these mechanisms might be represented in the brain. The same categorical gist representations of gains and losses account for choices and eye-tracking data in both value-allocation (add money to gambles) and risky-choice tasks, disconfirming prospect theory and confirming predictions of fuzzy-trace theory. The analysis is extended to delay discounting and disambiguated choices, explaining hidden-zero effects that similarly turn on categorical distinctions between some gain and no gain, certain gain and uncertain gain, gain and loss, and now and later. Bold activation implicates dorsolateral prefrontal and posterior parietal cortices in gist strategies that are not just one tool in a grab-bag of cognitive options but rather are general strategies that systematically predict behaviors across many different tasks involving probabilistic and temporal uncertainty. High valuation (e.g., ventral striatum; ventromedial prefrontal cortex) and low executive control (e.g., lateral prefrontal cortex) contribute to risky and impatient choices, especially in youth. However, valuation in ventral striatum supports reward-maximizing and gist strategies in adulthood. Indeed, processing becomes less "rational" in the sense of maximizing gains and more noncompensatory (eye movements indicate fewer tradeoffs) as development progresses from adolescence to adulthood, as predicted. Implications for theoretically predicted "public-health paradoxes" are discussed, including gist versus verbatim thinking in drug experimentation and addiction.

Uncertainty deconstructed: conceptual analysis and state-of-the-art review of the ERP correlates of risk and ambiguity in decision-making

Risk and uncertainty are central concepts of decision neuroscience. However, a comprehensive review of the literature shows that most studies define risk and uncertainty in an unclear fashion or use both terms interchangeably, which hinders the integration of the existing findings. We suggest uncertainty as an umbrella term that comprises scenarios characterized by outcome variance where relevant information about the type and likelihood of outcomes may be somewhat unavailable (ambiguity) and scenarios where the likelihood of outcomes is known (risk).These conceptual issues are problematic for studies on the temporal neurodynamics of decision-making under risk and ambiguity, because they lead to heterogeneity in task design and the interpretation of the results. To assess this problem, we conducted a state-of-the-art review of ERP studies on risk and ambiguity in decision-making. By employing the above definitions to 16 reviewed studies, our results suggest that: (a) research has focused more on risk than ambiguity processing; (b) studies assessing decision-making under risk often implemented descriptive-based paradigms, whereas studies assessing ambiguity processing equally implemented descriptive- and experience-based tasks; (c) descriptive-based studies link risk processing to increased frontal negativities (e.g., N2, N400) and both risk and ambiguity to reduced parietal positivities (e.g., P2, P3); (d) experience-based studies link risk to increased P3 amplitudes and ambiguity to increased frontal negativities and the LPC component; (e) both risk and ambiguity processing seem to be related with cognitive control, conflict monitoring, and increased cognitive demand; (f) further research and improved tasks are needed to dissociate risk and ambiguity processing.

Gambling on an empty stomach: Hunger modulates preferences for learned but not described risks

INTRODUCTION

We assess risks differently when they are explicitly described, compared to when we learn directly from experience, suggesting dissociable decision-making systems. Our needs, such as hunger, could globally affect our risk preferences, but do they affect described and learned risks equally? On one hand, decision-making from descriptions is often considered flexible and context sensitive, and might therefore be modulated by metabolic needs. On the other hand, preferences learned through reinforcement might be more strongly coupled to biological drives.

METHOD

Thirty-two healthy participants (females: 20, mean age: 25.6 ± 6.5 years) with a normal weight (Body Mass Index: 22.9 ± 3.2 kg/m2 ) were tested in a within-subjects counterbalanced, randomized crossover design for the effects of hunger on two separate risk-taking tasks. We asked participants to choose between two options with different risks to obtain monetary outcomes. In one task, the outcome probabilities were described numerically, whereas in a second task, they were learned.

RESULT

In agreement with previous studies, we found that rewarding contexts induced risk-aversion when risks were explicitly described (F1,31  = 55.01, p < .0001, ηp 2  = .64), but risk-seeking when they were learned through experience (F1,31  = 10.28, p < .003, ηp 2  = .25). Crucially, hunger attenuated these contextual biases, but only for learned risks (F1,31  = 8.38, p < .007, ηp 2  = .21).

CONCLUSION

The results suggest that our metabolic state determines risk-taking biases when we lack explicit descriptions.

Age-related differences in ERP correlates of value-based decision making

This study evaluates age-related differences in the temporal dynamics underlying neural processing of value for decision-making in younger and older adults. We applied a lottery-choice task with event-related potentials to determine how and when brain activity during choice and outcome processing diverge between younger and older adults. Behaviorally, older adults accepted more losing stakes than younger adults. During choice, younger adults evinced higher P2 ERP-response positivity with a later P3 positivity that monotonically increased with low to middle to high win probability. Older adults evinced lower P2 responses and P3 amplitudes with more positivity for high and low relative to middle win probability. Both age groups showed similar feedback-related negativity and late parietal positivity, indicating intact reward prediction error representations and salience integration. Feedback-P3 showed more complex sensitivity to expectancy violations in older than younger adults, suggesting subjective uncertainty about reward expectations. Reduced early general neural processing of objective stimulus value with greater contribution of downstream subjective processes might underlie older adult risk-taking behaviors.

Conceptualisation of Uncertainty in Decision Neuroscience Research: Do We Really Know What Types of Uncertainties The Measured Neural Correlates Relate To?

In the article "What are neural correlates neural correlates of?" published in the journal BioSocieties, Gabriel Abend points out that neuroscientists cannot avoid philosophical questions concerning the conceptualization and operationalization of social-psychological phenomena they deal with at the physiological level. In this article, we build on Abend's thesis and, through a systematic literature review of decision neuroscience studies, test it with the example of the social-psychological phenomenon of uncertainty in decision making. In this paper, we provide an overview of studies that appropriately attempt to conceptualise uncertainty, and then use these studies to analyse papers looking for neural correlates of uncertainty. Based on a systematic review of studies, we investigate what types of uncertainty authors in the field of decision neuroscience address and define, what criteria they use to distinguish between these types, what problems are associated with their conceptualization, and whether the neural correlates of different types of uncertainty can be accurately identified. The paper concludes that, particularly in the economic context, a collaboration between the natural and social sciences works well, and neuroscience studies use economic conceptualizations of uncertainty that are further developed by sophisticated decision tasks. However, the paper also highlights problematic aspects that obscure the understanding of the phenomena under study. These include the lack of criteria for distinguishing between different types of phenomena, the unclear use of the general concept of uncertainty, and the confusion of phenomena or their erroneous synonymous use.

The functional relevance of right DLPFC and VMPFC in risk-taking behavior

BACKGROUND

The prefrontal cortex can be partialized in various anatomical and functional sub regions. Among those regions, both right dorsolateral prefrontal cortex (rDLPFC) and ventromedial prefrontal cortex (VMPFC) have been associated with risk-taking behavior based on neuroimaging studies. Noninvasive brain stimulation (NIBS) studies aiming at demonstrating the functional relevance of neural activity in these areas almost exclusively focused on the rDLPFC, where its experimental stimulation with a (generally) inhibitory protocol lead to a measurable increase in risk-taking behavior due to reduced cognitive control. The functional relevance of VMPFC in risk-taking behavior has not yet been addressed using NIBS, although multiple neuroimaging studies correlate this area's activity with valuation.

OBJECTIVE/HYPOTHESIS

Here, we used NIBS to investigate the functional relevance of both, the rDLPFC and VMPFC in risk-taking behavior. We hypothesized that, compared to sham stimulation, VMPFC suppression leads to a reduction in risk-taking behavior by reducing the appeal to higher value options and consequently the attractiveness of riskier options, whereas rDLPFC suppression leads to an increase in risk taking, replicating previous findings.

METHODS

We applied continuous theta burst stimulation (cTBS), a generally inhibitory protocol, to stimulate either VMPFC or DLPFC before the execution of the computerized Maastricht Gambling Task (MGT) in a within-subject design with 30 participants. The MGT allowed the analysis of potential brain region-specific effects of cTBS on risk-taking behavior such as participants' choices of average values, probabilities, and response time.

RESULTS

cTBS applied to either rDLPFC or VMPFC both led to an increase in risk-taking behavior and in the average value chosen as compared to sham transcranial magnetic stimulation. No effect on the choice of probabilities was found. A significant increase in response time was observed exclusively after suppressing rDLPFC. We speculate that these similar behavioral consequences following cTBS over DLPFC and VMPFC are likely due to the strong anatomical and functional interconnection between both brain regions.

Emotions as computations

Emotions ubiquitously impact action, learning, and perception, yet their essence and role remain widely debated. Computational accounts of emotion aspire to answer these questions with greater conceptual precision informed by normative principles and neurobiological data. We examine recent progress in this regard and find that emotions may implement three classes of computations, which serve to evaluate states, actions, and uncertain prospects. For each of these, we use the formalism of reinforcement learning to offer a new formulation that better accounts for existing evidence. We then consider how these distinct computations may map onto distinct emotions and moods. Integrating extensive research on the causes and consequences of different emotions suggests a parsimonious one-to-one mapping, according to which emotions are integral to how we evaluate outcomes (pleasure & pain), learn to predict them (happiness & sadness), use them to inform our (frustration & content) and others' (anger & gratitude) actions, and plan in order to realize (desire & hope) or avoid (fear & anxiety) uncertain outcomes.

Behavioral and neural representation of expected reward and risk

When faced with uncertainty, individuals' value-based decisions are influenced by the expected rewards and risks. Understanding how reward and risk are processed and integrated at the behavioral and neural levels is essential for building up utility theories. Using a modified monetary incentive delay task in which the mean of two possible outcomes (expected reward) and the standard deviation (SD) of the possible outcomes (risk) were parametrically manipulated and orthogonalized, we measured eye movements, response times (RTs), and brain activity when participants seek to secure a reward. We found that RTs varied as a function of the mean but not the SD of the potential reward, suggesting that expected rewards are the main driver of RTs. Moreover, the difference between gazes focused on high vs. low value rewards became smaller when the magnitude of the potential reward (mean of possible outcomes) was larger and when risk (SD of possible outcomes) became smaller, highlighting that reward and risk have different effects on attention deployment. Processing the mean reward activated the striatum. The positive striatal connectivity to the amygdala and negative striatal connectivity to the superior frontal gyrus were correlated with individuals' sensitivity to the expected reward. In contrast, processing risk activated the anterior insula. Its positive connectivity to the ventromedial prefrontal cortex and negative connectivity to the anterior midcingulate cortex were correlated with individual differences in risk sensitivity, further suggesting the functional dissociation of reward and risk at the neural level. Our findings, based on several different measures, delineate the distinct representations of reward and risk in non-decision contexts and provide insight into how these utility parameters modulate attention, motivation, and brain networks.

A neuronal prospect theory model in the brain reward circuitry

Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals’ choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.

It is unclear how the activity of individual neurons conform to prospect theory. Here, the authors demonstrate that the activity of single neurons in various reward-related regions in the monkey brain can be described as encoding a multiplicative combination of utility and probability weighting, and that this subjective valuation process is achieved via a distributed coding scheme.

The neural substrates of risky rewards and losses in healthy volunteers and patient groups: a PET imaging study

BACKGROUND

Risk is an essential trait of most daily decisions. Our behaviour when faced with risks involves evaluation of many factors including the outcome probabilities, the valence (gains or losses) and past experiences. Several psychiatric disorders belonging to distinct diagnostic categories, including pathological gambling and addiction, show pathological risk-taking and implicate abnormal dopaminergic, opioidergic and serotonergic neurotransmission. In this study, we adopted a transdiagnostic approach to delineate the neurochemical substrates of decision making under risk.

METHODS

We recruited 39 participants, including 17 healthy controls, 15 patients with pathological gambling and seven binge eating disorder patients, who completed an anticipatory risk-taking task. Separately, participants underwent positron emission tomography (PET) imaging with three ligands, [¹⁸F]fluorodopa (FDOPA), [¹¹C]MADAM and [¹¹C]carfentanil to assess presynaptic dopamine synthesis capacity and serotonin transporter and mu-opioid receptor binding respectively.

RESULTS

Risk-taking behaviour when faced with gains positively correlated with dorsal cingulate [¹¹C]carfentanil binding and risk-taking to losses positively correlated with [¹¹C]MADAM binding in the caudate and putamen across all subjects.

CONCLUSIONS

We show distinct neurochemical substrates underlying risk-taking with the dorsal cingulate cortex mu-opioid receptor binding associated with rewards and dorsal striatal serotonin transporter binding associated with losses. Risk-taking and goal-directed control appear to dissociate between dorsal and ventral fronto-striatal systems. Our findings thus highlight the potential role of pharmacological agents or neuromodulation on modifying valence-specific risk-taking biases.

Dorsolateral prefrontal cortex plays causal role in probability weighting during risky choice

In this study, we provide causal evidence that the dorsolateral prefrontal cortex (DLPFC) supports the computation of subjective value in choices under risk via its involvement in probability weighting. Following offline continuous theta-burst transcranial magnetic stimulation (cTBS) of the DLPFC subjects (N = 30, mean age 23.6, 56% females) completed a computerized task consisting of 96 binary lottery choice questions presented in random order. Using the hierarchical Bayesian modeling approach, we then estimated the structural parameters of risk preferences (the degree of risk aversion and the curvature of the probability weighting function) and analyzed the obtained posterior distributions to determine the effect of stimulation on model parameters. On a behavioral level, temporary downregulation of the left DLPFC excitability through cTBS decreased the likelihood of choosing an option with higher expected reward while the probability of choosing a riskier lottery did not significantly change. Modeling the stimulation effects on risk preference parameters showed anecdotal evidence as assessed by Bayes factors that probability weighting parameter increased after the left DLPFC TMS compared to sham.

Neuroprotection in late life attention-deficit/hyperactivity disorder: A review of pharmacotherapy and phenotype across the lifespan

For decades, psychostimulants have been the gold standard pharmaceutical treatment for attention-deficit/hyperactivity disorder (ADHD). In the United States, an astounding 9% of all boys and 4% of girls will be prescribed stimulant drugs at some point during their childhood. Recent meta-analyses have revealed that individuals with ADHD have reduced brain volume loss later in life (>60 y.o.) compared to the normal aging brain, which suggests that either ADHD or its treatment may be neuroprotective. Crucially, these neuroprotective effects were significant in brain regions (e.g., hippocampus, amygdala) where severe volume loss is linked to cognitive impairment and Alzheimer's disease. Historically, the ADHD diagnosis and its pharmacotherapy came about nearly simultaneously, making it difficult to evaluate their effects in isolation. Certain evidence suggests that psychostimulants may normalize structural brain changes typically observed in the ADHD brain. If ADHD itself is neuroprotective, perhaps exercising the brain, then psychostimulants may not be recommended across the lifespan. Alternatively, if stimulant drugs are neuroprotective, then this class of medications may warrant further investigation for their therapeutic effects. Here, we take a bottom-up holistic approach to review the psychopharmacology of ADHD in the context of recent models of attention. We suggest that future studies are greatly needed to better appreciate the interactions amongst an ADHD diagnosis, stimulant treatment across the lifespan, and structure-function alterations in the aging brain.

Sometimes I feel the fear of uncertainty: How intolerance of uncertainty and trait anxiety impact fear acquisition, extinction and the return of fear

It is hypothesized that the ability to discriminate between threat and safety is impaired in individuals with high dispositional negativity, resulting in maladaptive behavior. A large body of research investigated differential learning during fear conditioning and extinction protocols depending on individual differences in intolerance of uncertainty (IU) and trait anxiety (TA), two closely-related dimensions of dispositional negativity, with heterogenous results. These might be due to varying degrees of induced threat/safety uncertainty. Here, we compared two groups with high vs. low IU/TA during periods of low (instructed fear acquisition) and high levels of uncertainty (delayed non-instructed extinction training and reinstatement). Dependent variables comprised subjective (US expectancy, valence, arousal), psychophysiological (skin conductance response, SCR, and startle blink), and neural (fMRI BOLD) measures of threat responding. During fear acquisition, we found strong threat/safety discrimination for both groups. During early extinction (high uncertainty), the low IU/TA group showed an increased physiological response to the safety signal, resulting in a lack of CS discrimination. In contrast, the high IU/TA group showed strong initial threat/safety discrimination in physiology, lacking discriminative learning on startle, and reduced neural activation in regions linked to threat/safety processing throughout extinction training indicating sustained but non-adaptive and rigid responding. Similar neural patterns were found after the reinstatement test. Taken together, we provide evidence that high dispositional negativity, as indicated here by IU and TA, is associated with greater responding to threat cues during the beginning of delayed extinction, and, thus, demonstrates altered learning patterns under changing environments.

Blunted reward prediction error signals in internet gaming disorder

BACKGROUND

Internet gaming disorder (IGD) is a type of behavioural addictions. One of the key features of addiction is the excessive exposure to addictive objectives (e.g. drugs) reduces the sensitivity of the brain reward system to daily rewards (e.g. money). This is thought to be mediated via the signals expressed as dopaminergic reward prediction error (RPE). Emerging evidence highlights blunted RPE signals in drug addictions. However, no study has examined whether IGD also involves alterations in RPE signals that are observed in other types of addictions.

METHODS

To fill this gap, we used functional magnetic resonance imaging data from 45 IGD and 42 healthy controls (HCs) during a reward-related prediction-error task and utilised a psychophysiological interaction (PPI) analysis to characterise the underlying neural correlates of RPE and related functional connectivity.

RESULTS

Relative to HCs, IGD individuals showed impaired reinforcement learning, blunted RPE signals in multiple regions of the brain reward system, including the right caudate, left orbitofrontal cortex (OFC), and right dorsolateral prefrontal cortex (DLPFC). Moreover, the PPI analysis revealed a pattern of hyperconnectivity between the right caudate, right putamen, bilateral DLPFC, and right dorsal anterior cingulate cortex (dACC) in the IGD group. Finally, linear regression suggested that the connection between the right DLPFC and right dACC could significantly predict the variation of RPE signals in the left OFC.

CONCLUSIONS

These results highlight disrupted RPE signalling and hyperconnectivity between regions of the brain reward system in IGD. Reinforcement learning deficits may be crucial underlying characteristics of IGD pathophysiology.

Advances in modeling learning and decision-making in neuroscience

An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.

Striatal BOLD and Midfrontal Theta Power Express Motivation for Action

Action selection is biased by the valence of anticipated outcomes. To assess mechanisms by which these motivational biases are expressed and controlled, we measured simultaneous EEG-fMRI during a motivational Go/NoGo learning task (N = 36), leveraging the temporal resolution of EEG and subcortical access of fMRI. VmPFC BOLD encoded cue valence, importantly predicting trial-by-trial valence-driven response speed differences and EEG theta power around cue onset. In contrast, striatal BOLD encoded selection of active Go responses and correlated with theta power around response time. Within trials, theta power ramped in the fashion of an evidence accumulation signal for the value of making a "Go" response, capturing the faster responding to reward cues. Our findings reveal a dual nature of midfrontal theta power, with early components reflecting the vmPFC contribution to motivational biases, and late components reflecting their striatal translation into behavior, in line with influential recent "value of work" theories of striatal processing.

Testing models at the neural level reveals how the brain computes subjective value

In recent years, models have played an increasingly important role for understanding the brain in cognitive, behavioral, and systems neuroscience. Decision neuroscience in particular has benefitted greatly from the application of economic models of choice preferences to neural data. However, an often-overlooked aspect is that many models of preferences have a generic problem—they make extremely similar behavioral predictions. Here, we demonstrate that to understand the mechanisms of valuation in the brain, it is useful to compare models of choice preferences not only at the behavioral but also at the neural level.

Decisions are based on the subjective values of choice options. However, subjective value is a theoretical construct and not directly observable. Strikingly, distinct theoretical models competing to explain how subjective values are assigned to choice options often make very similar behavioral predictions, which poses a major difficulty for establishing a mechanistic, biologically plausible explanation of decision-making based on behavior alone. Here, we demonstrate that model comparison at the neural level provides insights into model implementation during subjective value computation even though the distinct models parametrically identify common brain regions as computing subjective value. We show that frontal cortical regions implement a model based on the statistical distributions of available rewards, whereas intraparietal cortex and striatum compute subjective value signals according to a model based on distortions in the representations of probabilities. Thus, better mechanistic understanding of how cognitive processes are implemented arises from model comparisons at the neural level, over and above the traditional approach of comparing models at the behavioral level alone.

Additively Combining Utilities and Beliefs: Research Gaps and Algorithmic Developments

Value-based decision making in complex environments, such as those with uncertain and volatile mapping of reward probabilities onto options, may engender computational strategies that are not necessarily optimal in terms of normative frameworks but may ensure effective learning and behavioral flexibility in conditions of limited neural computational resources. In this article, we review a suboptimal strategy - additively combining reward magnitude and reward probability attributes of options for value-based decision making. In addition, we present computational intricacies of a recently developed model (named MIX model) representing an algorithmic implementation of the additive strategy in sequential decision-making with two options. We also discuss its opportunities; and conceptual, inferential, and generalization issues. Furthermore, we suggest future studies that will reveal the potential and serve the further development of the MIX model as a general model of value-based choice making.

Neuro-Computational Foundations of Moral Preferences

Moral preferences pervade many aspects of our lives, dictating how we ought to behave, whom we can marry, and even what we eat. Despite their relevance, one fundamental question remains unanswered: Where do individual moral preferences come from? It is often thought that all types of preferences reflect properties of domain-general neural decision mechanisms that employ a common "neural currency" to value choice options in many different contexts. This view, however, appears at odds with the observation that many humans consider it intuitively wrong to employ the same scale to compare moral value (e.g., of a human life) with material value (e.g., of money). In this paper, we directly test if moral subjective values are represented by similar neural processes as financial subjective values. In a study combining fMRI with a novel behavioral paradigm, we identify neural representations of the subjective values of human lives or financial payoffs by means of structurally identical computational models. Correlating isomorphic model variables from both domains with brain activity reveals specific patterns of neural activity that selectively represent values in the moral (rTPJ) or financial (vmPFC) domain. Intriguingly, our findings show that human lives and money are valued in (at least partially) distinct neural currencies, supporting theoretical proposals that human moral behavior is guided by processes that are distinct from those underlying behavior driven by personal material benefit.

Aberrant orbitofrontal cortex reactivity to erotic cues in Compulsive Sexual Behavior Disorder

BACKGROUND AND AIMS

Compulsive Sexual Behavior Disorder (CSBD) is characterized by increased reactivity to erotic reward cues. Cue-encoded reward parameters, such as type (e.g. erotic or monetary) or probability of anticipated reward, shape reward-related motivational processes, increase the attractiveness of cues and therefore might enhance maladaptive behavioral patterns in CSBD. Studies on the neural patterns of cue processing in individuals with CSBD have been limited mainly to ventral striatal responses. Therefore, here we aimed to examine the cue reactivity of multiple key structures in the brain's reward system, taking into account not only the type of predicted reward but also its probability.

METHODS

Twenty Nine men seeking professional help due to CSBD and 24 healthy volunteers took part in an fMRI study with a modified Incentive Delay Task with erotic and monetary rewards preceded by cues indicating a 25%, 50%, or 75% chance of reward. Analyses of functional patterns of activity related to cue type and probability were conducted on the whole-brain and ROI levels.

RESULTS

Increased anticipatory response to cues predictive of erotic rewards was observed among CSBD participants when compared to controls, in the ventral striatum and anterior orbitofrontal cortex (aOFC). The activity in aOFC was modulated by reward probability.

DISCUSSION AND CONCLUSIONS

Type of anticipated reward (erotic vs monetary) affects reward-related behavioral motivation in CSBD more strongly than reward probability. We present evidence of abnormal aOFC function in CSBD by demonstrating the recruitment of additional subsections of this region by erotic reward cues.

Motivational system modulates brain responses during exploratory decision-making

Managers face risk in explorative decision-making and those who are better at such decisions can achieve future viability. To understand what makes a manager effective at explorative decision-making requires an analysis of the manager’s motivational characteristics. The behavioral activation/inhibition system (BAS/BIS), fitting the motivational orientation of “approach” or “avoidance,” can affect individual decision-making. However, very little is known about the neural correlates of BAS/BIS orientation and their interrelationship with the mental activity during explorative decision-making. We conducted an fMRI study on 111 potential managers to investigate how the brain responses of explorative decision-making interact with BAS/BIS. Participants were separated into high- and low-performance groups based on the median exploration-score. The low-performance group showed significantly higher BAS than that of the high-performance group, and its BAS had significant negative association with neural networks related to reward-seeking during explorative decision-making. Moreover, the BIS of the low-performance group was negatively correlated with the activation of cerebral regions responding to risk-choice during explorative decision-making. Our finding showed that BAS/BIS was associated with the brain activation during explorative decision-making only in the low-performance group. This study contributed to the understanding of the micro-foundations of strategically relevant decision-making and has an implication for management development.

Activation patterns of the dorsal medial prefrontal cortex and frontal pole predict individual differences in decision impulsivity

Intertemporal choice refers to decisions that need to weigh different rewards at different time points in the future. Decision impulsivity manifests in the tendency of choosing smaller immediate options rather than larger later ones. Previous studies have suggested that decision impulsivity in intertemporal decision-making shares similar cognitive and neural mechanisms with risky decision-making. The present study theorizes on and examines whether the activation patterns of the dorsal medial prefrontal cortex (DMPFC) and the frontal pole (FP) during the risk-taking "cups task", as captured in the scanner, can predict the delay discounting rate (k) based on an intertemporal decision task performed outside the scanner. To this end, we scanned with functional magnetic resonance imaging (fMRI) techniques a sample of 257 college students (N = 257) while performing the cups task. Univariate analyses showed that activation levels of the DMPFC and the FP were inversely correlated with risk preference, but not with the delay discounting rate k. Multivariate pattern analysis, which can overcome key limitations of the univariate analyses, showed that activation patterns of these two regions predict the delay discounting rate k. These results confirmed the important roles of DMPFC and FP in decision impulsivity and the utility of using multivariate pattern analysis with fMRI data involving decision making tasks.

Neuroanatomical substrates accounting for the effect of present hedonistic time perspective on risk preference: the mediating role of right posterior parietal cortex

The preference for taking risk troubles people across multiple domains including health, economics, and social well-being. Prior research has demonstrated that risk preference can be influenced by time perspective (TP). However, little is known about the neural substrates underlying the effect of TP on risk preference. Here, we used a voxel-based morphometry (VBM) method across two samples to address this question. In Sample 1, the behavioral results showed a positive correlation between present hedonistic TP (PHTP) and gambling rate (the index of risk preference), indicating the higher PHTP, the greater the preference for risk. Subsequently, the whole-brain VBM results found that gambling rate was negatively correlated with the gray matter (GM) volume of a cluster in the right posterior parietal cortex (rPPC). The PHTP score was also negatively related to the GM volume of another cluster in the rPPC. We then examined an overlapping region in the rPPC using a conjunction analysis method. The GM volume of this overlapping brain region was related to both PHTP score and gambling rate. Finally, the mediation analysis found that the GM volume of overlapping region in rPPC played a role in explaining the effect of PHTP on risk preference. This result was also reproduced and validated in another independent sample. Taken together, our findings manifest that the structural variation of rPPC can account for the influence that PHTP has upon the risk preference.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Reaching Out for Food: How Food Incentives Modulate Peripersonal Space Perception

Two experiments were conducted to determine, first, whether food items influence participants’ estimations of the size of their subjective peripersonal space. It was of particular interest whether this representation is influenced by satiated/hungry states and is differentially affected by valence and calorie content of depicted stimuli. Second, event-related brain potentials (ERPs) were used, in order to obtain information about the time course of the observed effects and how they depend on the spatial location of the food pictures. For that purpose, participants had to decide whether food items shown at various distances along a horizontal plane in front of them, were reachable or not. In Experiment 1, when participants were hungry, they perceived an increase of their peripersonal space modulated by high-calorie items which were experienced as being more reachable than low-calorie items. In Experiment 2, the reachability findings were replicated and early and late components of ERPs showed an attentional enhancement in far space for food items when participants were hungry. These findings suggest that participants’ subjective peripersonal space increased while being hungry, especially for high-calorie contents. Attention also seems to be oriented more strongly to far space items due to their expected incentive-related salience, expanding the subjective representation of peripersonal space.

Reduced risk-taking behavior during frontal oscillatory theta band neurostimulation

BACKGROUND

Most of our decisions involve a certain degree of risk regarding the outcomes of our choices. People vary in the way they make decisions, resulting in different levels of risk-taking behavior. These differences have been linked to prefrontal theta band activity. However, a direct functional relationship between prefrontal theta band activity and risk-taking has not yet been demonstrated.

OBJECTIVE

We used noninvasive brain stimulation to test the functional relevance of prefrontal oscillatory theta activity for the regulatory control of risk-taking behavior.

METHODS

In a within-subject experiment, 31 healthy participants received theta (6.5 Hertz [Hz]), gamma (40 Hz), and sham transcranial alternating current stimulation (tACS) over the left prefrontal cortex (lPFC). During stimulation, participants completed a task assessing their risk-taking behavior as well as response times and sensitivity to value and outcome probabilities. Electroencephalography (EEG) was recorded before and immediately after stimulation to investigate possible long-lasting stimulation effects.

RESULTS

Theta band, but not gamma band or sham, tACS led to a significant reduction in risk-taking behavior, indicating a frequency-specific effect of prefrontal brain stimulation on the modulation of risk-taking behavior. Moreover, theta band stimulation led to increased response times and decreased sensitivity to reward values. EEG data analyses did not show an offline increase in power in the stimulated frequencies after the stimulation protocol.

CONCLUSION

These findings provide direct empirical evidence for the effects of prefrontal theta band stimulation on behavioral risk-taking regulation.

Neural Computations of Threat

A host of learning, memory, and decision-making processes form the individual's response to threat and may be disrupted in anxiety and post-trauma psychopathology. Here we review the neural computations of threat, from the first encounter with a dangerous situation, through learning, storing, and updating cues that predict it, to making decisions about the optimal course of action. The overview highlights the interconnected nature of these processes and their reliance on shared neural and computational mechanisms. We propose an integrative approach to the study of threat-related processes, in which specific computations are studied across the various stages of threat experience rather than in isolation. This approach can generate new insights about the evolution, diagnosis, and treatment of threat-related psychopathology.

Dorsomedial striatal contributions to different forms of risk/reward decision making

Optimal decision making involving reward uncertainty is integral to adaptive goal-directed behavior. In some instances, these decisions are guided by internal representations of reward history, whereas in other situations, external cues inform a decision maker about how likely certain actions are to yield reward. Different regions of the frontal lobe form distributed networks with striatal and amygdalar regions that facilitate different types of risk/reward decision making. The dorsal medial striatum (DMS) is one key output region of the prefrontal cortex, yet there have been few preclinical studies investigating the involvement of the DMS in different forms of risk/reward decision making. The present study addressed this issue, wherein separate groups of male rats were trained on one of two tasks where they chose between a small/certain or a large/risky reward. In a probabilistic discounting task, reward probabilities changed systematically over blocks of trials (100-6.25% or 6.25-100%), requiring rats to use internal representations of reward history to guide choice. Cue-guided decision-making was assessed with a "Blackjack" task, where different auditory cues indicated the odds associated with the large/risky option (50 or 12.5%). Inactivation of the DMS with GABA agonists impaired adjustments in choice biases during probabilistic discounting, resulting in either increases or decreases in risky choice as the probabilities associated with the large/risky reward decreased or increased over a session. In comparison, DMS inactivation increased risky choices on poor-odds trials on the Blackjack task, which was associated with a reduced impact that non-rewarded choices had on subsequent choices. DMS inactivation also impaired performance of an auditory conditional discrimination. These findings highlight a previously uncharacterized role for the DMS in facilitating flexible action selection during multiple forms of risk/reward decision making.

Uncertainty modulates value-driven attentional capture

The majority of previous studies on the value modulation of attention have shown that the magnitude of value-driven attentional bias correlates with the strength of reward association. However, relatively little is known about how uncertainty affects value-based attentional bias. We investigated whether attentional capture by previously rewarded stimuli is modulated by the uncertainty of the learned value without the influence of the strength of reward association. Participants were instructed to identify the line orientation in the target color circle. Importantly, each target color was associated with a different level of uncertainty by tuning the variation in reward delivery (Experiment 1) or reward magnitude (Experiment 2). Attentional interference for uncertainty-related distractors was greater than that for certainty distractors in Experiments 1 and 2. In addition, uncertainty-induced attentional bias disappeared earlier than attentional bias for certainty. The study demonstrated that uncertainty modulates value-based attentional capture in terms of strength and persistence, even when the effect of expected value remains constant.

Neural tracking of subjective value under riskand ambiguity in adolescence

Although many neuroimaging studies on adolescent risk taking have focused on brain activation during outcome valuation, less attention has been paid to the neural correlates of choice valuation. Subjective choice valuation may be particularly influenced by whether a choice presents risk (known probabilities) or ambiguity (unknown probabilities), which has rarely been studied in developmental samples. Therefore, we examined the neural tracking of subjective value during choice under risk and ambiguity in a large sample of adolescents (N = 188, 12–22 years). Specifically, we investigated which brain regions tracked subjective value coding under risk and ambiguity. A model-based approach to estimate individuals’ risk and ambiguity attitudes showed prominent variation in individuals’ aversions to risk and ambiguity. Furthermore, participants subjectively experienced the ambiguous options as being riskier than the risky options. Subjective value tracking under risk was coded by activation in ventral striatum and superior parietal cortex. Subjective value tracking under ambiguity was coded by dorsolateral prefrontal cortex (PFC) and superior temporal gyrus activation. Finally, overlapping activation in the dorsomedial PFC was observed for subjective value under both conditions. Overall, this is the first study to chart brain activation patterns for subjective choice valuation under risk and ambiguity in an adolescent sample, which shows that the building blocks for risk and ambiguity processing are already present in early adolescence. Finally, we highlight the potential of combining behavioral modeling with fMRI for investigating choice valuation in adolescence, which may ultimately aid in understanding who takes risks and why.

Ventral Caudate and Anterior Insula Recruitment During Value Estimation of Passionate Rewarding Cues

“Wanting”, a component of reward processing, is a motivational property that guides decision making in goal-oriented behavior. This includes behavior aiming at supporting relational bonds, even at the group level. Accordingly, group belongingness works as this motivational property, which is fundamentally different from romantic or maternal love. While primary rewards (or learned associations, such as money) have been largely used to study the conceptual framework associated with “wanting,” other cues triggering behavior, such as passionate motives, are less well-studied. We investigated the neural correlates of value estimation of a passion-driven incentive in neuropsychologically defined football fans. We asked the participants (n = 57) to compute the value of football tickets (the cues that trigger passionate behavior in this “tribal love” context). The trials were all different, comprising tickets for different matches. The participants had no restrictions on the amount to be introduced. This enabled a parametric functional magnetic resonance imaging design based on the explicit estimated value given by the participants in a trial-by-trial approach. Using a whole-brain approach (to prevent biased focus on value-related regions), only the activity in the ventral caudate and left anterior insula showed a critical relationship with the reported value. Higher normalized values led to more activity in the striatum and left insula. The parametric map shows that these regions encode the magnitude of incentive by indexing self-relevant value. Other regions were involved in value computation, such as the ventromedial prefrontal cortex, lateral orbitofrontal cortex, and dorsolateral prefrontal cortex, but did not exhibit parametric patterns. The involvement of the nucleus accumbens in value estimation was only found in region of interest -based analysis, which emphasizes the role of the ventral caudate for the presently studied social “reinforcer” cue.

Affective evaluation of others' altruistic decisions under risk and ambiguity

Gratitude arises when one is the target of an altruistic decision, particularly when this decision incurs cost to the agent. Here we examined how individuals evaluate others' altruistic decisions under risky (uncertainty with known probabilities) and ambiguous (uncertainty with unknown probabilities) costs and respond with gratitude and reciprocity. Participants played an interactive game in an fMRI scanner in which they would receive painful electric shocks. An anonymous co-player either intentionally (Human conditions) or unintentionally (Computer conditions) decided whether to help the participant reduce half of the pain by undertaking an amount of pain (i.e., cost) with varying level of uncertainty (Certain vs. Risky vs. Ambiguous). Participants could then transfer monetary points to the co-player knowing that the co-player was unaware of this transfer. Behaviorally, monetary allocation and gratitude rating increased as the uncertainty level of cost increased in Human conditions; these effects were reduced in Computer conditions. The effect of cost uncertainty on gratitude was mediated by the perceived kind intention behind the help. FMRI revealed both shared and differential neurocognitive substrates for evaluating the benefactor's altruistic decisions under risk and ambiguity: both were associated with fear- and anxiety-related processes, involving right lateral orbitofrontal cortex and anterior insula; ambiguity additionally recruited mentalizing- and conflict monitoring-related processes, involving dorsal medial prefrontal cortex and dorsal anterior cingulate cortex. These findings underscore the crucial role of social uncertainty perception in the generation of gratitude.

Dissociable neural correlates of uncertainty underlie different exploration strategies

Most real-world decisions involve a delicate balance between exploring unfamiliar alternatives and committing to the best known option. Previous work has shown that humans rely on different forms of uncertainty to negotiate this "explore-exploit" trade-off, yet the neural basis of the underlying computations remains unclear. Using fMRI (n = 31), we find that relative uncertainty is represented in right rostrolateral prefrontal cortex and drives directed exploration, while total uncertainty is represented in right dorsolateral prefrontal cortex and drives random exploration. The decision value signal combining relative and total uncertainty to compute choice is reflected in motor cortex activity. The variance of this signal scales with total uncertainty, consistent with a sampling mechanism for random exploration. Overall, these results are consistent with a hybrid computational architecture in which different uncertainty computations are performed separately and then combined by downstream decision circuits to compute choice.

Individual intelligence and brain neural correlates associated with outcome expectancies for risk behaviors in adults

Although adults have matured developments of general intelligence, brain structure and brain function, many people continue to be risk takers, despite the harm that can result. The neural basis underlying risk-taking behaviors has been studied extensively in adolescents, but less so in adults. Outcome expectancies are important factors influencing individuals' risk behaviors, which comprise the expected risks (ER) and expected benefits (EB) associated with risk behaviors. In the current study, we systematically investigated inter-individual differences in adults' outcome expectancies for risk behaviors, considering the general intelligence, brain function, and brain structure. At the intelligence level, individuals with higher intelligence scores showed lower ER but higher EB associated with risk behaviors. At the brain function level, resting-state functional connectivity (FC) between regions within the default mode network is negatively correlated with ER but positively correlated with EB associated with risk behaviors, while FC between the insula and motor cortex is negatively correlated with EB associated with the risk behaviors. At the brain structure level, gray matter volume in posterior cingulate cortex (PCC) and bilateral parahippocampus were negatively correlated with the ER associated with risk behaviors. Furthermore, the relationship between the outcome expectancy associated with risk behaviors and the FC between anterior cingulate cortex and PCC is partially mediated by the general intelligence. The current study provides new insight that furthers our understanding of how individual differences in adults' risk attitudes and behaviors are modulated by general intelligence and reflected in resting-state FC and brain structures related to self-reference and inhibitory control processing.

Context effects on probability estimation

Many decisions rely on how we evaluate potential outcomes and estimate their corresponding probabilities of occurrence. Outcome evaluation is subjective because it requires consulting internal preferences and is sensitive to context. In contrast, probability estimation requires extracting statistics from the environment and therefore imposes unique challenges to the decision maker. Here, we show that probability estimation, like outcome evaluation, is subject to context effects that bias probability estimates away from other events present in the same context. However, unlike valuation, these context effects appeared to be scaled by estimated uncertainty, which is largest at intermediate probabilities. Blood-oxygen-level-dependent (BOLD) imaging showed that patterns of multivoxel activity in the dorsal anterior cingulate cortex (dACC), ventromedial prefrontal cortex (VMPFC), and intraparietal sulcus (IPS) predicted individual differences in context effects on probability estimates. These results establish VMPFC as the neurocomputational substrate shared between valuation and probability estimation and highlight the additional involvement of dACC and IPS that can be uniquely attributed to probability estimation. Because probability estimation is a required component of computational accounts from sensory inference to higher cognition, the context effects found here may affect a wide array of cognitive computations.

This study shows how probability estimation can be affected by the context of our recent experience, namely, how the presence of multiple events experienced closed in time can influence their respective probability estimates.

Adaptive learning under expected and unexpected uncertainty

The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.

Neurocognitive Underpinnings of Aggressive Predation in Economic Contests

Competitions are part and parcel of daily life and require people to invest time and energy to gain advantage over others and to avoid (the risk of) falling behind. Whereas the behavioral mechanisms underlying competition are well documented, its neurocognitive underpinnings remain poorly understood. We addressed this using neuroimaging and computational modeling of individual investment decisions aimed at exploiting one's counterpart ("attack") or at protecting against exploitation by one's counterpart ("defense"). Analyses revealed that during attack relative to defense (i) individuals invest less and are less successful; (ii) computations of expected reward are strategically more sophisticated (reasoning level k = 4 vs. k = 3 during defense); (iii) ventral striatum activity tracks reward prediction errors; (iv) risk prediction errors were not correlated with neural activity in either ROI or whole-brain analyses; and (v) successful exploitation correlated with neural activity in the bilateral ventral striatum, left OFC, left anterior insula, left TPJ, and lateral occipital cortex. We conclude that, in economic contests, coming out ahead (vs. not falling behind) involves sophisticated strategic reasoning that engages both reward and value computation areas and areas associated with theory of mind.

Collective Optimism and Selection Against Male Twins in Utero

Scholarly literature claims that health declines in populations when optimism about investing in the future wanes. This claim leads us to describe collective optimism as a predictor of selection in utero. Based on the literature, we argue that the incidence of suicide gauges collective optimism in a population and therefore willingness to invest in the future. Using monthly data from Sweden for the years 1973-2016, we test the hypothesis that the incidence of suicide among women of child-bearing age correlates inversely with male twin births, an indicator of biological investment in high-risk gestations. We find that, as predicted by our theory, the incidence of suicide at month t varies inversely with the ratio of twin to singleton male births at month t + 3. Our results illustrate the likely sensitivity of selection in utero to change in the social environment and so the potential for viewing collective optimism as a component of public health infrastructure.

Neural Dynamics Underlying the Evaluation Process of Ambiguous Options During Reward-Related Decision-Making

Ambiguous decision-making involves different processes. However, few studies have focused on the evaluation process. In this study, event-related potentials (ERPs) and event-related spectrum perturbation (ERSP) techniques were used to explore the neural dynamics underlying the evaluation process of ambiguous options through an ambiguous choice task. Some important results emerged. We found a preference for lotteries with low ambiguity regardless of reward amount, suggesting that subjects were averse to ambiguity in our paradigm. Our electroencephalography (EEG) results clarified the neural dynamics underlying the evaluation process. In the time domain, lotteries with both a larger reward and lower ambiguity elicited a larger P3. In the time-frequency domain, larger amplitudes of delta activity at 200–400 ms and 500–600 ms post-stimulus were elicited by lotteries with low ambiguity. Moreover, lotteries with a larger reward elicited larger amplitudes of delta activity at 400–600 ms post-stimulus. Our ERPs and ERSP results suggested that individuals in our paradigm evaluated ambiguity and reward separately, and then integrated their evaluation to form subjective values of different lotteries.

Towards a Unifying Cognitive, Neurophysiological, and Computational Neuroscience Account of Schizophrenia

Psychotic experiences may be understood as altered information processing due to aberrant neural computations. A prominent example of such neural computations is the computation of prediction errors (PEs), which signal the difference between expected and experienced events. Among other areas showing PE coding, hippocampal-prefrontal-striatal neurocircuits play a prominent role in information processing. Dysregulation of dopaminergic signaling, often secondary to psychosocial stress, is thought to interfere with the processing of biologically important events (such as reward prediction errors) and result in the aberrant attribution of salience to irrelevant sensory stimuli and internal representations. Bayesian hierarchical predictive coding offers a promising framework for the identification of dysfunctional neurocomputational processes and the development of a mechanistic understanding of psychotic experience. According to this framework, mismatches between prior beliefs encoded at higher levels of the cortical hierarchy and lower-level (sensory) information can also be thought of as PEs, with important consequences for belief updating. Low levels of precision in the representation of prior beliefs relative to sensory data, as well as dysfunctional interactions between prior beliefs and sensory data in an ever-changing environment, have been suggested as a general mechanism underlying psychotic experiences. Translating the promise of the Bayesian hierarchical predictive coding into patient benefit will come from integrating this framework with existing knowledge of the etiology and pathophysiology of psychosis, especially regarding hippocampal-prefrontal-striatal network function and neural mechanisms of information processing and belief updating.

Common and Distinct Functional Brain Networks for Intuitive and Deliberate Decision Making

Reinforcement learning studies in rodents and primates demonstrate that goal-directed and habitual choice behaviors are mediated through different fronto-striatal systems, but the evidence is less clear in humans. In this study, functional magnetic resonance imaging (fMRI) data were collected whilst participants (n = 20) performed a conditional associative learning task in which blocks of novel conditional stimuli (CS) required a deliberate choice, and blocks of familiar CS required an intuitive choice. Using standard subtraction analysis for fMRI event-related designs, activation shifted from the dorso-fronto-parietal network, which involves dorsolateral prefrontal cortex (DLPFC) for deliberate choice of novel CS, to ventro-medial frontal (VMPFC) and anterior cingulate cortex for intuitive choice of familiar CS. Supporting this finding, psycho-physiological interaction (PPI) analysis, using the peak active areas within the PFC for novel and familiar CS as seed regions, showed functional coupling between caudate and DLPFC when processing novel CS and VMPFC when processing familiar CS. These findings demonstrate separable systems for deliberate and intuitive processing, which is in keeping with rodent and primate reinforcement learning studies, although in humans they operate in a dynamic, possibly synergistic, manner particularly at the level of the striatum.

Rewards interact with repetition-dependent learning to enhance long-term retention of motor memories

The combination of behavioral experiences that enhance long-term retention remains largely unknown. Informed by neurophysiological lines of work, this study tested the hypothesis that performance-contingent monetary rewards potentiate repetition-dependent forms of learning, as induced by extensive practice at asymptote, to enhance long-term retention of motor memories. To this end, six groups of 14 participants (n = 84) acquired novel motor behaviors by adapting to a gradual visuomotor rotation while these factors were manipulated. Retention was assessed 24 h later. While all groups similarly acquired the novel motor behaviors, results from the retention session revealed an interaction indicating that rewards enhanced long-term retention, but only when practice was extended to asymptote. Specifically, the interaction indicated that this effect selectively occurred when rewards were intermittently available (i.e., 50%), but not when they were absent (i.e., 0%) or continuously available (i.e., 100%) during acquisition. This suggests that the influence of rewards on extensive practice and long-term retention is nonlinear, as continuous rewards did not further enhance retention as compared with intermittent rewards. One possibility is that rewards' intermittent availability allows to maintain their subjective value during acquisition, which may be key to potentiate long-term retention.

Biophysical Properties of Somatic and Axonal Voltage-Gated Sodium Channels in Midbrain Dopaminergic Neurons

Spiking activities of midbrain dopaminergic neurons are critical for key brain functions including motor control and affective behaviors. Voltage-gated Na⁺ channels determine neuronal excitability and action potential (AP) generation. Previous studies on dopaminergic neuron excitability mainly focused on Na⁺ channels at the somatodendritic compartments. Properties of axonal Na⁺ channels, however, remain largely unknown. Using patch-clamp recording from somatic nucleated patches and isolated axonal blebs from the axon initial segment (AIS) of dopaminergic neurons in mouse midbrain slices, we found that AIS channel density is approximately 4–9 fold higher than that at the soma. Similar voltage dependence of channel activation and inactivation was observed between somatic and axonal channels in both SNc and VTA cells, except that SNc somatic channels inactivate at more hyperpolarized membrane potentials (V_m). In both SNc and VTA, axonal channels take longer time to inactivate at a subthreshold depolarization V_m level, but are faster to recover from inactivation than somatic channels. Moreover, we found that immunosignals of Nav1.2 accumulate at the AIS of dopaminergic neurons. In contrast, Nav1.1 and Nav1.6 immunosignals are not detectible. Together, our results reveal a high density of Na⁺ channels at the AIS and their molecular identity. In general, somatic and axonal channels of both SNc and VTA dopaminergic neurons share similar biophysical properties. The relatively delayed inactivation onset and faster recovery from inactivation of axonal Na⁺ channels may ensure AP initiation at high frequencies and faithful signal conduction along the axon.

Subjective Beliefs About Trust and Reciprocity Activate an Expected Reward Signal in the Ventral Striatum

There is overwhelming evidence that the evaluation of both reward decisions and their associated outcomes are closely linked with bilateral activation of the ventral striatum, with these insights stemming from tasks such as the monetary incentive delay task for lotteries and multiround Trust Games for social settings. The essential element in these tasks is an externally provided cue associated with specific gains/trustworthy partners and losses/non-trustworthy partners. However, in reality people typically use their own beliefs to guide their decision-making and assess the likelihood of positive or and negative outcomes. As when participants assess the relationship between cues and rewards, individuals should anticipate rewards in correspondence to their beliefs, i.e., the higher the belief of obtaining a reward in the future, the higher the anticipation of reward. In this study, we use decision-makers’ own, naturally occurring, beliefs about both social and non-social contexts to examine the subsequent outcome of their choices. We hypothesize that mechanisms of belief-mediated reward processing are mediated by neural activation in the ventral striatum. An essential feature of our design is the elicitation of individuals’ beliefs prior to the decision-making task itself. Furthermore, our incentivized, non-deceptive, decision-making task distinguishes between social – implemented by a Trust Game – and non-social sources, as well as risk and ambiguity as underlying types of uncertainty. Our main result shows that individual beliefs regarding reciprocity likelihoods in both the Trust Game and the lottery influence the amount invested. Subsequently, only the investment amount in the Trust Game parametrically modulates anticipatory reward and outcome evaluation in the ventral striatum. This study demonstrates a first approach at using participants’ subjective sets of beliefs to examine reward processing. We discuss its potential promise, outline some limitations, and propose follow-up studies to extend the current approach.

Hierarchical and Nonlinear Dynamics in Prefrontal Cortex Regulate the Precision of Perceptual Beliefs

Actions are shaped not only by the content of our percepts but also by our confidence in them. To study the cortical representation of perceptual precision in decision making, we acquired functional imaging data whilst participants performed two vibrotactile forced-choice discrimination tasks: a fast-slow judgment, and a same-different judgment. The first task requires a comparison of the perceived vibrotactile frequencies to decide which one is faster. However, the second task requires that the estimated difference between those frequencies is weighed against the precision of each percept-if both stimuli are very precisely perceived, then any slight difference is more likely to be identified than if the percepts are uncertain. We additionally presented either pure sinusoidal or temporally degraded "noisy" stimuli, whose frequency/period differed slightly from cycle to cycle. In this way, we were able to manipulate the perceptual precision. We report a constellation of cortical regions in the rostral prefrontal cortex (PFC), dorsolateral PFC (DLPFC) and superior frontal gyrus (SFG) associated with the perception of stimulus difference, the presence of stimulus noise and the interaction between these factors. Dynamic causal modeling (DCM) of these data suggested a nonlinear, hierarchical model, whereby activity in the rostral PFC (evoked by the presence of stimulus noise) mutually interacts with activity in the DLPFC (evoked by stimulus differences). This model of effective connectivity outperformed competing models with serial and parallel interactions, hence providing a unique insight into the hierarchical architecture underlying the representation and appraisal of perceptual belief and precision in the PFC.