From bad to worse: striatal coding of the relative value of painful decisions

Modulation of aversive value coding in the vertebrate and invertebrate brain

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.

Subjective utility moderates bidirectional effects of conflicting motivations on pain perception

Minimizing pain and maximizing pleasure are conflicting motivations when pain and reward co-occur. Decisions to prioritize reward consumption or pain avoidance are assumed to lead to pain inhibition or facilitation, respectively. Such decisions are a function of the subjective utility of the stimuli involved, i.e. the relative value assigned to the stimuli to compare the potential outcomes of a decision. To test perceptual pain modulation by varying degrees of motivational conflicts and the role of subjective utility, we implemented a task in which healthy volunteers had to decide between accepting a reward at the cost of receiving a nociceptive electrocutaneous stimulus or rejecting both. Subjective utility of the stimuli was assessed by a matching task between the stimuli. Accepting reward coupled to a nociceptive stimulus resulted in decreased perceived intensity, while rejecting the reward to avoid pain resulted in increased perceived intensity, but in both cases only if a high motivational conflict was present. Subjective utility of the stimuli involved moderated these bidirectional perceptual effects: the more a person valued money over pain, the more perceived intensity increased or decreased. These findings demonstrate pain modulation when pain and reward are simultaneously present and highlight the importance of subjective utility for such modulation.

The Effect of Wealth Shocks on Loss Aversion: Behavior and Neural Correlates

Kahneman and Tversky (1979) first demonstrated that when individuals decide whether or not to accept a gamble, potential losses receive more weight than possible gains in the decision. This phenomenon is referred to as loss aversion. We investigated how loss aversion in risky financial decisions is influenced by sudden changes to wealth, employing both behavioral and neurobiological measures. We implemented an fMRI experimental paradigm, based on that employed by Tom et al. (2007). There are two treatments, called RANDOM and CONTINGENT. In RANDOM, the baseline setting, the changes to wealth, referred to as wealth shocks in economics, are independent of the actual choices participants make. Under CONTINGENT, we induce the belief that the changes in income are a consequence of subjects' own decisions. The magnitudes and sequence of the shocks to wealth are identical between the CONTINGENT and RANDOM treatments. We investigated whether more loss aversion existed in one treatment than another. The behavioral results showed significantly greater loss aversion in CONTINGENT compared to RANDOM after a negative wealth shock. No differences were observed in the response to positive shocks. The fMRI results revealed a neural loss aversion network, comprising the bilateral striatum, amygdala and dorsal anterior cingulate cortex that was common to the CONTINGENT and RANDOM tasks. However, the ventral prefrontal cortex, primary somatosensory cortex and superior occipital cortex, showed greater activation in response to a negative change in wealth due to individual's own decisions than when the change was exogenous. These results indicate that striatum activation correlates with loss aversion independently of the source of the shock, and that the ventral prefrontal cortex (vPFC) codes the experimental manipulation of agency in one's actions influencing loss aversion.

Is loss-aversion magnitude-dependent? Measuring prospective affective judgments regarding gains and losses

Prospect Theory proposed that the (dis)utility of losses is always more than gains due to a phenomena called ‘loss-aversion’, a result obtained in multiple later studies over the years. However, some researchers found reversed or no loss-aversion for affective judgments of small monetary amounts but, those findings have been argued to stem from the way gains versus losses were measured. Thus, it was not clear whether loss-aversion does not show with affective judgments for smaller magnitudes, or it is a measurement error. This paper addresses the debate concerning loss-aversion (in the prospect theoretic sense) and judgments about the intensity of gains and losses. We measured affective prospective judgments for monetary amounts using measurement scales that have been argued to be suitable for measuring loss-aversion and hence rule out any explanations regarding measurement. Both in a gambling scenario (Experiments 1 and 2) and in the context of fluctuating prices (Experiments 3a and 3b), potential losses never loomed larger than gains for low magnitudes, indicating that it is not simply a measurement error. Moreover, for the same participant, loss aversion was observable at high magnitudes. Further, we show that loss-aversion disappears even for higher monetary values, if contextually an even larger anchor is provided. The results imply that Prospect Theory’s value function is contextually dependent on magnitudes.

Neural loss aversion differences between depression patients and healthy individuals: A functional MRI investigation

Neuroeconomics employs neuroscience techniques to explain decision-making behaviours. Prospect theory, a prominent model of decision-making, features a value function with parameters for risk and loss aversion. Recent work with normal participants identified activation related to loss aversion in brain regions including the amygdala, ventral striatum, and ventromedial prefrontal cortex. However, the brain network for loss aversion in pathologies such as depression has yet to be identified. The aim of the current study is to employ the value function from prospect theory to examine behavioural and neural manifestations of loss aversion in depressed and healthy individuals to identify the neurobiological markers of loss aversion in economic behaviour. We acquired behavioural data and fMRI scans while healthy controls and patients with depression performed an economic decision-making task. Behavioural loss aversion was higher in patients with depression than in healthy controls. fMRI results revealed that the two groups shared a brain network for value function including right ventral striatum, ventromedial prefrontal cortex, and right amygdala. However, the neural loss aversion results revealed greater activations in the right dorsal striatum and the right anterior insula for controls compared with patients with depression, and higher activations in the midbrain region ventral tegmental area for patients with depression compared with controls. These results suggest that while the brain network for loss aversion is shared between depressed and healthy individuals, some differences exist with respect to differential activation of additional areas. Our findings are relevant to identifying neurobiological markers for altered decision-making in the depressed.

Relative valuation of pain in human orbitofrontal cortex

The valuation of health-related states, including pain, is a critical issue in clinical practice, health economics, and pain neuroscience. Surprisingly the monetary value people associate with pain is highly context-dependent, with participants willing to pay more to avoid medium-level pain when presented in a context of low-intensity, rather than high-intensity, pain. Here, we ask whether context impacts upon the neural representation of pain itself, or alternatively the transformation of pain into valuation-driven behavior. While undergoing fMRI, human participants declared how much money they would be willing to pay to avoid repeated instances of painful cutaneous electrical stimuli delivered to the foot. We also implemented a contextual manipulation that involved presenting medium-level painful stimuli in blocks with either low- or high-level stimuli. We found no evidence of context-dependent activity within a conventional "pain matrix," where pain-evoked activity reflected absolute stimulus intensity. By contrast, in right lateral orbitofrontal cortex, a strong contextual dependency was evident, and here activity tracked the contextual rank of the pain. The findings are in keeping with an architecture where an absolute pain valuation system and a rank-dependent system interact to influence willing to pay to avoid pain, with context impacting value-based behavior high in a processing hierarchy. This segregated processing hints that distinct neural representations reflect sensory aspects of pain and components that are less directly nociceptive whose integration also guides pain-related actions. A dominance of the latter might account for puzzling phenomena seen in somatization disorders where perceived pain is a dominant driver of behavior.

Risky monetary behavior in chronic back pain is associated with altered modular connectivity of the nucleus accumbens

Background

The nucleus accumbens (NAc) has a well established role in reward processing. Yet, there is growing evidence showing that NAc function, and its connections to other parts of the brain, is also critically involved in the emergence of chronic back pain (CBP). Pain patients are known to perform abnormally in reward-related tasks, which suggests an intriguing link between pain, NAc connectivity, and reward behavior. In the present study, we compared performance on a gambling task (indicating willingness to risk losing money) between healthy pain-free controls (CON) and individuals with CBP. We then measured modular connectivity of each participants’ NAc with resting state functional MRI to investigate how connectivity accounts for reward behavior in the presence and absence of pain.

Results

We found gain sensitivity was significantly higher in CBP patients. These scores were significantly correlated to connectivity within the NAc module defined by CON subjects ( which had strong connections to the frontal cortex), but not within that defined by CBP patients ( which was more strongly connected to subcortical areas). An important part of our study was based on the precedence that a range of behaviors, from simple to complex, can be predicted from brain activity during rest. Thus, to corroborate our results we compared them closely to an independent study correlating the same connectivity metric to impulsive behaviors in healthy participants. We found that our CBP patients were highly similarin connectivity to this study’s highly-impulsive healthy subjects, strengthening the notion that there is an important link between the brain systems that support chronic pain and reward processing.

Conclusions

Our results support previous findings that chronic back pain is accompanied by altered connectivity of the NAc. This lends itself to riskier behavior in these patients, a finding which establishes a potential cognitive consequence or co-morbidity of long-term pain and provides a behavioral link to growing research showing that chronic pain is related to abnormal changes in the dopaminergic system.

Informatic parcellation of the network involved in the computation of subjective value

Understanding how the brain computes value is a basic question in neuroscience. Although individual studies have driven this progress, meta-analyses provide an opportunity to test hypotheses that require large collections of data. We carry out a meta-analysis of a large set of functional magnetic resonance imaging studies of value computation to address several key questions. First, what is the full set of brain areas that reliably correlate with stimulus values when they need to be computed? Second, is this set of areas organized into dissociable functional networks? Third, is a distinct network of regions involved in the computation of stimulus values at decision and outcome? Finally, are different brain areas involved in the computation of stimulus values for different reward modalities? Our results demonstrate the centrality of ventromedial prefrontal cortex (VMPFC), ventral striatum and posterior cingulate cortex (PCC) in the computation of value across tasks, reward modalities and stages of the decision-making process. We also find evidence of distinct subnetworks of co-activation within VMPFC, one involving central VMPFC and dorsal PCC and another involving more anterior VMPFC, left angular gyrus and ventral PCC. Finally, we identify a posterior-to-anterior gradient of value representations corresponding to concrete-to-abstract rewards.

Aversive stimuli and loss in the mesocorticolimbic dopamine system

There is mounting evidence that the mesolimbic dopamine system carries valuation signals not only for appetitive or gain-related stimuli, with which it is traditionally associated, but also for aversive and loss-related stimuli. Cellular-level studies demonstrate that the neuronal architecture to support aversive stimuli encoding in this system does exist. Both cellular-level and human neuroimaging research suggest the co-existence of appetitive and aversive prediction-error signals within the mesocorticolimbic system. These findings shift the view of the mesocorticolimbic system as a singular pathway for reward to a system with multiple signals across a wide range of domains that drive value-based decision making.

Contextual and social influences on valuation and choice

To survive in our complex environment, we have to adapt to changing contexts. Prior research that investigated how contextual changes are processed in the human brain has demonstrated important modulatory influences on multiple cognitive processes underlying decision-making, including perceptual judgments, working memory, as well as cognitive and attentional control. However, in everyday life, the importance of context is even more obvious during economic and social interactions, which often have implicit rule sets that need to be recognized by a decision-maker. Here, we review recent evidence from an increasing number of studies in the fields of Neuroeconomics and Social Neuroscience that investigate the neurobiological basis of contextual effects on valuation and social choice. Contrary to the assumptions of rational choice theory, multiple contextual factors, such as the availability of alternative choice options, shifts in reference point, and social context, have been shown to modulate behavior, as well as signals in task-relevant neural networks. A consistent picture that emerges from neurobiological results is that valuation-related activity in striatum and ventromedial prefrontal cortex is highly context dependent during both social and nonsocial choice. Alternative approaches to model and explain choice behavior, such as comparison-based choice models, as well as implications for future research are discussed.

How costs influence decision values for mixed outcomes

The things that we hold dearest often require a sacrifice, as epitomized in the maxim “no pain, no gain.” But how is the subjective value of outcomes established when they consist of mixtures of costs and benefits? We describe theoretical models for the integration of costs and benefits into a single value, drawing on both the economic and the empirical literatures, with the goal of rendering them accessible to the neuroscience community. We propose two key assays that go beyond goodness of fit for deciding between the dominant additive model and four varieties of interactive models. First, how they model decisions between costs when reward is not on offer; and second, whether they predict changes in reward sensitivity when costs are added to outcomes, and in what direction. We provide a selective review of relevant neurobiological work from a computational perspective, focusing on those studies that illuminate the underlying valuation mechanisms. Cognitive neuroscience has great potential to decide which of the theoretical models is actually employed by our brains, but empirical work has yet to fully embrace this challenge. We hope that future research improves our understanding of how our brain decides whether mixed outcomes are worthwhile.

Neural insensitivity to upticks in value is associated with the disposition effect

The disposition effect is a phenomenon in which investors hold onto losing assets longer than they hold onto gaining assets. In this study, we used functional magnetic resonance imaging (fMRI) to measure the response of valuation regions in the brain during the decision to keep or to sell an asset that followed a random walk in price. The most common explanation for the disposition effect is preference-based: namely, that people are risk-averse over gains and risk-seeking over losses. This explanation would predict correlations between individuals' risk-preferences, the magnitude of their disposition effect, and activation in valuation structures of the brain. We did not observe these correlations. Nor did we find evidence for a realization utility explanation, which would predict differential responses in valuation regions during the decision to sell versus keep an asset that correlated with the magnitude of the disposition effect. Instead, we found an attenuated ventral striatum response to upticks in value below the purchase price in some individuals with a large disposition effect. Given the role of the striatum in signaling prediction error, the blunted striatal response is consistent with the expectation that an asset will rise when it is below the purchase price, thus spurring loss-holding behavior. This suggests that for some individuals, the disposition effect is likely driven by a belief that the asset will eventually return to the purchase price, also known as mean reversion.

The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives

Remarkable progress has been made in recent years toward understanding the functions of the orbitofrontal cortex (OFC). The finding that neurons in this area encode the subjective value monkeys assign to different goods while making choices has been confirmed and extended by numerous studies using both primate neurophysiology and human imaging. Moreover, new lesion studies demonstrated that subjective values computed in the OFC are causally and specifically related to choice behavior. Importantly, values in the OFC are attached to goods, not to actions or to spatial locations. Furthermore, subjective values appear to be computed in this area even if the situation does not require a choice. In the light of this growing body of work, we propose that the primary function of the OFC is the computation of good identities and subjective values in an abstract representation. In this view, OFC neurons compute the subjective value of a good whenever that good is behaviorally relevant.

Neurobiology of economic choice: a good-based model

Traditionally the object of economic theory and experimental psychology, economic choice recently became a lively research focus in systems neuroscience. Here I summarize the emerging results and propose a unifying model of how economic choice might function at the neural level. Economic choice entails comparing options that vary on multiple dimensions. Hence, while choosing, individuals integrate different determinants into a subjective value; decisions are then made by comparing values. According to the good-based model, the values of different goods are computed independently of one another, which implies transitivity. Values are not learned as such, but rather computed at the time of choice. Most importantly, values are compared within the space of goods, independent of the sensorimotor contingencies of choice. Evidence from neurophysiology, imaging, and lesion studies indicates that abstract representations of value exist in the orbitofrontal and ventromedial prefrontal cortices. The computation and comparison of values may thus take place within these regions.

Neurobiology of value integration: when value impacts valuation

Everyday choice options have advantages (positive values) and disadvantages (negative values) that need to be integrated into an overall subjective value. For decades, economic models have assumed that when a person evaluates a choice option, different values contribute independently to the overall subjective value of the option. However, human choice behavior often violates this assumption, suggesting interactions between values. To investigate how qualitatively different advantages and disadvantages are integrated into an overall subjective value, we measured the brain activity of human subjects using fMRI while they were accepting or rejecting choice options that were combinations of monetary reward and physical pain. We compared different subjective value models on behavioral and neural data. These models all made similar predictions of choice behavior, suggesting that behavioral data alone are not sufficient to uncover the underlying integration mechanism. Strikingly, a direct model comparison on brain data decisively demonstrated that interactive value integration (where values interact and affect overall valuation) predicts neural activity in value-sensitive brain regions significantly better than the independent mechanism. Furthermore, effective connectivity analyses revealed that value-dependent changes in valuation are associated with modulations in subgenual anterior cingulate cortex-amygdala coupling. These results provide novel insights into the neurobiological underpinnings of human decision making involving the integration of different values.