Neuroeconomics: making risky choices in the brain

Sub-optimality in motor planning is retained throughout 9 days practice of 2250 trials

Optimality in motor planning, as well as accuracy in motor execution, is required to maximize expected gain under risk. In this study, we tested whether humans are able to update their motor planning. Participants performed a coincident timing task with an asymmetric gain function, in which optimal response timing to gain the highest total score depends on response variability. Their behaviours were then compared using a Bayesian optimal decision model. After 9 days of practicing 2250 trials, the total score increased, and temporal variance decreased. On the other hand, the participants showed consistent risk-seeking or risk-averse behaviour, preserving suboptimal motor planning. These results suggest that a human's computational ability to calculate an optimal motor plan is limited, and it is difficult to improve it through repeated practice with a score feedback.

Motor planning under temporal uncertainty is suboptimal when the gain function is asymmetric

For optimal action planning, the gain/loss associated with actions and the variability in motor output should both be considered. A number of studies make conflicting claims about the optimality of human action planning but cannot be reconciled due to their use of different movements and gain/loss functions. The disagreement is possibly because of differences in the experimental design and differences in the energetic cost of participant motor effort. We used a coincident timing task, which requires decision making with constant energetic cost, to test the optimality of participant's timing strategies under four configurations of the gain function. We compared participant strategies to an optimal timing strategy calculated from a Bayesian model that maximizes the expected gain. We found suboptimal timing strategies under two configurations of the gain function characterized by asymmetry, in which higher gain is associated with higher risk of zero gain. Participants showed a risk-seeking strategy by responding closer than optimal to the time of onset/offset of zero gain. Meanwhile, there was good agreement of the model with actual performance under two configurations of the gain function characterized by symmetry. Our findings show that human ability to make decisions that must reflect uncertainty in one's own motor output has limits that depend on the configuration of the gain function.

Neural reward and punishment sensitivity in cigarette smokers

BACKGROUND

Nicotine addiction remains a major public health problem but the neural substrates of addictive behavior remain unknown. One characteristic of smoking behavior is impulsive choice, selecting the immediate reward of smoking despite the potential long-term negative consequences. This suggests that drug users, including cigarette smokers, may be more sensitive to rewards and less sensitive to punishment.

METHODS

We used event-related potentials (ERPs) to test the hypothesis that smokers are more responsive to reward signals and less responsive to punishment, potentially predisposing them to risky behavior. We conducted two experiments, one using a reward prediction design to elicit a Medial Frontal Negativity (MFN) and one using a reward- and punishment-motivated flanker task to elicit an Error Related Negativity (ERN), ERP components thought to index activity in the cortical projection of the dopaminergic reward system.

RESULTS AND CONCLUSIONS

The smokers had a greater MFN response to unpredicted rewards, and non-smokers, but not smokers, had a larger ERN on punishment motivated trials indicating that smokers are more reward sensitive and less punishment sensitive than nonsmokers, overestimating the appetitive value and underestimating aversive outcomes of stimuli and actions.

Nonhuman gamblers: lessons from rodents, primates, and robots

The search for neuronal and psychological underpinnings of pathological gambling in humans would benefit from investigating related phenomena also outside of our species. In this paper, we present a survey of studies in three widely different populations of agents, namely rodents, non-human primates, and robots. Each of these populations offers valuable and complementary insights on the topic, as the literature demonstrates. In addition, we highlight the deep and complex connections between relevant results across these different areas of research (i.e., cognitive and computational neuroscience, neuroethology, cognitive primatology, neuropsychiatry, evolutionary robotics), to make the case for a greater degree of methodological integration in future studies on pathological gambling.

Decision-making theories: linking the disparate research areas of individual and collective cognition

In order to maximize their fitness, animals have to deal with different environmental and social factors that affect their everyday life. Although the way an animal behaves might enhance its fitness or survival in regard to one factor, it could compromise them regarding another. In the domain of decision sciences, research concerning decision making focuses on performances at the individual level but also at the collective one. However, between individual and collective decision making, different terms are used resulting in little or no connection between both research areas. In this paper, we reviewed how different branches of decision sciences study the same concept, mainly called speed-accuracy trade-off, and how the different results are on the same track in terms of showing the optimality of decisions. Whatever the level, individual or collective, each decision might be defined with three parameters: time or delay to decide, risk and accuracy. We strongly believe that more progress would be possible in this domain of research if these different branches were better linked, with an exchange of their results and theories. A growing amount of literature describes economics in humans and eco-ethology in birds making compromises between starvation, predation and reproduction. Numerous studies have been carried out on social cognition in primates but also birds and carnivores, and other publications describe market or reciprocal exchanges of commodities. We therefore hope that this paper will lead these different areas to a common decision science.

Contextual factors explain risk-seeking preferences in rhesus monkeys

In contrast to humans and most other animals, rhesus macaques strongly prefer risky rewards to safe ones with similar expected value. Why macaques prefer risk while other animals typically avoid it remains puzzling and challenges the idea that monkeys provide a model for human economic behavior. Here we argue that monkeys’ risk-seeking preferences are neither mysterious nor unique. Risk-seeking in macaques is possibly induced by specific elements of the tasks that have been used to measure their risk preferences. The most important of these elements are (1) very small stakes, (2) serially repeated gambles with short delays between trials, and (3) task parameters that are learned through experience, not described verbally. Together, we hypothesize that these features will readily induce risk-seeking in monkeys, humans, and rats. Thus, elements of task design that are often ignored when comparing studies of risk attitudes can easily overwhelm basal risk preferences. More broadly, these results highlight the fundamental importance of understanding the psychological basis of economic decisions in interpreting preference data and corresponding neural measures.

A behavioral and neural evaluation of prospective decision-making under risk

Making the best choice when faced with a chain of decisions requires a person to judge both anticipated outcomes and future actions. Although economic decision-making models account for both risk and reward in single-choice contexts, there is a dearth of similar knowledge about sequential choice. Classical utility-based models assume that decision-makers select and follow an optimal predetermined strategy, regardless of the particular order in which options are presented. An alternative model involves continuously reevaluating decision utilities, without prescribing a specific future set of choices. Here, using behavioral and functional magnetic resonance imaging (fMRI) data, we studied human subjects in a sequential choice task and use these data to compare alternative decision models of valuation and strategy selection. We provide evidence that subjects adopt a model of reevaluating decision utilities, in which available strategies are continuously updated and combined in assessing action values. We validate this model by using simultaneously acquired fMRI data to show that sequential choice evokes a pattern of neural response consistent with a tracking of anticipated distribution of future reward, as expected in such a model. Thus, brain activity evoked at each decision point reflects the expected mean, variance, and skewness of possible payoffs, consistent with the idea that sequential choice evokes a prospective evaluation of both available strategies and possible outcomes.

Neuroeconomic approaches to mental disorders

The pervasiveness of decision-making in every area of human endeavor highlights the importance of understanding choice mechanisms and their detailed relationship to underlying neurobiological function. This review surveys the recent and productive application of game-theoretic probes (economic games) to mental disorders. Such games typically possess concrete concepts of optimal play, thus providing quantitative ways to track when subjects' choices match or deviate from optimal. This feature equips economic games with natural classes of control signals that should guide learning and choice in the agents that play them. These signals and their underlying physical correlates in the brain are now being used to generate objective biomarkers that may prove useful for exposing and understanding the neurogenetic basis of normal and pathological human cognition. Thus, game-theoretic probes represent some of the first steps toward producing computationally principled, objective measures of cognitive function and dysfunction useful for the diagnosis, treatment, and understanding of mental disorders.

Brain correlates of risky decision-making

Understanding the neurocognitive basis of risk-taking behavior is an important issue, especially in economic decision-making. Classical behavioral studies have shown that risk-attitude changes across different contexts, but little is so far known about the brain correlates of processing of outcomes across such context shifts. In this study, EEG was recorded while subjects performed a gambling task. Participants could choose between a risky and a safer option, within two different contexts: one in which options yielded gains and losses of the same magnitude (Zero Expected Value context) and another in which gains were larger than losses (Positive Expected Value context). Based on their risk-attitude, two groups were compared: subjects who are risk-seekers in the zero Expected Value context (Zero-Oriented group) and subjects who are risk-seekers in the positive Expected Value condition (Positive-Oriented group). The Feedback Related Negativity (FRN) reflects this distinction, with each group being insensitive to magnitude of outcomes in the condition in which they were risk-prone. P300 amplitude mirrored the behavioral results, with larger amplitudes in the condition in which each group showed a higher risk-tendency. Source analyses highlighted the involvement of posterior cingulate cortex in risky decision-making. Taken together, the findings make a contribution to the clarification of the neurocognitive substrates of risky decision-making.

To Bet or Not to Bet? The Error Negativity or Error-related Negativity Associated with Risk-taking Choices

The functional significance of error-related negativity (Ne/ERN), which occurs at approximately the same time as erroneous responses, has been investigated extensively using reaction time (RT) tasks. The error detection theory assumes that the Ne/ERN reflects the mismatch detected by comparing representations of the intended and the actually performed actions. The conflict monitoring theory asserts that the Ne/ERN reflects the detection of response conflict between intended and actually performed actions during response selection. In this study, we employed a gambling task in which participants were required to choose whether they would take part in betting in each trial and they were presented with gain or loss feedback in both the “to bet” and the “not to bet” trials. The response-locked ERP magnitudes were more negative for “to bet” than for “not to bet” choices for both large and small stakes and were more negative for choices involving large rather than small stakes. Dipole source analysis localized the ERP responses to the anterior cingulate cortex (ACC). These findings suggest that the ACC signals the riskiness of choices and may function as an early warning system that alerts the brain to prepare for the potential negative consequence associated with a risky action.

Neural correlates of economic game playing

The theory of games provides a mathematical formalization of strategic choices, which have been studied in both economics and neuroscience, and more recently has become the focus of neuroeconomics experiments with human and non-human actors. This paper reviews the results from a number of game experiments that establish a unitary system for forming subjective expected utility maps in the brain, and acting on these maps to produce choices. Social situations require the brain to build an understanding of the other person using neuronal mechanisms that share affective and intentional mental states. These systems allow subjects to better predict other players' choices, and allow them to modify their subjective utility maps to value pro-social strategies. New results for a trust game are presented, which show that the trust relationship includes systems common to both trusting and trustworthy behaviour, but they also show that the relative temporal positions of first and second players require computations unique to that role.

The role of strategies in deciding advantageously in ambiguous and risky situations

In decision situations of everyday life, the potential positive or negative consequences of a decision are often specified and the associated probabilities are known or they are principally calculable ("decisions under risk"). On the basis of correlations reported in patient studies, it has been recently proposed that decisions under risk involve strategic components, i.e. calculation of the risk, as well as emotional processes, i.e. processing feedback from previous decisions. However, the potential impact of calculative strategies on decision-making under risk has not been investigated systematically, so far. In the current study, we examined 42 healthy subjects (21 females) with the Game of Dice Task measuring decisions under risk, and a questionnaire assessing strategy application in items comparable to the choices in the Game of Dice Task. In addition, the subjects performed the Iowa Gambling Task, examining decision-making under ambiguity, and a neuropsychological test battery focusing on executive functions. Results indicate that deciding advantageously in a decision-making task with explicit and stable rules is linked to applying calculative strategies. In contrast, individuals who decide intuitively prefer risky or disadvantageous choices in the Game of Dice Task. Applying calculative strategies was correlated with executive functions but not with performance on the Iowa Gambling Task. The results support the view that calculative processes and strategies may improve decision-making under explicit risk conditions.