Visual fixations and the computation and comparison of value in simple choice

From Trust in Automation to Decision Neuroscience: Applying Cognitive Neuroscience Methods to Understand and Improve Interaction Decisions Involved in Human Automation Interaction

Human automation interaction (HAI) systems have thus far failed to live up to expectations mainly because human users do not always interact with the automation appropriately. Trust in automation (TiA) has been considered a central influence on the way a human user interacts with an automation; if TiA is too high there will be overuse, if TiA is too low there will be disuse. However, even though extensive research into TiA has identified specific HAI behaviors, or trust outcomes, a unique mapping between trust states and trust outcomes has yet to be clearly identified. Interaction behaviors have been intensely studied in the domain of HAI and TiA and this has led to a reframing of the issues of problems with HAI in terms of reliance and compliance. We find the behaviorally defined terms reliance and compliance to be useful in their functionality for application in real-world situations. However, we note that once an inappropriate interaction behavior has occurred it is too late to mitigate it. We therefore take a step back and look at the interaction decision that precedes the behavior. We note that the decision neuroscience community has revealed that decisions are fairly stereotyped processes accompanied by measurable psychophysiological correlates. Two literatures were therefore reviewed. TiA literature was extensively reviewed in order to understand the relationship between TiA and trust outcomes, as well as to identify gaps in current knowledge. We note that an interaction decision precedes an interaction behavior and believe that we can leverage knowledge of the psychophysiological correlates of decisions to improve joint system performance. As we believe that understanding the interaction decision will be critical to the eventual mitigation of inappropriate interaction behavior, we reviewed the decision making literature and provide a synopsis of the state of the art understanding of the decision process from a decision neuroscience perspective. We forward hypotheses based on this understanding that could shape a research path toward the ability to mitigate interaction behavior in the real world.

Decoding subjective decisions from orbitofrontal cortex

When making a subjective choice, the brain must compute a value for each option and compare those values to make a decision. The orbitofrontal cortex (OFC) is critically involved in this process, but the neural mechanisms remain obscure, in part due to limitations in our ability to measure and control the internal deliberations that can alter the dynamics of the decision process. Here, we tracked the dynamics by recovering temporally precise neural states from multi-dimensional data in OFC. During individual choices, OFC alternated between states associated with the value of two available options, with dynamics that predicted whether a subject would decide quickly or vacillate between the two alternatives. Ensembles of value-encoding neurons contributed to these states, with individual neurons shifting activity patterns as the network evaluated each option. Thus, the mechanism of subjective decision-making involves the dynamic activation of OFC states associated with each choice alternative.

Orbitofrontal Cortex Value Signals Depend on Fixation Location during Free Viewing

In the natural world, monkeys and humans judge the economic value of numerous competing stimuli by moving their gaze from one object to another, in a rapid series of eye movements. This suggests that the primate brain processes value serially, and that value-coding neurons may be modulated by changes in gaze. To test this hypothesis, we presented monkeys with value-associated visual cues and took the unusual step of allowing unrestricted free viewing while we recorded neurons in the orbitofrontal cortex (OFC). By leveraging natural gaze patterns, we found that a large proportion of OFC cells encode gaze location and, that in some cells, value coding is amplified when subjects fixate near the cue. These findings provide the first cellular-level mechanism for previously documented behavioral effects of gaze on valuation and suggest a major role for gaze in neural mechanisms of valuation and decision-making under ecologically realistic conditions.

Neural Mechanisms Underlying Compensatory and Noncompensatory Strategies in Risky Choice

Individuals may differ systematically in their applied decision strategies, which has critical implications for decision neuroscience but is yet scarcely studied. Our study's main focus was therefore to investigate the neural mechanisms underlying compensatory versus noncompensatory strategies in risky choice. Here, we compared people using a compensatory expected value maximization with people using a simplified noncompensatory loss-minimizing choice strategy. To this end, we used a two-choice paradigm including a set of "simple" items (e.g., simple condition), in which one option was superior on all attributes, and a set of "conflict" items, in which one option was superior on one attribute but inferior on other attributes. A binomial mixture analysis of the decisions elicited by these items differentiated between decision-makers using either a compensatory or a noncompensatory strategy. Behavioral differences were particularly pronounced in the conflict condition, and these were paralleled by neural results. That is, we expected compensatory decision-makers to use an integrated value comparison during choice in the conflict condition. Accordingly, the compensatory group tracked the difference in expected value between choice options reflected in neural activation in the parietal cortex. Furthermore, we expected noncompensatory, compared with compensatory, decision-makers to experience increased conflict when attributes provided conflicting information. Accordingly, the noncompensatory group showed greater dorsomedial PFC activation only in the conflict condition. These pronounced behavioral and neural differences indicate the need for decision neuroscience to account for individual differences in risky choice strategies and to broaden its scope to noncompensatory risky choice strategies.

Over a Decade of Neuroeconomics: What Have We Learned?

At its inception, neuroeconomics promised to revolutionize economics. That promise has not yet been realized, and neuroeconomics has seen limited penetration into mainstream economics. Nevertheless, it would be a mistake to declare that neuroeconomics has failed. Quite to the contrary, the yearly rate of neuroeconomics papers has roughly doubled since 2005. While the number of direct applications to economics remains limited, due to the infancy of the field, we have learned an amazing amount about how the brain makes decisions. In this article, we review some of the major topics that have emerged in neuroeconomics and highlight findings that we believe will form the basis for future applications to economics. When possible, we focus on existing applications to economics and future directions for that research.

Graphs versus numbers: How information format affects risk aversion in gambling

In lottery gambling, the common phenomenon of risk aversion shows up as preference of the option with the higher win probability, even if a riskier alternative offers a greater expected value. Because riskier choices would optimize profitability in such cases, the present study investigates the visual format, with which lotteries are conveyed, as potential instrument to modulate risk attitudes. Previous research has shown that enhanced attention to graphical compared to numerical probabilities can increase risk aversion, but evidence for the reverse effect — reduced risk aversion through a graphical display of outcomes — is sparse. We conducted three experiments, in which participants repeatedly selected one of two lotteries. Probabilities and outcomes were either presented numerically or in a graphical format that consisted of pie charts (Experiment 1) or icon arrays (Experiment 2 and 3). Further, expected values were either higher in the safer or in the riskier lottery, or they did not differ between the options. Despite a marked risk aversion in all experiments, our results show that presenting outcomes as graphs can reduce — albeit not eliminate — risk aversion (Experiment 3). Yet, not all formats prove suitable, and non-intuitive outcome graphs can even enhance risk aversion (Experiment 1). Joint analyses of choice proportions and response times (RTs) further uncovered that risk aversion leads to safe choices particularly in fast decisions. This pattern is expressed under graphical probabilities, whereas graphical outcomes can weaken the rapid dominance of risk aversion and the variability over RTs (Experiment 1 and 2). Together, our findings demonstrate the relevance of information format for risky decisions.

Neural substrates of cognitive biases during probabilistic inference

Decision making often requires simultaneously learning about and combining evidence from various sources of information. However, when making inferences from these sources, humans show systematic biases that are often attributed to heuristics or limitations in cognitive processes. Here we use a combination of experimental and modelling approaches to reveal neural substrates of probabilistic inference and corresponding biases. We find systematic deviations from normative accounts of inference when alternative options are not equally rewarding; subjects' choice behaviour is biased towards the more rewarding option, whereas their inferences about individual cues show the opposite bias. Moreover, inference bias about combinations of cues depends on the number of cues. Using a biophysically plausible model, we link these biases to synaptic plasticity mechanisms modulated by reward expectation and attention. We demonstrate that inference relies on direct estimation of posteriors, not on combination of likelihoods and prior. Our work reveals novel mechanisms underlying cognitive biases and contributions of interactions between reward-dependent learning, decision making and attention to high-level reasoning.

Humans are often biased in estimating the precise influence of probabilistic events on their decisions. Here, Khorsand and colleagues report a behavioural task that produces these biases in inference and describe a biophysically-plausible model that captures these behavioural deviations from optimal decision making.

Different Attentional Patterns for Regret and Disappointment: An Eye-tracking Study

The unfavorable comparison between the obtained and expected outcomes of our choices may elicit disappointment. When the comparison is made with the outcome of alternative actions, emotions like regret can serve as a learning signal. Previous work showed that both anticipated disappointment and regret influence decisions. In addition, experienced regret is associated with higher emotional responses than disappointment. Yet it is not clear whether this amplification is due to additive effects of disappointment and regret when the outcomes of alternative actions are available, or whether it reflects the learning feature of regret signals. In this perspective, we used eye-tracking to measure the visual pattern of information acquisition in a probabilistic lottery task. In the partial feedback condition, only the outcome of the chosen lottery was revealed, while in the complete feedback condition, participants could compare their outcome with that of the non-chosen lottery, giving them the opportunity to experience regret. During the decision phase, visual patterns of information acquisition were consistent with the assessment of anticipated regret, in addition to a clear assessment of lotteries' expected values. During the feedback phase, subjective ratings and eye-tracking results confirmed that participants compared their outcome with the outcome of the non-chosen lottery in the complete feedback condition, particularly after a loss, and ignored the non-realized outcome of the chosen option. Moreover, participants who made more visual saccades consistent with counterfactual comparisons during the feedback period anticipated regret more in their decisions. These results are consistent with the proposed adaptive function of regret.

Diffusion Decision Model: Current Issues and History

There is growing interest in diffusion models to represent the cognitive and neural processes of speeded decision making. Sequential-sampling models like the diffusion model have a long history in psychology. They view decision making as a process of noisy accumulation of evidence from a stimulus. The standard model assumes that evidence accumulates at a constant rate during the second or two it takes to make a decision. This process can be linked to the behaviors of populations of neurons and to theories of optimality. Diffusion models have been used successfully in a range of cognitive tasks and as psychometric tools in clinical research to examine individual differences. In this review, we relate the models to both earlier and more recent research in psychology.

Neural underpinnings of the evidence accumulator

Gradual accumulation of evidence favoring one or another choice is considered a core component of many different types of decisions, and has been the subject of many neurophysiological studies in non-human primates. But its neural circuit mechanisms remain mysterious. Investigating it in rodents has recently become possible, facilitating perturbation experiments to delineate the relevant causal circuit, as well as the application of other tools more readily available in rodents. In addition, advances in stimulus design and analysis have aided studying the relevant neural encoding. In complement to ongoing non-human primate studies, these newly available model systems and tools place the field at an exciting time that suggests that the dynamical circuit mechanisms underlying accumulation of evidence could soon be revealed.

Economic irrationality is optimal during noisy decision making

According to normative theories, reward-maximizing agents should have consistent preferences. Thus, when faced with alternatives A, B, and C, an individual preferring A to B and B to C should prefer A to C. However, it has been widely argued that humans can incur losses by violating this axiom of transitivity, despite strong evolutionary pressure for reward-maximizing choices. Here, adopting a biologically plausible computational framework, we show that intransitive (and thus economically irrational) choices paradoxically improve accuracy (and subsequent economic rewards) when decision formation is corrupted by internal neural noise. Over three experiments, we show that humans accumulate evidence over time using a "selective integration" policy that discards information about alternatives with momentarily lower value. This policy predicts violations of the axiom of transitivity when three equally valued alternatives differ circularly in their number of winning samples. We confirm this prediction in a fourth experiment reporting significant violations of weak stochastic transitivity in human observers. Crucially, we show that relying on selective integration protects choices against "late" noise that otherwise corrupts decision formation beyond the sensory stage. Indeed, we report that individuals with higher late noise relied more strongly on selective integration. These findings suggest that violations of rational choice theory reflect adaptive computations that have evolved in response to irreducible noise during neural information processing.

The dynamics of deferred decision

Decision makers are often unable to choose between the options that they are offered. In these settings they typically defer their decision, that is, delay the decision to a later point in time or avoid the decision altogether. In this paper, we outline eight behavioral findings regarding the causes and consequences of choice deferral that cognitive theories of decision making should be able to capture. We show that these findings can be accounted for by a deferral-based time limit applied to existing sequential sampling models of preferential choice. Our approach to modeling deferral as a time limit in a sequential sampling model also makes a number of novel predictions regarding the interactions between choice probabilities, deferral probabilities, and decision times, and we confirm these predictions in an experiment. Choice deferral is a key feature of everyday decision making, and our paper illustrates how established theoretical approaches can be used to understand the cognitive underpinnings of this important behavioral phenomenon.

Do Political and Economic Choices Rely on Common Neural Substrates? A Systematic Review of the Emerging Neuropolitics Literature

The methods of cognitive neuroscience are beginning to be applied to the study of political behavior. The neural substrates of value-based decision-making have been extensively examined in economic contexts; this might provide a powerful starting point for understanding political decision-making. Here, we asked to what extent the neuropolitics literature to date has used conceptual frameworks and experimental designs that make contact with the reward-related approaches that have dominated decision neuroscience. We then asked whether the studies of political behavior that can be considered in this light implicate the brain regions that have been associated with subjective value related to "economic" reward. We performed a systematic literature review to identify papers addressing the neural substrates of political behavior and extracted the fMRI studies reporting behavioral measures of subjective value as defined in decision neuroscience studies of reward. A minority of neuropolitics studies met these criteria and relatively few brain activation foci from these studies overlapped with regions where activity has been related to subjective value. These findings show modest influence of reward-focused decision neuroscience on neuropolitics research to date. Whether the neural substrates of subjective value identified in economic choice paradigms generalize to political choice thus remains an open question. We argue that systematically addressing the commonalities and differences in these two classes of value-based choice will be important in developing a more comprehensive model of the brain basis of human decision-making.

Information processing biases in the brain: Implications for decision-making and self-governance

To make behavioral choices that are in line with our goals and our moral beliefs, we need to gather and consider information about our current situation. Most information present in our environment is not relevant to the choices we need or would want to make and thus could interfere with our ability to behave in ways that reflect our underlying values. Certain sources of information could even lead us to make choices we later regret, and thus it would be beneficial to be able to ignore that information. Our ability to exert successful self-governance depends on our ability to attend to sources of information that we deem important to our decision-making processes. We generally assume that, at any moment, we have the ability to choose what we pay attention to. However, recent research indicates that what we pay attention to is influenced by our prior experiences, including reward history and past successes and failures, even when we are not aware of this history. Even momentary distractions can cause us to miss or discount information that should have a greater influence on our decisions given our values. Such biases in attention thus raise questions about the degree to which the choices that we make may be poorly informed and not truly reflect our ability to otherwise exert self-governance.

Modeling one-choice and two-choice driving tasks

An experiment is presented in which subjects were tested on both one-choice and two-choice driving tasks and on non-driving versions of them. Diffusion models for one- and two-choice tasks were successful in extracting model-based measures from the response time and accuracy data. These include measures of the quality of the information from the stimuli that drove the decision process (drift rate in the model), the time taken up by processes outside the decision process and, for the two-choice model, the speed/accuracy decision criteria that subjects set. Drift rates were only marginally different between the driving and non-driving tasks, indicating that nearly the same information was used in the two kinds of tasks. The tasks differed in the time taken up by other processes, reflecting the difference between them in response processing demands. Drift rates were significantly correlated across the two two-choice tasks showing that subjects that performed well on one task also performed well on the other task. Nondecision times were correlated across the two driving tasks, showing common abilities on motor processes across the two tasks. These results show the feasibility of using diffusion modeling to examine decision making in driving and so provide for a theoretical examination of factors that might impair driving, such as extreme aging, distraction, sleep deprivation, and so on.

Active sensing in the categorization of visual patterns

Interpreting visual scenes typically requires us to accumulate information from multiple locations in a scene. Using a novel gaze-contingent paradigm in a visual categorization task, we show that participants' scan paths follow an active sensing strategy that incorporates information already acquired about the scene and knowledge of the statistical structure of patterns. Intriguingly, categorization performance was markedly improved when locations were revealed to participants by an optimal Bayesian active sensor algorithm. By using a combination of a Bayesian ideal observer and the active sensor algorithm, we estimate that a major portion of this apparent suboptimality of fixation locations arises from prior biases, perceptual noise and inaccuracies in eye movements, and the central process of selecting fixation locations is around 70% efficient in our task. Our results suggest that participants select eye movements with the goal of maximizing information about abstract categories that require the integration of information from multiple locations.

DOI:http://dx.doi.org/10.7554/eLife.12215.001

To interact with the world around us, we need to decide how best to direct our eyes and other senses to extract relevant information. When viewing a scene, people fixate on a sequence of locations by making fast eye movements to shift their gaze between locations. Previous studies have shown that these fixations are not random, but are actively chosen so that they depend on both the scene and the task. For example, in order to determine the gender or emotion from a face, we fixate around the eyes or the nose, respectively.

Previous studies have only analyzed whether humans choose the optimal fixation locations in very simple situations, such as searching for a square among a set of circles. Therefore, it is not known how efficient we are at optimizing our rapid eye movements to extract high-level information from visual scenes, such as determining whether an image of fur belongs to a cheetah or a zebra.

Yang, Lengyel and Wolpert developed a mathematical model that determines the amount of information that can be extracted from an image by any set of fixation locations. The model could also work out the next best fixation location that would maximize the amount of information that could be collected. This model shows that humans are about 70% efficient in planning each eye movement. Furthermore, it suggests that the inefficiencies are largely caused by imperfect vision and inaccurate eye movements.

Yang, Lengyel and Wolpert’s findings indicate that we combine information from multiple locations to direct our eye movements so that we can maximize the information we collect from our surroundings. The next challenge is to extend this mathematical model and experimental approach to even more complex visual tasks, such as judging an individual’s intentions, or working out the relationships between people in real-life settings.

DOI:http://dx.doi.org/10.7554/eLife.12215.002

Transcranial Stimulation over Frontopolar Cortex Elucidates the Choice Attributes and Neural Mechanisms Used to Resolve Exploration-Exploitation Trade-Offs

Optimal behavior requires striking a balance between exploiting tried-and-true options or exploring new possibilities. Neuroimaging studies have identified different brain regions in humans where neural activity is correlated with exploratory or exploitative behavior, but it is unclear whether this activity directly implements these choices or simply reflects a byproduct of the behavior. Moreover, it remains unknown whether arbitrating between exploration and exploitation can be influenced with exogenous methods, such as brain stimulation. In our study, we addressed these questions by selectively upregulating and downregulating neuronal excitability with anodal or cathodal transcranial direct current stimulation over right frontopolar cortex during a reward-learning task. This caused participants to make slower, more exploratory or faster, more exploitative decisions, respectively. Bayesian computational modeling revealed that stimulation affected how much participants took both expected and obtained rewards into account when choosing to exploit or explore: Cathodal stimulation resulted in an increased focus on the option expected to yield the highest payout, whereas anodal stimulation led to choices that were less influenced by anticipated payoff magnitudes and were more driven by recent negative reward prediction errors. These findings suggest that exploration is triggered by a neural mechanism that is sensitive to prior less-than-expected choice outcomes and thus pushes people to seek out alternative courses of action. Together, our findings establish a parsimonious neurobiological mechanism that causes exploration and exploitation, and they provide new insights into the choice features used by this mechanism to direct decision-making.

Irrational time allocation in decision-making

Time is an extremely valuable resource but little is known about the efficiency of time allocation in decision-making. Empirical evidence suggests that in many ecologically relevant situations, decision difficulty and the relative reward from making a correct choice, compared to an incorrect one, are inversely linked, implying that it is optimal to use relatively less time for difficult choice problems. This applies, in particular, to value-based choices, in which the relative reward from choosing the higher valued item shrinks as the values of the other options get closer to the best option and are thus more difficult to discriminate. Here, we experimentally show that people behave sub-optimally in such contexts. They do not respond to incentives that favour the allocation of time to choice problems in which the relative reward for choosing the best option is high; instead they spend too much time on problems in which the reward difference between the options is low. We demonstrate this by showing that it is possible to improve subjects' time allocation with a simple intervention that cuts them off when their decisions take too long. Thus, we provide a novel form of evidence that organisms systematically spend their valuable time in an inefficient way, and simultaneously offer a potential solution to the problem.

A new framework for modeling decisions about changing information: The Piecewise Linear Ballistic Accumulator model

In the real world, decision making processes must be able to integrate non-stationary information that changes systematically while the decision is in progress. Although theories of decision making have traditionally been applied to paradigms with stationary information, non-stationary stimuli are now of increasing theoretical interest. We use a random-dot motion paradigm along with cognitive modeling to investigate how the decision process is updated when a stimulus changes. Participants viewed a cloud of moving dots, where the motion switched directions midway through some trials, and were asked to determine the direction of motion. Behavioral results revealed a strong delay effect: after presentation of the initial motion direction there is a substantial time delay before the changed motion information is integrated into the decision process. To further investigate the underlying changes in the decision process, we developed a Piecewise Linear Ballistic Accumulator model (PLBA). The PLBA is efficient to simulate, enabling it to be fit to participant choice and response-time distribution data in a hierarchal modeling framework using a non-parametric approximate Bayesian algorithm. Consistent with behavioral results, PLBA fits confirmed the presence of a long delay between presentation and integration of new stimulus information, but did not support increased response caution in reaction to the change. We also found the decision process was not veridical, as symmetric stimulus change had an asymmetric effect on the rate of evidence accumulation. Thus, the perceptual decision process was slow to react to, and underestimated, new contrary motion information.

Testing necessary regional frontal contributions to value assessment and fixation-based updating

Value-based decisions are biased by the time people spend viewing each option: Options fixated longer are chosen more often, even when previously rated as less appealing. This bias is thought to reflect ‘value updating' as new evidence is accumulated. Prior work has shown that ventromedial prefrontal cortex (PFC) carries a fixation-dependent value comparison signal, while other studies implicate dorsomedial PFC in representing the value of alternative options. Here, we test whether these regions are necessary for fixation-related value updating in 33 people with frontal lobe damage and 27 healthy controls performing a simple choice task. We show that damage to dorsomedial PFC leads to an exaggerated influence of fixations on choice, while damage to ventromedial or lateral PFC has no effect on this bias. These findings suggest a critical role for dorsomedial, and not ventromedial PFC, in mediating the relative influence of current fixations and a priori value on choice.

In value-based decisions, the longer one fixates on an option, the more likely one is to choose it. Here, the authors compare the performance of people with focal frontal lobe damage in a simple choice task and show that damage to the dorsomedial PFC leads to exaggerated fixation-related value updating.

Capturing the temporal evolution of choice across prefrontal cortex

Activity in prefrontal cortex (PFC) has been richly described using economic models of choice. Yet such descriptions fail to capture the dynamics of decision formation. Describing dynamic neural processes has proven challenging due to the problem of indexing the internal state of PFC and its trial-by-trial variation. Using primate neurophysiology and human magnetoencephalography, we here recover a single-trial index of PFC internal states from multiple simultaneously recorded PFC subregions. This index can explain the origins of neural representations of economic variables in PFC. It describes the relationship between neural dynamics and behaviour in both human and monkey PFC, directly bridging between human neuroimaging data and underlying neuronal activity. Moreover, it reveals a functionally dissociable interaction between orbitofrontal cortex, anterior cingulate cortex and dorsolateral PFC in guiding cost-benefit decisions. We cast our observations in terms of a recurrent neural network model of choice, providing formal links to mechanistic dynamical accounts of decision-making.

DOI:http://dx.doi.org/10.7554/eLife.11945.001

In 1848, a railroad worker named Phineas Gage suffered an accident that was to secure him a place in neuroscience lore. While constructing a new railway line, a mistimed explosion propelled an iron bar into the base of his skull, where it passed behind his left eye before exiting through the top of his head. Gage survived the accident, but those who knew him reported significant changes in his personality and behaviour.

Gage’s ability to make decisions was particularly impaired by his injury. Decision-making involves weighing up the costs and benefits associated with alternative courses of action. It entails looking into the future to decide whether an anticipated reward will justify the effort or expense necessary to obtain it. This process is dependent on a region of the brain called the prefrontal cortex, the area that sustained the most damage in Phineas Gage.

While many studies have shown correlations between activity in particular parts of prefrontal cortex and the outcome of decisions, little is known about how this activity evolves over time as a decision is made. To explore this process, Hunt et al. trained macaque monkeys to choose between pairs of images that were associated with specific rewards (quantities of fruit juice) and costs (either amounts of work or fixed delays).

Electrode recordings revealed changes in prefrontal activity that varied over time as the monkeys deliberated over each pair of images, choosing for example between a large reward after a long delay versus a smaller reward immediately. This activity was consistent with a mathematical model of decision-making, which also explains data from brain imaging experiments in humans. This provides an important link between human data and electrode recordings in animals.

However, some of the patterns of activity observed in both macaques and humans appeared to reflect the speed at which decisions were made, rather than the outcome of the decisions themselves. By extracting information about decision speed on each decision from each region, it was shown that communication between regions of prefrontal cortex changes when choices are between two different amounts of work, as opposed to two different delays. Further experiments are needed to explore this phenomenon and to determine how other brain regions interact with the prefrontal cortex to support the decision-making process.

DOI:http://dx.doi.org/10.7554/eLife.11945.002

Eye Movements in Strategic Choice

In risky and other multiattribute choices, the process of choosing is well described by random walk or drift diffusion models in which evidence is accumulated over time to threshold. In strategic choices, level‐k and cognitive hierarchy models have been offered as accounts of the choice process, in which people simulate the choice processes of their opponents or partners. We recorded the eye movements in 2 × 2 symmetric games including dominance‐solvable games like prisoner's dilemma and asymmetric coordination games like stag hunt and hawk–dove. The evidence was most consistent with the accumulation of payoff differences over time: we found longer duration choices with more fixations when payoffs differences were more finely balanced, an emerging bias to gaze more at the payoffs for the action ultimately chosen, and that a simple count of transitions between payoffs—whether or not the comparison is strategically informative—was strongly associated with the final choice. The accumulator models do account for these strategic choice process measures, but the level‐k and cognitive hierarchy models do not. © 2015 The Authors. Journal of Behavioral Decision Making published by John Wiley & Sons Ltd.

A Common Mechanism Underlying Food Choice and Social Decisions

People make numerous decisions every day including perceptual decisions such as walking through a crowd, decisions over primary rewards such as what to eat, and social decisions that require balancing own and others’ benefits. The unifying principles behind choices in various domains are, however, still not well understood. Mathematical models that describe choice behavior in specific contexts have provided important insights into the computations that may underlie decision making in the brain. However, a critical and largely unanswered question is whether these models generalize from one choice context to another. Here we show that a model adapted from the perceptual decision-making domain and estimated on choices over food rewards accurately predicts choices and reaction times in four independent sets of subjects making social decisions. The robustness of the model across domains provides behavioral evidence for a common decision-making process in perceptual, primary reward, and social decision making.

One critical question that concerns all disciplines involved in the study of human decision-making is whether different types of decisions are made in different ways, or whether there exists a common decision mechanism that underlies human choices. If the latter, what are the properties of that mechanism? Here we characterize a dynamical model of decision making that was initially fit to subjects making food choices but was later able to accurately predict choices and reaction times of separate groups of subjects making social decisions. The robustness of the model across different subjects, tasks, and environments supports the idea that the brain uses a consistent process for making decisions.

Endocannabinoid Signaling in Autism

Autism spectrum disorder (ASD) is a complex behavioral condition with onset during early childhood and a lifelong course in the vast majority of cases. To date, no behavioral, genetic, brain imaging, or electrophysiological test can specifically validate a clinical diagnosis of ASD. However, these medical procedures are often implemented in order to screen for syndromic forms of the disorder (i.e., autism comorbid with known medical conditions). In the last 25 years a good deal of information has been accumulated on the main components of the "endocannabinoid (eCB) system", a rather complex ensemble of lipid signals ("endocannabinoids"), their target receptors, purported transporters, and metabolic enzymes. It has been clearly documented that eCB signaling plays a key role in many human health and disease conditions of the central nervous system, thus opening the avenue to the therapeutic exploitation of eCB-oriented drugs for the treatment of psychiatric, neurodegenerative, and neuroinflammatory disorders. Here we present a modern view of the eCB system, and alterations of its main components in human patients and animal models relevant to ASD. This review will thus provide a critical perspective necessary to explore the potential exploitation of distinct elements of eCB system as targets of innovative therapeutics against ASD.

The Rational Adolescent: Strategic Information Processing during Decision Making Revealed by Eye Tracking

Adolescence is often viewed as a time of irrational, risky decision-making - despite adolescents' competence in other cognitive domains. In this study, we examined the strategies used by adolescents (N=30) and young adults (N=47) to resolve complex, multi-outcome economic gambles. Compared to adults, adolescents were more likely to make conservative, loss-minimizing choices consistent with economic models. Eye-tracking data showed that prior to decisions, adolescents acquired more information in a more thorough manner; that is, they engaged in a more analytic processing strategy indicative of trade-offs between decision variables. In contrast, young adults' decisions were more consistent with heuristics that simplified the decision problem, at the expense of analytic precision. Collectively, these results demonstrate a counter-intuitive developmental transition in economic decision making: adolescents' decisions are more consistent with rational-choice models, while young adults more readily engage task-appropriate heuristics.

A Neurocomputational Model of Altruistic Choice and Its Implications

We propose a neurocomputational model of altruistic choice and test it using behavioral and fMRI data from a task in which subjects make choices between real monetary prizes for themselves and another. We show that a multi-attribute drift-diffusion model, in which choice results from accumulation of a relative value signal that linearly weights payoffs for self and other, captures key patterns of choice, reaction time, and neural response in ventral striatum, temporoparietal junction, and ventromedial prefrontal cortex. The model generates several novel insights into the nature of altruism. It explains when and why generous choices are slower or faster than selfish choices, and why they produce greater response in TPJ and vmPFC, without invoking competition between automatic and deliberative processes or reward value for generosity. It also predicts that when one's own payoffs are valued more than others', some generous acts may reflect mistakes rather than genuinely pro-social preferences.

Sequential Sampling Models in Cognitive Neuroscience: Advantages, Applications, and Extensions

Sequential sampling models assume that people make speeded decisions by gradually accumulating noisy information until a threshold of evidence is reached. In cognitive science, one such model--the diffusion decision model--is now regularly used to decompose task performance into underlying processes such as the quality of information processing, response caution, and a priori bias. In the cognitive neurosciences, the diffusion decision model has recently been adopted as a quantitative tool to study the neural basis of decision making under time pressure. We present a selective overview of several recent applications and extensions of the diffusion decision model in the cognitive neurosciences.

Evidence integration in model-based tree search

Research on the dynamics of reward-based, goal-directed decision making has largely focused on simple choice, where participants decide among a set of unitary, mutually exclusive options. Recent work suggests that the deliberation process underlying simple choice can be understood in terms of evidence integration: Noisy evidence in favor of each option accrues over time, until the evidence in favor of one option is significantly greater than the rest. However, real-life decisions often involve not one, but several steps of action, requiring a consideration of cumulative rewards and a sensitivity to recursive decision structure. We present results from two experiments that leveraged techniques previously applied to simple choice to shed light on the deliberation process underlying multistep choice. We interpret the results from these experiments in terms of a new computational model, which extends the evidence accumulation perspective to multiple steps of action.

Effective Connectivity between Hippocampus and Ventromedial Prefrontal Cortex Controls Preferential Choices from Memory

Although many preferential choices in everyday life require remembering relevant information, the interplay of neural systems mediating decisions and memory has rarely been studied. We addressed this question by combining a task, in which choice options had to be retrieved from memory, with cognitive modeling and fMRI. We found that memory-guided decisions are captured by established process models of choice (sequential sampling models) but constrained by forgetting. People are biased toward remembered options and reject them only if they are very unattractive. Using a Bayesian modeling approach, we determined the posterior probability that options were remembered given the observed choices. This probability correlated with hippocampal activation during encoding. During decision making, the bias toward remembered options was linked to increased connectivity between hippocampus and ventromedial prefrontal cortex. Our results provide insights into the dependency of decisions on memory constraints and show that memory-related activation can be inferred from decisions.

Distinct dynamics of ramping activity in the frontal cortex and caudate nucleus in monkeys

The prefronto-striatal network is involved in many cognitive functions, including perceptual decision making and reward-modulated behaviors. For well-trained subjects, neural responses frequently show similar patterns in the prefrontal cortex and striatum, making it difficult to tease apart distinct regional contributions. Here I show that, despite similar mean firing rate patterns, prefrontal and striatal responses differ in other temporal dynamics for both perceptual and reward-based tasks. Compared with simulation results, the temporal dynamics of prefrontal activity are consistent with an accumulation of sensory evidence used to solve a perceptual task but not with an accumulation of reward context-related information used for the development of a reward bias. In contrast, the dynamics of striatal activity is consistent with an accumulation of reward context-related information and with an accumulation of sensory evidence during early stimulus viewing. These results suggest that prefrontal and striatal neurons may have specialized functions for different tasks even with similar average activity.

Rethinking fast and slow based on a critique of reaction-time reverse inference

Do people intuitively favour certain actions over others? In some dual-process research, reaction-time (RT) data have been used to infer that certain choices are intuitive. However, the use of behavioural or biological measures to infer mental function, popularly known as ‘reverse inference', is problematic because it does not take into account other sources of variability in the data, such as discriminability of the choice options. Here we use two example data sets obtained from value-based choice experiments to demonstrate that, after controlling for discriminability (that is, strength-of-preference), there is no evidence that one type of choice is systematically faster than the other. Moreover, using specific variations of a prominent value-based choice experiment, we are able to predictably replicate, eliminate or reverse previously reported correlations between RT and selfishness. Thus, our findings shed crucial light on the use of RT in inferring mental processes and strongly caution against using RT differences as evidence favouring dual-process accounts.

In cognitive neuroscience, it is common practice to use reaction time data to infer whether decisions are intuitive or deliberate. Here the authors demonstrate that they can replicate, eliminate and reverse previously reported correlations between selfishness and reaction time.

Neuronal Reward and Decision Signals: From Theories to Data

Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neuronal populations. The approach and choice functions involve subjective value, which is objectively assessed by behavioral choices eliciting internal, subjective reward preferences. Utility is the formal mathematical characterization of subjective value and a prime decision variable in economic choice theory. It is coded as utility prediction error by phasic dopamine responses. Utility can incorporate various influences, including risk, delay, effort, and social interaction. Appropriate for formal decision mechanisms, rewards are coded as object value, action value, difference value, and chosen value by specific neurons. Although all reward, reinforcement, and decision variables are theoretical constructs, their neuronal signals constitute measurable physical implementations and as such confirm the validity of these concepts. The neuronal reward signals provide guidance for behavior while constraining the free will to act.

Search, Memory, and Choice Error: An Experiment

Multiple attribute search is a central feature of economic life: we consider much more than price when purchasing a home, and more than wage when choosing a job. An experiment is conducted in order to explore the effects of cognitive limitations on choice in these rich settings, in accordance with the predictions of a new model of search memory load. In each task, subjects are made to search the same information in one of two orders, which differ in predicted memory load. Despite standard models of choice treating such variations in order of acquisition as irrelevant, lower predicted memory load search orders are found to lead to substantially fewer choice errors. An implication of the result for search behavior, more generally, is that in order to reduce memory load (thus choice error) a limited memory searcher ought to deviate from the search path of an unlimited memory searcher in predictable ways-a mechanism that can explain the systematic deviations from optimal sequential search that have recently been discovered in peoples' behavior. Further, as cognitive load is induced endogenously (within the task), and found to affect choice behavior, this result contributes to the cognitive load literature (in which load is induced exogenously), as well as the cognitive ability literature (in which cognitive ability is measured in a separate task). In addition, while the information overload literature has focused on the detrimental effects of the quantity of information on choice, this result suggests that, holding quantity constant, the order that information is observed in is an essential determinant of choice failure.

A neuro-computational model of economic decisions

Neuronal recordings and lesion studies indicate that key aspects of economic decisions take place in the orbitofrontal cortex (OFC). Previous work identified in this area three groups of neurons encoding the offer value, the chosen value, and the identity of the chosen good. An important and open question is whether and how decisions could emerge from a neural circuit formed by these three populations. Here we adapted a biophysically realistic neural network previously proposed for perceptual decisions (Wang XJ. Neuron 36: 955-968, 2002; Wong KF, Wang XJ. J Neurosci 26: 1314-1328, 2006). The domain of economic decisions is significantly broader than that for which the model was originally designed, yet the model performed remarkably well. The input and output nodes of the network were naturally mapped onto two groups of cells in OFC. Surprisingly, the activity of interneurons in the network closely resembled that of the third group of cells, namely, chosen value cells. The model reproduced several phenomena related to the neuronal origins of choice variability. It also generated testable predictions on the excitatory/inhibitory nature of different neuronal populations and on their connectivity. Some aspects of the empirical data were not reproduced, but simple extensions of the model could overcome these limitations. These results render a biologically credible model for the neuronal mechanisms of economic decisions. They demonstrate that choices could emerge from the activity of cells in the OFC, suggesting that chosen value cells directly participate in the decision process. Importantly, Wang's model provides a platform to investigate the implications of neuroscience results for economic theory.

Cognitive effort: A neuroeconomic approach

Cognitive effort has been implicated in numerous theories regarding normal and aberrant behavior and the physiological response to engagement with demanding tasks. Yet, despite broad interest, no unifying, operational definition of cognitive effort itself has been proposed. Here, we argue that the most intuitive and epistemologically valuable treatment is in terms of effort-based decision-making, and advocate a neuroeconomics-focused research strategy. We first outline psychological and neuroscientific theories of cognitive effort. Then we describe the benefits of a neuroeconomic research strategy, highlighting how it affords greater inferential traction than do traditional markers of cognitive effort, including self-reports and physiologic markers of autonomic arousal. Finally, we sketch a future series of studies that can leverage the full potential of the neuroeconomic approach toward understanding the cognitive and neural mechanisms that give rise to phenomenal, subjective cognitive effort.

Reappraising abstract paintings after exposure to background information

Can knowledge help viewers when they appreciate an artwork? Experts’ judgments of the aesthetic value of a painting often differ from the estimates of naïve viewers, and this phenomenon is especially pronounced in the aesthetic judgment of abstract paintings. We compared the changes in aesthetic judgments of naïve viewers while they were progressively exposed to five pieces of background information. The participants were asked to report their aesthetic judgments of a given painting after each piece of information was presented. We found that commentaries by the artist and a critic significantly increased the subjective aesthetic ratings. Does knowledge enable experts to attend to the visual features in a painting and to link it to the evaluative conventions, thus potentially causing different aesthetic judgments? To investigate whether a specific pattern of attention is essential for the knowledge-based appreciation, we tracked the eye movements of subjects while viewing a painting with a commentary by the artist and with a commentary by a critic. We observed that critics’ commentaries directed the viewers’ attention to the visual components that were highly relevant to the presented commentary. However, attention to specific features of a painting was not necessary for increasing the subjective aesthetic judgment when the artists’ commentary was presented. Our results suggest that at least two different cognitive mechanisms may be involved in knowledge- guided aesthetic judgments while viewers reappraise a painting.

Utility-free heuristic models of two-option choice can mimic predictions of utility-stage models under many conditions

Economists often model choices as if decision-makers assign each option a scalar value variable, known as utility, and then select the option with the highest utility. It remains unclear whether as-if utility models describe real mental and neural steps in choice. Although choices alone cannot prove the existence of a utility stage, utility transformations are often taken to provide the most parsimonious or psychologically plausible explanation for choice data. Here, we show that it is possible to mathematically transform a large set of common utility-stage two-option choice models (specifically ones in which dimensions are can be decomposed into additive functions) into a heuristic model (specifically, a dimensional prioritization heuristic) that has no utility computation stage. We then show that under a range of plausible assumptions, both classes of model predict similar neural responses. These results highlight the difficulties in using neuroeconomic data to infer the existence of a value stage in choice.

Eyetracking of social preference choices reveals normal but faster processing in autism

People with Autism Spectrum Disorder (ASD) have been reported to show atypical attention and evaluative processing, in particular for social stimuli such as faces. The usual measure in these studies is an explicit, subjective judgment, which is the culmination of complex-temporally extended processes that are not typically dissected in detail. Here we addressed a neglected aspect of social decision-making in order to gain further insight into the underlying mechanisms: the temporal evolution of the choice. We investigated this issue by quantifying the alternating patterns of gaze onto faces, as well as nonsocial stimuli, while subjects had to decide which of the two stimuli they preferred. Surprisingly, the temporal profile of fixations relating to choice (the so-called "gaze cascade") was entirely normal in ASD, as were the eventual preference choices. Despite these similarities, we found two key abnormalities: people with ASD made choices more rapidly than did control subjects across the board, and their reaction times for social preference judgments were insensitive to choice difficulty. We suggest that ASD features an altered decision-making process when basing choice on social preferences. One hypothesis motivated by these data is that a choice criterion is reached in ASD regardless of the discriminability of the options.

Cognitive mechanisms underlying instructed choice exploration of small city maps

We investigated the cognitive mechanisms underlying the exploration and decision-making in realistic and novel environments. Twelve human subjects were shown small circular U.S. city maps with two locations highlighted on the circumference, as possible choices for a post office (“targets”). At the beginning of a trial, subjects fixated a spot at the center of the map and ultimately chose one of the two locations. A space syntax analysis of the map paths (from the center to each target) revealed that the chosen location was associated with the less convoluted path, as if subjects navigated mentally the paths in an “ant's way,” i.e., by staying within street boundaries, and ultimately choosing the target that could be reached from the center in the shortest way, and the fewest turns and intersections. The subjects' strategy for map exploration and decision making was investigated by monitoring eye position during the task. This revealed a restricted exploration of the map delimited by the location of the two alternative options and the center of the map. Specifically, subjects explored the areas around the two target options by repeatedly looking at them before deciding which one to choose, presumably implementing an evaluation and decision-making process. The ultimate selection of a specific target was significantly associated with the time spent exploring the area around that target. Finally, an analysis of the sequence of eye fixations revealed that subjects tended to look systematically toward the target ultimately chosen even from the beginning of the trial. This finding indicates an early cognitive selection bias for the ensuing decision process.