Visual fixations and the computation and comparison of value in simple choice

Probabilistic brains: knowns and unknowns

There is strong behavioral and physiological evidence that the brain both represents probability distributions and performs probabilistic inference. Computational neuroscientists have started to shed light on how these probabilistic representations and computations might be implemented in neural circuits. One particularly appealing aspect of these theories is their generality: they can be used to model a wide range of tasks, from sensory processing to high-level cognition. To date, however, these theories have only been applied to very simple tasks. Here we discuss the challenges that will emerge as researchers start focusing their efforts on real-life computations, with a focus on probabilistic learning, structural learning and approximate inference.

Looking time predicts choice but not aesthetic value

Although visual fixations are commonly used to index stimulus-driven or internally-determined preference, recent evidence suggests that visual fixations can also be a source of decisional bias that moves selection toward the fixated object. These contrasting results raise the question of whether visual fixations always index comparative processes during choice-based tasks, or whether they might better reflect internal preferences when the decision does not carry any economic or corporeal consequences. In two experiments, participants chose which of two objects were more aesthetically pleasing (Exp.1) or appeared more organic (Exp.2), and provided independent aesthetic ratings of the stimuli. Our results demonstrated that fixation parameters were a better index of choice in both decisional domains than of aesthetic preference. The data support models in which visual fixations are specifically related to the evolution of decision processes even when the decision has no tangible consequences.

The contributions of value-based decision-making and attentional bias to alcohol-seeking following devaluation

AIMS

To investigate the mediating role of attentional bias for alcohol cues on alcohol-seeking following devaluation of alcohol.

DESIGN

Between subject.

SETTING

Eye-tracking laboratory at the University of Liverpool.

PARTICIPANTS

Student social drinkers (n = 64).

MEASUREMENTS

An operant choice task in which participants chose between simultaneously presented alcohol and non-alcohol drink rewards, while attentional bias for alcohol and non-alcohol drink cues was inferred from eye movements. Participants then consumed 30 mL of an alcoholic beverage, which was either presented alone (no devaluation: n = 32) or had been adulterated to taste unpleasant (devaluation: n = 32). Choice and attentional bias for the alcohol and non-alcohol drink pictures were then measured again.

FINDINGS

Alcohol devaluation reduced behavioural choice for alcohol (F = 32.64, P < 0.001) and attentional bias for the alcohol pictures indexed by dwell time (F = 22.68, P < 0.001), initial fixation (F = 7.08, P = 0.01) and final fixation (F = 22.44, P < 0.001). Mediation analysis revealed that attentional bias partially mediated the effect of devaluation on alcohol choice; however, the proportion of the variance accounted for by attentional bias is low to moderate (~30%).

CONCLUSIONS

Among student social drinkers, attentional bias is only a partial mediator of alcohol choice following devaluation of alcohol. Value-based decision-making may be a more important determinant of drinking behaviour among student social drinkers than attentional bias.

Choice to view cocaine images predicts concurrent and prospective drug use in cocaine addiction

BACKGROUND

Identifying variables that predict drug use in treatment-seeking drug addicted individuals is a crucial research and therapeutic goal. This study tested the hypothesis that choice to view cocaine images is associated with concurrent and prospective drug use in cocaine addiction.

METHODS

To establish choice-concurrent drug use associations, 71 cocaine addicted subjects (43 current users and 28 treatment seekers) provided data on (A) choice to view cocaine images and affectively pleasant, unpleasant, and neutral images [collected under explicit contingencies (when choice was made between two fully visible side-by-side images) and under more probabilistic contingencies (when choice was made between pictures hidden under flipped-over cards)]; and (B) past-month cocaine and other drug use. To establish choice-prospective drug use associations, 20 of these treatment-seeking subjects were followed over the next 6 months.

RESULTS

Baseline cocaine-related picture choice as measured by both tasks positively correlated with subjects' concurrent cocaine and other drug use as driven by the actively-using subjects. In a subsequent multiple regression analysis, choice to view cocaine images as compared with affectively pleasant images (under probabilistic contingencies) was the only predictor that continued to be significantly associated with drug use. Importantly, this same baseline cocaine>pleasant probabilistic choice also predicted the number of days drugs were used (cocaine, alcohol, and marijuana) over the next 6 months.

CONCLUSIONS

Simulated cocaine choice - especially when probabilistic and when compared with other positive reinforcers - may provide a valid laboratory marker of current and future drug use in cocaine addiction.

Influence of Branding on Preference-Based Decision Making

Branding has become one of the most important determinants of consumer choices. Intriguingly, the psychological mechanisms of how branding influences decision making remain elusive. In the research reported here, we used a preference-based decision-making task and computational modeling to identify which internal components of processing are affected by branding. We found that a process of noisy temporal integration of subjective value information can model preference-based choices reliably and that branding biases are explained by changes in the rate of the integration process itself. This result suggests that branding information and subjective preference are integrated into a single source of evidence in the decision-making process, thereby altering choice behavior.

Predicting targets of human reaching motions using different sensing technologies

Rapid recognition of voluntary motions is crucial in human-computer interaction, but few studies compare the predictive abilities of different sensing technologies. This paper thus compares performances of different technologies when predicting targets of human reaching motions: electroencephalography (EEG), electrooculography, camera-based eye tracking, electromyography (EMG), hand position, and the user's preferences. Supervised machine learning is used to make predictions at different points in time (before and during limb motion) with each individual sensing modality. Different modalities are then combined using an algorithm that takes into account the different times at which modalities provide useful information. Results show that EEG can make predictions before limb motion onset, but requires subject-specific training and exhibits decreased performance as the number of possible targets increases. EMG and hand position give high accuracy, but only once the motion has begun. Eye tracking is robust and exhibits high accuracy at the very onset of limb motion. Several advantages of combining different modalities are also shown, including advantages of combining measurements with contextual data. Finally, some recommendations are given for sensing modalities with regard to different criteria and applications. The information could aid human-computer interaction designers in selecting and evaluating appropriate equipment for their applications.

Rats and humans can optimally accumulate evidence for decision-making

The gradual and noisy accumulation of evidence is a fundamental component of decision-making, with noise playing a key role as the source of variability and errors. However, the origins of this noise have never been determined. We developed decision-making tasks in which sensory evidence is delivered in randomly timed pulses, and analyzed the resulting data with models that use the richly detailed information of each trial's pulse timing to distinguish between different decision-making mechanisms. This analysis allowed measurement of the magnitude of noise in the accumulator's memory, separately from noise associated with incoming sensory evidence. In our tasks, the accumulator's memory was noiseless, for both rats and humans. In contrast, the addition of new sensory evidence was the primary source of variability. We suggest our task and modeling approach as a powerful method for revealing internal properties of decision-making processes.

Neural evidence for adaptive strategy selection in value-based decision-making

In everyday life, humans often encounter complex environments in which multiple sources of information can influence their decisions. We propose that in such situations, people select and apply different strategies representing different cognitive models of the decision problem. Learning advances by evaluating the success of using a strategy and eventually by switching between strategies. To test our strategy selection model, we investigated how humans solve a dynamic learning task with complex auditory and visual information, and assessed the underlying neural mechanisms with functional magnetic resonance imaging. Using the model, we were able to capture participants' choices and to successfully attribute expected values and reward prediction errors to activations in the dopaminoceptive system (e.g., ventral striatum [VS]) as well as decision conflict to signals in the anterior cingulate cortex. The model outperformed an alternative approach that did not update decision strategies, but the relevance of information itself. Activation of sensory areas depended on whether the selected strategy made use of the respective source of information. Selection of a strategy also determined how value-related information influenced effective connectivity between sensory systems and the VS. Our results suggest that humans can structure their search for and use of relevant information by adaptively selecting between decision strategies.

Rational temporal predictions can underlie apparent failures to delay gratification

An important category of seemingly maladaptive decisions involves failure to postpone gratification. A person pursuing a desirable long-run outcome may abandon it in favor of a short-run alternative that has been available all along. Here we present a theoretical framework in which this seemingly irrational behavior emerges from stable preferences and veridical judgments. Our account recognizes that decision makers generally face uncertainty regarding the time at which future outcomes will materialize. When timing is uncertain, the value of persistence depends crucially on the nature of a decision maker's prior temporal beliefs. Certain forms of temporal beliefs imply that a delay's predicted remaining length increases as a function of time already waited. In this type of situation, the rational, utility-maximizing strategy is to persist for a limited amount of time and then give up. We show empirically that people's explicit predictions of remaining delay lengths indeed increase as a function of elapsed time in several relevant domains, implying that temporal judgments offer a rational basis for limiting persistence. We then develop our framework into a simple working model and show how it accounts for individual differences in a laboratory task (the well-known "marshmallow test"). We conclude that delay-of-gratification failure, generally viewed as a manifestation of limited self-control capacity, can instead arise as an adaptive response to the perceived statistics of one's environment.

Decision value computation in DLPFC and VMPFC adjusts to the available decision time

It is increasingly clear that simple decisions are made by computing decision values for the options under consideration, and then comparing these values to make a choice. Computational models of this process suggest that it involves the accumulation of information over time, but little is known about the temporal course of valuation in the brain. To examine this, we manipulated the available decision time and observed the consequences in the brain and behavioral correlates of choice. Participants were scanned with functional magnetic resonance imaging while they chose to eat or not eat basic food items, in two conditions differing in the amount of time provided for choice. After identifying valuation-related regions with unbiased whole-brain general linear models, we analyzed two regions of interest: ventromedial prefrontal cortex (VMPFC) and dorsolateral prefrontal cortex (DLPFC). Finite impulse response models of the upsampled estimated neural activity from those regions allowed us to examine the onset, duration and termination of decision value signals, and to compare across regions. We found evidence for the immediate onset of value computation in both regions, but an extended duration with longer decision time. However, this was not accompanied by behavioral changes in either the accuracy or determinants of choice. Finally, there was modest evidence that DLPFC computation correlated with, but lagged behind, VMPFC computation, suggesting the sharing of information across these regions. These findings have important implications for models of decision value computation and choice.

Activity in inferior parietal and medial prefrontal cortex signals the accumulation of evidence in a probability learning task

In an uncertain environment, probabilities are key to predicting future events and making adaptive choices. However, little is known about how humans learn such probabilities and where and how they are encoded in the brain, especially when they concern more than two outcomes. During functional magnetic resonance imaging (fMRI), young adults learned the probabilities of uncertain stimuli through repetitive sampling. Stimuli represented payoffs and participants had to predict their occurrence to maximize their earnings. Choices indicated loss and risk aversion but unbiased estimation of probabilities. BOLD response in medial prefrontal cortex and angular gyri increased linearly with the probability of the currently observed stimulus, untainted by its value. Connectivity analyses during rest and task revealed that these regions belonged to the default mode network. The activation of past outcomes in memory is evoked as a possible mechanism to explain the engagement of the default mode network in probability learning. A BOLD response relating to value was detected only at decision time, mainly in striatum. It is concluded that activity in inferior parietal and medial prefrontal cortex reflects the amount of evidence accumulated in favor of competing and uncertain outcomes.

Neurocomputational account of how the human brain decides when to have a break

No pain, no gain: cost-benefit trade-off has been formalized in classical decision theory to account for how we choose whether to engage effort. However, how the brain decides when to have breaks in the course of effort production remains poorly understood. We propose that decisions to cease and resume work are triggered by a cost evidence accumulation signal reaching upper and lower bounds, respectively. We developed a task in which participants are free to exert a physical effort knowing that their payoff would be proportional to their effort duration. Functional MRI and magnetoencephalography recordings conjointly revealed that the theoretical cost evidence accumulation signal was expressed in proprioceptive regions (bilateral posterior insula). Furthermore, the slopes and bounds of the accumulation process were adapted to the difficulty of the task and the money at stake. Cost evidence accumulation might therefore provide a dynamical mechanistic account of how the human brain maximizes benefits while preventing exhaustion.

Choice theories: What are they good for?

Simonson et al. present an ambitious sketch of an integrative theory of context. Provoked by this thoughtful proposal, I discuss what is the function of theories of choice in the coming decades. Traditionally, choice models and theory have attempted to predict choices as a function of the attributes of options. I argue that to be truly useful, they need to generate specific and quantitative predictions of the effect of the choice environment upon choice probability. To do this, we need to focus on rigorously modeling and measuring the underlying processes causing these effects, and use the Simonson et al. proposal to provide some examples. I also present some examples from research in decision-making and decision neuroscience, and argue that models that fail, and fail spectacularly are particularly useful. I close with a challenge: How would consumer researcher aid the design of real world choice environments such as the health exchanges under the Patient Protection and Affordable Care Act?

Effectiveness of different sensing modalities in predicting targets of reaching movements

Human motion recognition is essential for many biomedical applications, but few studies compare the abilities of multiple sensing modalities. This paper thus evaluates the effectiveness of different modalities when predicting targets of human reaching movements. Electroencephalography, electrooculography, camera-based eye tracking, electromyography, hand tracking and the user's preferences are used to make predictions at different points in time. Prediction accuracies are calculated based on data from 10 subjects in within-subject crossvalidation. Results show that electroencephalography can make predictions before limb motion onset, but its accuracy decreases as the number of potential targets increases. Electromyography and hand tracking give high accuracy, but only after motion onset. Eye tracking is robust and gives high accuracy at limb motion onset. Combining multiple modalities can increase accuracy, though not always. While many studies have evaluated individual sensing modalities, this study provides quantitative data on many modalities at different points of time in a single setting. The information could help biomedical engineers choose the most appropriate equipment for a particular application.

Contextual and social influences on valuation and choice

To survive in our complex environment, we have to adapt to changing contexts. Prior research that investigated how contextual changes are processed in the human brain has demonstrated important modulatory influences on multiple cognitive processes underlying decision-making, including perceptual judgments, working memory, as well as cognitive and attentional control. However, in everyday life, the importance of context is even more obvious during economic and social interactions, which often have implicit rule sets that need to be recognized by a decision-maker. Here, we review recent evidence from an increasing number of studies in the fields of Neuroeconomics and Social Neuroscience that investigate the neurobiological basis of contextual effects on valuation and social choice. Contrary to the assumptions of rational choice theory, multiple contextual factors, such as the availability of alternative choice options, shifts in reference point, and social context, have been shown to modulate behavior, as well as signals in task-relevant neural networks. A consistent picture that emerges from neurobiological results is that valuation-related activity in striatum and ventromedial prefrontal cortex is highly context dependent during both social and nonsocial choice. Alternative approaches to model and explain choice behavior, such as comparison-based choice models, as well as implications for future research are discussed.

Attention, learning, and the value of information

Despite many studies on selective attention, fundamental questions remain about its nature and neural mechanisms. Here I draw from the animal and machine learning fields that describe attention as a mechanism for active learning and uncertainty reduction and explore the implications of this view for understanding visual attention and eye movement control. I propose that a closer integration of these different views has the potential greatly to expand our understanding of oculomotor control and our ability to use this system as a window into high level but poorly understood cognitive functions, including the capacity for curiosity and exploration and for inferring internal models of the external world.

Selection and inhibition mechanisms for human voluntary action decisions

One can choose between action alternatives that have no apparent difference in their outcomes. Such voluntary action decisions are associated with widespread frontal-parietal activation, and a tendency to inhibit the repetition of a previous action. However, the mechanism of initiating voluntary actions and the functions of different brain regions during this process remains largely unknown. Here, we combine computational modeling and functional magnetic resonance imaging to test the selection and inhibition mechanisms that mediate trial-to-trial voluntary action decisions. We fitted an optimized accumulator model to behavioral responses in a finger-tapping task in which participants were instructed to make chosen actions or specified actions. Model parameters derived from each individual were then applied to estimate the expected accumulated metabolic activity (EAA) engaged in every single trial. The EAA was associated with blood oxygenation level-dependent responses in a decision work that was maximal in the supplementary motor area and the caudal anterior cingulate cortex, consistent with a competitive accumulation-to-threshold mechanism for action decision by these regions. Furthermore, specific inhibition of the previous action's accumulator was related to the suppression of response repetition. This action-specific inhibition correlated with the activity of the right inferior frontal gyrus, when the option to repeat existed. Our findings suggest that human voluntary action decisions are mediated by complementary processes of intentional selection and inhibition.

The dynamics of decision making in risky choice: an eye-tracking analysis

In the last years, research on risky choice has moved beyond analyzing choices only. Models have been suggested that aim to describe the underlying cognitive processes and some studies have tested process predictions of these models. Prominent approaches are evidence accumulation models such as decision field theory (DFT), simple serial heuristic models such as the adaptive toolbox, and connectionist approaches such as the parallel constraint satisfaction (PCS) model. In two studies involving measures of attention and pupil dilation, we investigate hypotheses derived from these models in choices between two gambles with two outcomes each. We show that attention to an outcome of a gamble increases with its probability and its value and that attention shifts toward the subsequently favored gamble after about two thirds of the decision process, indicating a gaze-cascade effect. Information search occurs mostly within-gambles, and the direction of search does not change over the course of decision making. Pupil dilation, which reflects both cognitive effort and arousal, increases during the decision process and increases with mean expected value. Overall, the results support aspects of automatic integration models for risky choice such as DFT and PCS, but in their current specification none of them can account for the full pattern of results.

Cognitive regulation during decision making shifts behavioral control between ventromedial and dorsolateral prefrontal value systems

Cognitive regulation is often used to influence behavioral outcomes. However, the computational and neurobiological mechanisms by which it affects behavior remain unknown. We studied this issue using an fMRI task in which human participants used cognitive regulation to upregulate and downregulate their cravings for foods at the time of choice. We found that activity in both ventromedial prefrontal cortex (vmPFC) and dorsolateral prefrontal cortex (dlPFC) correlated with value. We also found evidence that two distinct regulatory mechanisms were at work: value modulation, which operates by changing the values assigned to foods in vmPFC and dlPFC at the time of choice, and behavioral control modulation, which operates by changing the relative influence of the vmPFC and dlPFC value signals on the action selection process used to make choices. In particular, during downregulation, activation decreased in the value-sensitive region of dlPFC (indicating value modulation) but not in vmPFC, and the relative contribution of the two value signals to behavior shifted toward the dlPFC (indicating behavioral control modulation). The opposite pattern was observed during upregulation: activation increased in vmPFC but not dlPFC, and the relative contribution to behavior shifted toward the vmPFC. Finally, ventrolateral PFC and posterior parietal cortex were more active during both upregulation and downregulation, and were functionally connected with vmPFC and dlPFC during cognitive regulation, which suggests that they help to implement the changes to the decision-making circuitry generated by cognitive regulation.

The road not taken: neural correlates of decision making in orbitofrontal cortex

Empirical research links human orbitofrontal cortex (OFC) to the evaluation of outcomes during decision making and the representation of alternative (better) outcomes after failures. When faced with a difficult decision, rats sometimes pause and turn back-and-forth toward goals, until finally orienting toward the chosen direction. Neural representations of reward in rodent OFC increased immediately following each reorientation, implying a transient representation of the expected outcome following self-initiated decisions. Upon reaching reward locations and finding no reward (having made an error), OFC representations of reward decreased locally indicating a disappointment signal that then switched to represent the unrewarded, non-local, would-have-been rewarded site. These results illustrate that following a decision to act, neural ensembles in OFC represent reward, and upon the realization of an error, represent the reward that could have been.

Metacognition in human decision-making: confidence and error monitoring

People are capable of robust evaluations of their decisions: they are often aware of their mistakes even without explicit feedback, and report levels of confidence in their decisions that correlate with objective performance. These metacognitive abilities help people to avoid making the same mistakes twice, and to avoid overcommitting time or resources to decisions that are based on unreliable evidence. In this review, we consider progress in characterizing the neural and mechanistic basis of these related aspects of metacognition—confidence judgements and error monitoring—and identify crucial points of convergence between methods and theories in the two fields. This convergence suggests that common principles govern metacognitive judgements of confidence and accuracy; in particular, a shared reliance on post-decisional processing within the systems responsible for the initial decision. However, research in both fields has focused rather narrowly on simple, discrete decisions—reflecting the correspondingly restricted focus of current models of the decision process itself—raising doubts about the degree to which discovered principles will scale up to explain metacognitive evaluation of real-world decisions and actions that are fluid, temporally extended, and embedded in the broader context of evolving behavioural goals.

Interactions between deliberation and delay-discounting in rats

When faced with decisions, rats sometimes pause and look back and forth between possible alternatives, a phenomenon termed vicarious trial and error (VTE). When it was first observed in the 1930s, VTE was theorized to be a mechanism for exploration. Later theories suggested that VTE aided the resolution of sensory or neuroeconomic conflict. In contrast, recent neurophysiological data suggest that VTE reflects a dynamic search and evaluation process. These theories make unique predictions about the timing of VTE on behavioral tasks. We tested these theories of VTE on a T-maze with return rails, where rats were given a choice between a smaller reward available after one delay or a larger reward available after an adjustable delay. Rats showed three clear phases of behavior on this task: investigation, characterized by discovery of task parameters; titration, characterized by iterative adjustment of the delay to a preferred interval; and exploitation, characterized by alternation to hold the delay at the preferred interval. We found that VTE events occurred during adjustment laps more often than during alternation laps. Results were incompatible with theories of VTE as an exploratory behavior, as reflecting sensory conflict, or as a simple neuroeconomic valuation process. Instead, our results were most consistent with VTE as reflecting a search process during deliberative decision making. This pattern of VTE that we observed is reminiscent of current navigational theories proposing a transition from a deliberative to a habitual decision-making mechanism.

Value normalization in decision making: theory and evidence

A sizable body of evidence has shown that the brain computes several types of value-related signals to guide decision making, such as stimulus values, outcome values, and prediction errors. A critical question for understanding decision-making mechanisms is whether these value signals are computed using an absolute or a normalized code. Under an absolute code, the neural response used to represent the value of a given stimulus does not depend on what other values might have been encountered. By contrast, under a normalized code, the neural response associated with a given value depends on its relative position in the distribution of values. This review provides a simple framework for thinking about value normalization, and uses it to evaluate the existing experimental evidence.

Contributions of the striatum to learning, motivation, and performance: an associative account

It has long been recognized that the striatum is composed of distinct functional sub-units that are part of multiple cortico-striatal-thalamic circuits. Contemporary research has focused on the contribution of striatal sub-regions to three main phenomena: learning of associations between stimuli, actions and rewards; selection between competing response alternatives; and motivational modulation of motor behavior. Recent proposals have argued for a functional division of the striatum along these lines, attributing, for example, learning to one region and performance to another. Here, we consider empirical data from human and animal studies, as well as theoretical notions from both the psychological and computational literatures, and conclude that striatal sub-regions instead differ most clearly in terms of the associations being encoded in each region.

The effects of evidence bounds on decision-making: theoretical and empirical developments

Converging findings from behavioral, neurophysiological, and neuroimaging studies suggest an integration-to-boundary mechanism governing decision formation and choice selection. This mechanism is supported by sequential sampling models of choice decisions, which can implement statistically optimal decision strategies for selecting between multiple alternative options on the basis of sensory evidence. This review focuses on recent developments in understanding the evidence boundary, an important component of decision-making raised by experimental findings and models. The article starts by reviewing the neurobiology of perceptual decisions and several influential sequential sampling models, in particular the drift-diffusion model, the Ornstein–Uhlenbeck model and the leaky-competing-accumulator model. In the second part, the article examines how the boundary may affect a model’s dynamics and performance and to what extent it may improve a model’s fits to experimental data. In the third part, the article examines recent findings that support the presence and site of boundaries in the brain. The article considers two questions: (1) whether the boundary is a spontaneous property of neural integrators, or is controlled by dedicated neural circuits; (2) if the boundary is variable, what could be the driving factors behind boundary changes? The review brings together studies using different experimental methods in seeking answers to these questions, highlights psychological and physiological factors that may be associated with the boundary and its changes, and further considers the evidence boundary as a generic mechanism to guide complex behavior.

Preference suppression caused by misattribution of task-irrelevant subliminal motion

It is well known that subjects tend to misattribute task-irrelevant signals, incorporating them into the information on which a decision is made. Such misattribution has been reported to originate only from a social or a cognitive stage of information processing. However, we provide the initial evidence that misattribution also originates at a lower, visuomotor stage. This type of misattribution occurs only when subjects do not notice a visuomotor conflict. Misattribution at a social or a cognitive stage facilitates decision-making if the misattributed information is consistent with the decision and impedes decision-making if the information is in conflict with the decision. However, misattribution originating at a visuomotor stage only impedes decision-making, suggesting a fundamental difference between the mechanisms for the two types of misattribution. Furthermore, misattribution effects that originate in a visuomotor interaction stage also affect subjective preference ratings, suggesting that the misattribution exerts an influence on global brain processing.

The attentional drift-diffusion model extends to simple purchasing decisions

How do we make simple purchasing decisions (e.g., whether or not to buy a product at a given price)? Previous work has shown that the attentional drift-diffusion model (aDDM) can provide accurate quantitative descriptions of the psychometric data for binary and trinary value-based choices, and of how the choice process is guided by visual attention. Here we extend the aDDM to the case of purchasing decisions, and test it using an eye-tracking experiment. We find that the model also provides a reasonably accurate quantitative description of the relationship between choice, reaction time, and visual fixations using parameters that are very similar to those that best fit the previous data. The only critical difference is that the choice biases induced by the fixations are about half as big in purchasing decisions as in binary choices. This suggests that a similar computational process is used to make binary choices, trinary choices, and simple purchasing decisions.

Building Bridges between Perceptual and Economic Decision-Making: Neural and Computational Mechanisms

Investigation into the neural and computational bases of decision-making has proceeded in two parallel but distinct streams. Perceptual decision-making (PDM) is concerned with how observers detect, discriminate, and categorize noisy sensory information. Economic decision-making (EDM) explores how options are selected on the basis of their reinforcement history. Traditionally, the sub-fields of PDM and EDM have employed different paradigms, proposed different mechanistic models, explored different brain regions, disagreed about whether decisions approach optimality. Nevertheless, we argue that there is a common framework for understanding decisions made in both tasks, under which an agent has to combine sensory information (what is the stimulus) with value information (what is it worth). We review computational models of the decision process typically used in PDM, based around the idea that decisions involve a serial integration of evidence, and assess their applicability to decisions between good and gambles. Subsequently, we consider the contribution of three key brain regions - the parietal cortex, the basal ganglia, and the orbitofrontal cortex (OFC) - to perceptual and EDM, with a focus on the mechanisms by which sensory and reward information are integrated during choice. We find that although the parietal cortex is often implicated in the integration of sensory evidence, there is evidence for its role in encoding the expected value of a decision. Similarly, although much research has emphasized the role of the striatum and OFC in value-guided choices, they may play an important role in categorization of perceptual information. In conclusion, we consider how findings from the two fields might be brought together, in order to move toward a general framework for understanding decision-making in humans and other primates.

Valuation and decision-making in frontal cortex: one or many serial or parallel systems?

We evaluate the merits of different conceptualizations of frontal cortex function in value-guided decision-making. According to one view each frontal cortical region is concerned with a different aspect of the process of learning about and evaluating choices and then selecting actions. An alternative view, however, sees sets of decision-making circuits working in parallel within the frontal lobes in order to make different types of decisions. While there is a neural circuit for making choices between pairs of simultaneously presented items in the manner that is frequently assessed in the laboratory, there is also evidence that other frontal lobe circuits have evolved to make other types of choices such as those made during the course of foraging.

New advances in understanding decisions among multiple alternatives

Experimental studies of decision-making have put a strong emphasis on choices between two alternatives. However, real-life decisions often involve multiple alternatives. This article provides an overview of theoretical frameworks that have been proposed to account for behavioral data from both economic and perceptual multialternative decision-making. We further review recent neurophysiological data collected in conjunction with decision-making behavior. These neural recordings provide constraints on putative models of the decision mechanism. For example, the time course of inhibition provides insight into how the competition between alternatives is mediated. Furthermore, whereas decision-related neural activity seems to reach a common threshold at the end of the decision period, the starting point tends to depend systematically on the number of alternatives. We discuss candidate mechanisms that could drive the reduction in firing rates on decisions among multiple alternatives.

Reinforcement-Based Decision Making in Corticostriatal Circuits: Mutual Constraints by Neurocomputational and Diffusion Models

In this letter, we examine the computational mechanisms of reinforce-ment-based decision making. We bridge the gap across multiple levels of analysis, from neural models of corticostriatal circuits—the basal ganglia (BG) model (Frank, 2005 , 2006 ) to simpler but mathematically tractable diffusion models of two-choice decision making. Specifically, we generated simulated data from the BG model and fit the diffusion model (Ratcliff, 1978 ) to it. The standard diffusion model fits underestimated response times under conditions of high response and reinforcement conflict. Follow-up fits showed good fits to the data both by increasing nondecision time and by raising decision thresholds as a function of conflict and by allowing this threshold to collapse with time. This profile captures the role and dynamics of the subthalamic nucleus in BG circuitry, and as such, parametric modulations of projection strengths from this nucleus were associated with parametric increases in decision boundary and its modulation by conflict. We then present data from a human reinforcement learning experiment involving decisions with low- and high-reinforcement conflict. Again, the standard model failed to fit the data, but we found that two variants similar to those that fit the BG model data fit the experimental data, thereby providing a convergence of theoretical accounts of complex interactive decision-making mechanisms consistent with available data. This work also demonstrates how to make modest modifications to diffusion models to summarize core computations of the BG model. The result is a better fit and understanding of reinforcement-based choice data than that which would have occurred with either model alone.

Micro-valences: perceiving affective valence in everyday objects

Perceiving the affective valence of objects influences how we think about and react to the world around us. Conversely, the speed and quality with which we visually recognize objects in a visual scene can vary dramatically depending on that scene’s affective content. Although typical visual scenes contain mostly “everyday” objects, the affect perception in visual objects has been studied using somewhat atypical stimuli with strong affective valences (e.g., guns or roses). Here we explore whether affective valence must be strong or overt to exert an effect on our visual perception. We conclude that everyday objects carry subtle affective valences – “micro-valences” – which are intrinsic to their perceptual representation.

A theoretical account of cognitive effects in delay discounting

Although delay discounting, the attenuation of the value of future rewards, is a robust finding, the mechanism of discounting is not known. We propose a potential mechanism for delay discounting such that discounting emerges from a search process that is trying to determine what rewards will be available in the future. In this theory, the delay dependence of the discounting of future expected rewards arises from three assumptions. First, that the evaluation of outcomes involves a search process. Second, that the value is assigned to an outcome proportionally to how easy it is to find. Third, that outcomes that are less delayed are typically easier for the search process to find. By relaxing this third assumption (e.g. by assuming that episodically-cued outcomes are easier to find), our model suggests that it is possible to dissociate discounting from delay. Our theory thereby explains the empirical result that discounting is slower to episodically-imagined outcomes, because these outcomes are easier for the search process to find. Additionally, the theory explains why improving cognitive resources such as working memory slows discounting, by improving searches and thereby making rewards easier to find. The three assumptions outlined here are likely to be instantiated during deliberative decision-making, but are unlikely in habitual decision-making. We model two simple implementations of this theory and show that they unify empirical results about the role of cognitive function in delay discounting, and make new neural, behavioral, and pharmacological predictions.

Neural basis of reinforcement learning and decision making

Reinforcement learning is an adaptive process in which an animal utilizes its previous experience to improve the outcomes of future choices. Computational theories of reinforcement learning play a central role in the newly emerging areas of neuroeconomics and decision neuroscience. In this framework, actions are chosen according to their value functions, which describe how much future reward is expected from each action. Value functions can be adjusted not only through reward and penalty, but also by the animal's knowledge of its current environment. Studies have revealed that a large proportion of the brain is involved in representing and updating value functions and using them to choose an action. However, how the nature of a behavioral task affects the neural mechanisms of reinforcement learning remains incompletely understood. Future studies should uncover the principles by which different computational elements of reinforcement learning are dynamically coordinated across the entire brain.

The role of eye movements in decision making and the prospect of exposure effects

The aim of the current study was to follow on from previous findings that eye movements can have a causal influence on preference formation. Shimojo et al. (2003) previously found that faces that were presented for a longer duration in a two alternative forced choice task were more likely to be judged as more attractive. This effect only occurred when an eye movement was made towards the faces (with no effect when faces were centrally presented). The current study replicated Shimojo et al.'s (2003) design, whilst controlling for potential inter-stimuli interference in central presentations. As per previous findings, when eye movements were made towards the stimuli, faces that were presented for longer durations were preferred. However, faces that were centrally presented (thus not requiring an eye movement) were also preferred in the current study. The presence of an exposure duration effect for centrally presented faces casts doubt on the necessity of the eye movement in this decision making process and has implications for decision theories that place an emphasis on the role of eye movements in decision making.

Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates

Recent work has given rise to the view that reward-based decision making is governed by two key controllers: a habit system, which stores stimulus-response associations shaped by past reward, and a goal-oriented system that selects actions based on their anticipated outcomes. The current literature provides a rich body of computational theory addressing habit formation, centering on temporal-difference learning mechanisms. Less progress has been made toward formalizing the processes involved in goal-directed decision making. We draw on recent work in cognitive neuroscience, animal conditioning, cognitive and developmental psychology, and machine learning to outline a new theory of goal-directed decision making. Our basic proposal is that the brain, within an identifiable network of cortical and subcortical structures, implements a probabilistic generative model of reward, and that goal-directed decision making is effected through Bayesian inversion of this model. We present a set of simulations implementing the account, which address benchmark behavioral and neuroscientific findings, and give rise to a set of testable predictions. We also discuss the relationship between the proposed framework and other models of decision making, including recent models of perceptual choice, to which our theory bears a direct connection.

The decision value computations in the vmPFC and striatum use a relative value code that is guided by visual attention

There is a growing consensus in behavioral neuroscience that the brain makes simple choices by first assigning a value to the options under consideration and then comparing them. Two important open questions are whether the brain encodes absolute or relative value signals, and what role attention might play in these computations. We investigated these questions using a human fMRI experiment with a binary choice task in which the fixations to both stimuli were exogenously manipulated to control for the role of visual attention in the valuation computation. We found that the ventromedial prefrontal cortex and the ventral striatum encoded fixation-dependent relative value signals: activity in these areas correlated with the difference in value between the attended and the unattended items. These attention-modulated relative value signals might serve as the input of a comparator system that is used to make a choice.

Focusing attention on the health aspects of foods changes value signals in vmPFC and improves dietary choice

Attention is thought to play a key role in the computation of stimulus values at the time of choice, which suggests that attention manipulations could be used to improve decision-making in domains where self-control lapses are pervasive. We used an fMRI food choice task with non-dieting human subjects to investigate whether exogenous cues that direct attention to the healthiness of foods could improve dietary choices. Behaviorally, we found that subjects made healthier choices in the presence of health cues. In parallel, stimulus value signals in ventromedial prefrontal cortex were more responsive to the healthiness of foods in the presence of health cues, and this effect was modulated by activity in regions of dorsolateral prefrontal cortex. These findings suggest that the neural mechanisms used in successful self-control can be activated by exogenous attention cues, and provide insights into the processes through which behavioral therapies and public policies could facilitate self-control.

What cognitive processes drive response biases? A diffusion model analysis

We used a diffusion model to examine the effects of response-bias manipulations on response time (RT) and accuracy data collected in two experiments involving a two-choice decision making task. We asked 18 subjects to respond “low” or “high” to the number of asterisks in a 10×10 grid, based on an experimenter-determined decision cutoff. In the model, evidence is accumulated until either a “low” or “high” decision criterion is reached, and this, in turn, initiates a response. We performed two experiments with four experimental conditions. In conditions 1 and 2, the decision cutoff between low and high judgments was fixed at 50. In condition 1, we manipulated the frequency with which low- and high-stimuli were presented. In condition 2, we used payoff structures that mimicked the frequency manipulation. We found that manipulating stimulus frequency resulted in a larger effect on RT and accuracy than did manipulating payoff structure. In the model, we found that manipulating stimulus frequency produced greater changes in the starting point of the evidence accumulation process than did manipulating payoff structure. In conditions 3 and 4, we set the decision cutoff at 40, 50, or 60 (Experiment 1) and at 45 or 55 (Experiment 2). In condition 3, there was an equal number of low- and high-stimuli, whereas in condition 4 there were unequal proportions of low- and high-stimuli. The model analyses showed that starting-point changes accounted for biases produced by changes in stimulus proportions, whereas evidence biases accounted for changes in the decision cutoff.

Decision making and reward in frontal cortex: complementary evidence from neurophysiological and neuropsychological studies

Patients with damage to the prefrontal cortex (PFC)—especially the ventral and medial parts of PFC—often show a marked inability to make choices that meet their needs and goals. These decision-making impairments often reflect both a deficit in learning concerning the consequences of a choice, as well as deficits in the ability to adapt future choices based on experienced value of the current choice. Thus, areas of PFC must support some value computations that are necessary for optimal choice. However, recent frameworks of decision making have highlighted that optimal and adaptive decision making does not simply rest on a single computation, but a number of different value computations may be necessary. Using this framework as a guide, we summarize evidence from both lesion studies and single-neuron physiology for the representation of different value computations across PFC areas.

Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold

It takes effort and time to tame one's impulses. Although medial prefrontal cortex (mPFC) is broadly implicated in effortful control over behavior, the subthalamic nucleus (STN) is specifically thought to contribute by acting as a brake on cortico-striatal function during decision conflict, buying time until the right decision can be made. Using the drift diffusion model of decision making, we found that trial-to-trial increases in mPFC activity (EEG theta power, 4-8 Hz) were related to an increased threshold for evidence accumulation (decision threshold) as a function of conflict. Deep brain stimulation of the STN in individuals with Parkinson's disease reversed this relationship, resulting in impulsive choice. In addition, intracranial recordings of the STN area revealed increased activity (2.5-5 Hz) during these same high-conflict decisions. Activity in these slow frequency bands may reflect a neural substrate for cortico-basal ganglia communication regulating decision processes.

Affective privilege: asymmetric interference by emotional distracters

Numerous theories posit that affectively salient stimuli are privileged in their capacity to capture attention and disrupt ongoing cognition. Two underlying assumptions in this theoretical position are that the potency of affective stimuli transcends task boundaries (i.e., emotional distracters do not have to belong to a current task-set to disrupt processing) and that there is an asymmetry between emotional and cognitive processing (i.e., emotional distracters disrupt cognitive processing, but not vice versa). These assumptions have remained largely untested, as common experimental probes of emotion-cognition interaction rarely manipulate task-relevance and only examine one side of the presumed asymmetry of interference. To test these propositions directly, a face-word Stroop protocol was adapted to independently manipulate (a) the congruency between target and distracter stimulus features, (b) the affective salience of distracter features, and (c) the task-relevance of emotional compared to non-emotional target features. A three-way interaction revealed interdependent effects of distracter relevance, congruence, and affective salience. Compared to task-irrelevant distracters, task-relevant congruent distracters facilitated performance and task-relevant incongruent distracters impaired performance, but the latter effect depended on the nature of the target feature and task. Specifically, task-irrelevant emotional distracters resulted in equivalent performance costs as task-relevant non-emotional distracters, whereas task-irrelevant non-emotional distracters did not produce performance costs comparable to those generated by task-relevant emotional distracters. These results document asymmetric cross-task interference effects for affectively salient stimuli, supporting the notion of affective prioritization in human information processing.

Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions

How do we make decisions when confronted with several alternatives (e.g., on a supermarket shelf)? Previous work has shown that accumulator models, such as the drift-diffusion model, can provide accurate descriptions of the psychometric data for binary value-based choices, and that the choice process is guided by visual attention. However, the computational processes used to make choices in more complicated situations involving three or more options are unknown. We propose a model of trinary value-based choice that generalizes what is known about binary choice, and test it using an eye-tracking experiment. We find that the model provides a quantitatively accurate description of the relationship between choice, reaction time, and visual fixation data using the same parameters that were estimated in previous work on binary choice. Our findings suggest that the brain uses similar computational processes to make binary and trinary choices.

Consumers can make decisions in as little as a third of a second

We make hundreds of decisions every day, many of them extremely quickly and without much explicit deliberation. This motivates two important open questions: What is the minimum time required to make choices with above chance accuracy? What is the impact of additional decision-making time on choice accuracy? We investigated these questions in four experiments in which subjects made binary food choices using saccadic or manual responses, under either “speed” or “accuracy” instructions. Subjects were able to make above chance decisions in as little as 313 ms, and choose their preferred food item in over 70% of trials at average speeds of 404 ms. Further, slowing down their responses by either asking them explicitly to be confident about their choices, or to respond with hand movements, generated about a 10% increase in accuracy. Together, these results suggest that consumers can make accurate every-day choices, akin to those made in a grocery store, at significantly faster speeds than previously reported.

The human prefrontal cortex mediates integration of potential causes behind observed outcomes

Prefrontal cortex has long been implicated in tasks involving higher order inference in which decisions must be rendered, not only about which stimulus is currently rewarded, but also which stimulus dimensions are currently relevant. However, the precise computational mechanisms used to solve such tasks have remained unclear. We scanned human participants with functional MRI, while they performed a hierarchical intradimensional/extradimensional shift task to investigate what strategy subjects use while solving higher order decision problems. By using a computational model-based analysis, we found behavioral and neural evidence that humans solve such problems not by occasionally shifting focus from one to the other dimension, but by considering multiple explanations simultaneously. Activity in human prefrontal cortex was better accounted for by a model that integrates over all available evidences than by a model in which attention is selectively gated. Importantly, our model provides an explanation for how the brain determines integration weights, according to which it could distribute its attention. Our results demonstrate that, at the point of choice, the human brain and the prefrontal cortex in particular are capable of a weighted integration of information across multiple evidences.

Multiplicity of control in the basal ganglia: computational roles of striatal subregions

The basal ganglia, in particular the striatum, are central to theories of behavioral control, and often identified as a seat of action selection. Reinforcement learning (RL) models--which have driven much recent experimental work on this region--cast striatum as a dynamic controller, integrating sensory and motivational information to construct efficient and enriching behavioral policies. Befitting this informationally central role, the BG sit at the nexus of multiple anatomical 'loops' of synaptic projections, connecting a wide range of cortical and subcortical structures. Numerous pioneering anatomical studies conducted over the past several decades have meticulously catalogued these loops, and labeled them according to the inferred functions of the connected regions. The specific cotermina of the projections are highly localized to several different subregions of the striatum, leading to the suggestion that these subregions perform complementary but distinct functions. However, until recently, the dominant computational framework outlined only a bipartite, dorsal/ventral, division of striatum. We review recent computational and experimental advances that argue for a more finely fractionated delineation. In particular, experimental data provide extensive insight into unique functions subserved by the dorsomedial striatum (DMS). These functions appear to correspond well with theories of a 'model-based' RL subunit, and may also shed light on the suborganization of ventral striatum. Finally, we discuss the limitations of these ideas and how they point the way toward future refinements of neurocomputational theories of striatal function, bringing them into contact with other areas of computational theory and other regions of the brain.