Visual fixations and the computation and comparison of value in simple choice

Noisy preferences in risky choice: A cautionary note

We examine the effects of multiple sources of noise in risky decision making. Noise in the parameters that characterize an individual's preferences can combine with noise in the response process to distort observed choice proportions. Thus, underlying preferences that conform to expected value maximization can appear to show systematic risk aversion or risk seeking. Similarly, core preferences that are consistent with expected utility theory, when perturbed by such noise, can appear to display nonlinear probability weighting. For this reason, modal choices cannot be used simplistically to infer underlying preferences. Quantitative model fits that do not allow for both sorts of noise can lead to wrong conclusions. (PsycINFO Database Record

Final Sampling Bias in Haptic Judgments: How Final Touch Affects Decision-Making

When people make a choice between multiple items, they usually evaluate each item one after the other repeatedly. The effect of the order and number of evaluating items on one’s choices is essential to understanding the decision-making process. Previous studies have shown that when people choose a favorable item from two items, they tend to choose the item that they evaluated last. This tendency has been observed regardless of sensory modalities. This study investigated the origin of this bias by using three experiments involving two-alternative forced-choice tasks using handkerchiefs. First, the bias appeared in a smoothness discrimination task, which indicates that the bias was not based on judgments of preference. Second, the handkerchief that was touched more often tended to be chosen more frequently in the preference task, but not in the smoothness discrimination task, indicating that a mere exposure effect enhanced the bias. Third, in the condition where the number of touches did not differ between handkerchiefs, the bias appeared when people touched a handkerchief they wanted to touch last, but not when people touched the handkerchief that was predetermined. This finding suggests a direct coupling between final voluntary touching and judgment.

Nutrition labeling reduces valuations of food through multiple health and taste channels

One popularized technique to promote healthy dietary choice involves posting calorie or other nutritional information at the time individuals make a consumption decision. While the evidence on the effectiveness of such interventions is mixed, relatively little work has focused on the underlying mechanisms of how such labels alter behavior. In the research reported here, we asked 87 hungry laboratory subjects to make bids over foods with or without nutrition labels present. We found that the presence of a nutrition label reduced bids by an average of 25 cents. Furthermore, we found this reduction was driven by differences in perceptions and the importance individuals placed on health features of the foods, but also by differences in the importance individuals placed on more visceral taste features. These results help explain the various methods in which nutritional information postings or other policy tools can nudge individuals to consume healthier options.

The Attentional Drift Diffusion Model of Simple Perceptual Decision-Making

Perceptual decisions requiring the comparison of spatially distributed stimuli that are fixated sequentially might be influenced by fluctuations in visual attention. We used two psychophysical tasks with human subjects to investigate the extent to which visual attention influences simple perceptual choices, and to test the extent to which the attentional Drift Diffusion Model (aDDM) provides a good computational description of how attention affects the underlying decision processes. We find evidence for sizable attentional choice biases and that the aDDM provides a reasonable quantitative description of the relationship between fluctuations in visual attention, choices and reaction times. We also find that exogenous manipulations of attention induce choice biases consistent with the predictions of the model.

Multi-attribute, multi-alternative models of choice: Choice, reaction time, and process tracing

The first aim of this research is to compare computational models of multi-alternative, multi-attribute choice when attribute values are explicit. The choice predictions of utility (standard random utility & weighted valuation), heuristic (elimination-by-aspects, lexicographic, & maximum attribute value), and dynamic (multi-alternative decision field theory, MDFT, & a version of the multi-attribute linear ballistic accumulator, MLBA) models are contrasted on both preferential and risky choice data. Using both maximum likelihood and cross-validation fit measures on choice data, the utility and dynamic models are preferred over the heuristic models for risky choice, with a slight overall advantage for the MLBA for preferential choice. The response time predictions of these models (except the MDFT) are then tested. Although the MLBA accurately predicts response time distributions, it only weakly accounts for stimulus-level differences. The other models completely fail to account for stimulus-level differences. Process tracing measures, i.e., eye and mouse tracking, were also collected. None of the qualitative predictions of the models are completely supported by that data. These results suggest that the models may not appropriately represent the interaction of attention and preference formation. To overcome this potential shortcoming, the second aim of this research is to test preference-formation assumptions, independently of attention, by developing the models of attentional sampling (MAS) model family which incorporates the empirical gaze patterns into a sequential sampling framework. An MAS variant that includes attribute values, but only updates the currently viewed alternative and does not contrast values across alternatives, performs well in both experiments. Overall, the results support the dynamic models, but point to the need to incorporate a framework that more accurately reflects the relationship between attention and the preference-formation process.

Finding the "self" in self-regulation: The identity-value model

Many psychological theories suggest a link between self-regulation and identity, but until now a mechanistic account that suggests ways to improve self-regulation has not been put forth. The identity-value model (IVM) connects the idea from social psychology, that aspects of identity such as core values and group affiliations hold positive subjective value, to the process-focused account from decision-making and behavioral economics, that self-regulation is driven by a dynamic value integration across a range of choice attributes. Together, these ideas imply that goal-directed behaviors that are identity-relevant are more likely to be enacted because they have greater subjective value than identity-irrelevant behaviors. A central hypothesis, therefore, is that interventions that increase the degree to which a target behavior is perceived as self-relevant will improve self-regulation. Additionally, identity-based changes in self-regulation are expected to be mediated by changes in subjective value and its underlying neural systems. In this paper, we define the key constructs relevant to the IVM, explicate the model and delineate its boundary conditions, and describe how it fits with related theories. We also review disparate results in the research literature that might share identity-related value as a common underlying mechanism of action. We close by discussing questions about the model whose answers could advance the study of self-regulation.

Dynamic Interaction between Reinforcement Learning and Attention in Multidimensional Environments

Little is known about the relationship between attention and learning during decision making. Using eye tracking and multivariate pattern analysis of fMRI data, we measured participants' dimensional attention as they performed a trial-and-error learning task in which only one of three stimulus dimensions was relevant for reward at any given time. Analysis of participants' choices revealed that attention biased both value computation during choice and value update during learning. Value signals in the ventromedial prefrontal cortex and prediction errors in the striatum were similarly biased by attention. In turn, participants' focus of attention was dynamically modulated by ongoing learning. Attentional switches across dimensions correlated with activity in a frontoparietal attention network, which showed enhanced connectivity with the ventromedial prefrontal cortex between switches. Our results suggest a bidirectional interaction between attention and learning: attention constrains learning to relevant dimensions of the environment, while we learn what to attend to via trial and error.

Metacognitive impairments extend perceptual decision making weaknesses in compulsivity

Awareness of one's own abilities is of paramount importance in adaptive decision making. Psychotherapeutic theories assume such metacognitive insight is impaired in compulsivity, though this is supported by scant empirical evidence. In this study, we investigate metacognitive abilities in compulsive participants using computational models, where these enable a segregation between metacognitive and perceptual decision making impairments. We examined twenty low-compulsive and twenty high-compulsive participants, recruited from a large population-based sample, and matched for other psychiatric and cognitive dimensions. Hierarchical computational modelling of the participants' metacognitive abilities on a visual global motion detection paradigm revealed that high-compulsive participants had a reduced metacognitive ability. This impairment was accompanied by a perceptual decision making deficit whereby motion-related evidence was accumulated more slowly in high compulsive participants. Our study shows that the compulsivity spectrum is associated with a reduced ability to monitor one's own performance, over and above any perceptual decision making difficulties.

A Two-Stage Process Model of Sensory Discrimination: An Alternative to Drift-Diffusion

Discrimination of the direction of motion of a noisy stimulus is an example of sensory discrimination under uncertainty. For stimuli that are extended in time, reaction time is quicker for larger signal values (e.g., discrimination of opposite directions of motion compared with neighboring orientations) and larger signal strength (e.g., stimuli with higher contrast or motion coherence, that is, lower noise). The standard model of neural responses (e.g., in lateral intraparietal cortex) and reaction time for discrimination is drift-diffusion. This model makes two clear predictions. (1) The effects of signal strength and value on reaction time should interact multiplicatively because the diffusion process depends on the signal-to-noise ratio. (2) If the diffusion process is interrupted, as in a cued-response task, the time to decision after the cue should be independent of the strength of accumulated sensory evidence. In two experiments with human participants, we show that neither prediction holds. A simple alternative model is developed that is consistent with the results. In this estimate-then-decide model, evidence is accumulated until estimation precision reaches a threshold value. Then, a decision is made with duration that depends on the signal-to-noise ratio achieved by the first stage.

SIGNIFICANCE STATEMENT

Sensory decision-making under uncertainty is usually modeled as the slow accumulation of noisy sensory evidence until a threshold amount of evidence supporting one of the possible decision outcomes is reached. Furthermore, it has been suggested that this accumulation process is reflected in neural responses, e.g., in lateral intraparietal cortex. We derive two behavioral predictions of this model and show that neither prediction holds. We introduce a simple alternative model in which evidence is accumulated until a sufficiently precise estimate of the stimulus is achieved, and then that estimate is used to guide the discrimination decision. This model is consistent with the behavioral data.

Choice Rules and Accumulator Networks

This article presents a preference accumulation model that can be used to implement a number of different multi-attribute heuristic choice rules, including the lexicographic rule, the majority of confirming dimensions (tallying) rule and the equal weights rule. The proposed model differs from existing accumulators in terms of attribute representation: Leakage and competition, typically applied only to preference accumulation, are also assumed to be involved in processing attribute values. This allows the model to perform a range of sophisticated attribute-wise comparisons, including comparisons that compute relative rank. The ability of a preference accumulation model composed of leaky competitive networks to mimic symbolic models of heuristic choice suggests that these 2 approaches are not incompatible, and that a unitary cognitive model of preferential choice, based on insights from both these approaches, may be feasible.

Decision Making and Sequential Sampling from Memory

Decisions take time, and as a rule more difficult decisions take more time. But this only raises the question of what consumes the time. For decisions informed by a sequence of samples of evidence, the answer is straightforward: more samples are available with more time. Indeed, the speed and accuracy of such decisions are explained by the accumulation of evidence to a threshold or bound. However, the same framework seems to apply to decisions that are not obviously informed by sequences of evidence samples. Here, we proffer the hypothesis that the sequential character of such tasks involves retrieval of evidence from memory. We explore this hypothesis by focusing on value-based decisions and argue that mnemonic processes can account for regularities in choice and decision time. We speculate on the neural mechanisms that link sampling of evidence from memory to circuits that represent the accumulated evidence bearing on a choice. We propose that memory processes may contribute to a wider class of decisions that conform to the regularities of choice-reaction time predicted by the sequential sampling framework.

Demotivating incentives and motivation crowding out in charitable giving

Research has shown that extrinsic incentives can crowd out intrinsic motivation in many contexts. Despite this, many nonprofits offer conditional thank-you gifts, such as mugs or tote bags, in exchange for donations. In collaboration with a nonprofit, this study implements a direct mail field experiment and demonstrates that thank-you gifts reduced donation rates in a fundraising campaign. Attention-based multiattribute choice models suggest that this is because prospective donors shift attention to the salient gift offer, causing them to underweight less salient intrinsic motives. Attention to the gift may also cause individuals to adopt a more cost-benefit mindset, further de-emphasizing intrinsic motives. Consistent with these hypotheses, crowding out was driven by those who donated higher amounts in the previous year (i.e., those who likely had higher intrinsic motivation). In a complementary online experiment, thank-you gifts also reduced donation rates but only when the gift was visually salient. This corroborates the mediating role of attention in crowding out. Taken together, the laboratory and field results demonstrate that this fundraising technique can be demotivating in some contexts and that this may occur through an attention-based mechanism.

Reminders of past choices bias decisions for reward in humans

We provide evidence that decisions are made by consulting memories for individual past experiences, and that this process can be biased in favour of past choices using incidental reminders. First, in a standard rewarded choice task, we show that a model that estimates value at decision-time using individual samples of past outcomes fits choices and decision-related neural activity better than a canonical incremental learning model. In a second experiment, we bias this sampling process by incidentally reminding participants of individual past decisions. The next decision after a reminder shows a strong influence of the action taken and value received on the reminded trial. These results provide new empirical support for a decision architecture that relies on samples of individual past choice episodes rather than incrementally averaged rewards in evaluating options and has suggestive implications for the underlying cognitive and neural mechanisms.

From faces to prosocial behavior: cues, tools, and mechanisms

In this review we ask how looking at people's faces can influence prosocial behaviors towards them. Components of this process have often been studied by disparate literatures: one focused on perception and judgment of faces, using both psychological and neuroscience approaches; and a second focused on actual social behaviors, as studied in behavioral economics and decision science. Bridging these disciplines requires a more mechanistic account of how processing of particular face attributes or features influences social judgments and behaviors. Here we review these two lines of research, and suggest that combining some of their methodological tools can provide the bridging mechanistic explanations.

Neural correlates of evidence accumulation during value-based decisions revealed via simultaneous EEG-fMRI

Current computational accounts posit that, in simple binary choices, humans accumulate evidence in favour of the different alternatives before committing to a decision. Neural correlates of this accumulating activity have been found during perceptual decisions in parietal and prefrontal cortex; however the source of such activity in value-based choices remains unknown. Here we use simultaneous EEG–fMRI and computational modelling to identify EEG signals reflecting an accumulation process and demonstrate that the within- and across-trial variability in these signals explains fMRI responses in posterior-medial frontal cortex. Consistent with its role in integrating the evidence prior to reaching a decision, this region also exhibits task-dependent coupling with the ventromedial prefrontal cortex and the striatum, brain areas known to encode the subjective value of the decision alternatives. These results further endorse the proposition of an evidence accumulation process during value-based decisions in humans and implicate the posterior-medial frontal cortex in this process.

Parietal and prefrontal cortices gather information to make perceptual decisions, but it is not known if the same is true for value-based choices. Here, authors use simultaneous EEG-fMRI and modelling to show that during value- and reward-based decisions this evidence is accumulated in the posterior medial frontal cortex.

The BCD of response time analysis in experimental economics

For decisions in the wild, time is of the essence. Available decision time is often cut short through natural or artificial constraints, or is impinged upon by the opportunity cost of time. Experimental economists have only recently begun to conduct experiments with time constraints and to analyze response time (RT) data, in contrast to experimental psychologists. RT analysis has proven valuable for the identification of individual and strategic decision processes including identification of social preferences in the latter case, model comparison/selection, and the investigation of heuristics that combine speed and performance by exploiting environmental regularities. Here we focus on the benefits, challenges, and desiderata of RT analysis in strategic decision making. We argue that unlocking the potential of RT analysis requires the adoption of process-based models instead of outcome-based models, and discuss how RT in the wild can be captured by time-constrained experiments in the lab. We conclude that RT analysis holds considerable potential for experimental economics, deserves greater attention as a methodological tool, and promises important insights on strategic decision making in naturally occurring environments.

Advancing Research on Cognitive Processes in Social and Personality Psychology

We provide a primer on a hierarchical extension of the drift diffusion model (DDM). This formal model of decisions is frequently used in the cognitive sciences but infrequently used in social and personality research. Recent advances in model estimation have overcome issues that previously made the hierarchical DDM impractical to implement. Using examples from two paradigms, the first-person shooter task and the flash gambling task, we demonstrate that the hierarchical DDM can provide novel insights into cognitive processes underlying decisions. Finally, we compare the DDM to dual-process models of decision-making. We hope this primer will provide researchers a new tool for investigating psychological processes.

Computational modeling of epiphany learning

Models of reinforcement learning (RL) are prevalent in the decision-making literature, but not all behavior seems to conform to the gradual convergence that is a central feature of RL. In some cases learning seems to happen all at once. Limited prior research on these "epiphanies" has shown evidence of sudden changes in behavior, but it remains unclear how such epiphanies occur. We propose a sequential-sampling model of epiphany learning (EL) and test it using an eye-tracking experiment. In the experiment, subjects repeatedly play a strategic game that has an optimal strategy. Subjects can learn over time from feedback but are also allowed to commit to a strategy at any time, eliminating all other options and opportunities to learn. We find that the EL model is consistent with the choices, eye movements, and pupillary responses of subjects who commit to the optimal strategy (correct epiphany) but not always of those who commit to a suboptimal strategy or who do not commit at all. Our findings suggest that EL is driven by a latent evidence accumulation process that can be revealed with eye-tracking data.

The socialization effect on decision making in the Prisoner's Dilemma game: An eye-tracking study

We used a mobile eye-tracking system (in the form of glasses) to study the characteristics of visual perception in decision making in the Prisoner's Dilemma game. In each experiment, one of the 12 participants was equipped with eye-tracking glasses. The experiment was conducted in three stages: an anonymous Individual Game stage against a randomly chosen partner (one of the 12 other participants of the experiment); a Socialization stage, in which the participants were divided into two groups; and a Group Game stage, in which the participants played with partners in the groups. After each round, the respondent received information about his or her personal score in the last round and the overall winner of the game at the moment. The study proves that eye-tracking systems can be used for studying the process of decision making and forecasting. The total viewing time and the time of fixation on areas corresponding to noncooperative decisions is related to the participants’ overall level of cooperation. The increase in the total viewing time and the time of fixation on the areas of noncooperative choice is due to a preference for noncooperative decisions and a decrease in the overall level of cooperation. The number of fixations on the group attributes is associated with group identity, but does not necessarily lead to cooperative behavior.

A martingale analysis of first passage times of time-dependent Wiener diffusion models

Research in psychology and neuroscience has successfully modeled decision making as a process of noisy evidence accumulation to a decision bound. While there are several variants and implementations of this idea, the majority of these models make use of a noisy accumulation between two absorbing boundaries. A common assumption of these models is that decision parameters, e.g., the rate of accumulation (drift rate), remain fixed over the course of a decision, allowing the derivation of analytic formulas for the probabilities of hitting the upper or lower decision threshold, and the mean decision time. There is reason to believe, however, that many types of behavior would be better described by a model in which the parameters were allowed to vary over the course of the decision process. In this paper, we use martingale theory to derive formulas for the mean decision time, hitting probabilities, and first passage time (FPT) densities of a Wiener process with time-varying drift between two time-varying absorbing boundaries. This model was first studied by Ratcliff (1980) in the two-stage form, and here we consider the same model for an arbitrary number of stages (i.e. intervals of time during which parameters are constant). Our calculations enable direct computation of mean decision times and hitting probabilities for the associated multistage process. We also provide a review of how martingale theory may be used to analyze similar models employing Wiener processes by re-deriving some classical results. In concert with a variety of numerical tools already available, the current derivations should encourage mathematical analysis of more complex models of decision making with time-varying evidence.

A Quantitative Relationship between Signal Detection in Attention and Approach/Avoidance Behavior

This study examines how the domains of reward and attention, which are often studied as independent processes, in fact interact at a systems level. We operationalize divided attention with a continuous performance task and variables from signal detection theory (SDT), and reward/aversion with a keypress task measuring approach/avoidance in the framework of relative preference theory (RPT). Independent experiments with the same subjects showed a significant association between one SDT and two RPT variables, visualized as a three-dimensional structure. Holding one of these three variables constant, further showed a significant relationship between a loss aversion-like metric from the approach/avoidance task, and the response bias observed during the divided attention task. These results indicate that a more liberal response bias under signal detection (i.e., a higher tolerance for noise, resulting in a greater proportion of false alarms) is associated with higher "loss aversion." Furthermore, our functional model suggests a mechanism for processing constraints with divided attention and reward/aversion. Together, our results argue for a systematic relationship between divided attention and reward/aversion processing in humans.

Deliberation period during easy and difficult decisions: re-examining Libet's "veto" window in a more ecologically valid framework

Whether consciousness plays a causal role in cognitive processing remains debated. According to Benjamin Libet, consciousness is needed to deliberate and veto an action that is initiated unconsciously. Libet offered that the deliberation window takes place between the time of conscious intent (W) and action (MR). We further examined this deliberation-veto hypothesis by measuring the length of the temporal window (W-MR) when making easy and difficult choices. If Libet were correct that the W-MR is intended for evaluation and cancelation, we should expect a shorter W-MR for an easy decision since less deliberation is presumably needed. Instead, we observed a less intuitive effect: The W-MR window in the easy trials was longer than the W-MR window in the difficult ones. Our results suggest several interpretations including the idea that consciousness may play a causal role in decision making but not in a straightforward manner as assumed by Libet's veto hypothesis.

What to Choose Next? A Paradigm for Testing Human Sequential Decision Making

Many of the decisions we make in our everyday lives are sequential and entail sparse rewards. While sequential decision-making has been extensively investigated in theory (e.g., by reinforcement learning models) there is no systematic experimental paradigm to test it. Here, we developed such a paradigm and investigated key components of reinforcement learning models: the eligibility trace (i.e., the memory trace of previous decision steps), the external reward, and the ability to exploit the statistics of the environment's structure (model-free vs. model-based mechanisms). We show that the eligibility trace decays not with sheer time, but rather with the number of discrete decision steps made by the participants. We further show that, unexpectedly, neither monetary rewards nor the environment's spatial regularity significantly modulate behavioral performance. Finally, we found that model-free learning algorithms describe human performance better than model-based algorithms.

What Is Trained During Food Go/No-Go Training? A Review Focusing on Mechanisms and a Research Agenda

Purpose of Review

During food go/no-go training, people consistently withhold responses toward no-go food items. We discuss how food go/no-go training may change people’s behavior toward no-go food items by comparing three accounts: (a) the training strengthens ‘top-down’ inhibitory control over food-related responses, (b) the training creates automatic ‘bottom-up’ associations between no-go food items and stopping responses, and (c) the training leads to devaluation of no-go food items.

Recent Findings

Go/no-go training can reduce intake of food and choices for food and facilitate short-term weight loss. It appears unlikely that food go/no-go training strengthens top-down inhibitory control. There is some evidence suggesting the training could create automatic stop associations. There is strong evidence suggesting go/no-go training reduces evaluations of no-go food items.

Summary

Food go/no-go training can change behavior toward food and evaluation of food items. To advance knowledge, more research is needed on the underlying mechanisms of the training, the role of attention during go/no-go training, and on when effects generalize to untrained food items.

Robust sampling of decision information during perceptual choice

Humans move their eyes to gather information about the visual world. However, saccadic sampling has largely been explored in paradigms that involve searching for a lone target in a cluttered array or natural scene. Here, we investigated the policy that humans use to overtly sample information in a perceptual decision task that required information from across multiple spatial locations to be combined. Participants viewed a spatial array of numbers and judged whether the average was greater or smaller than a reference value. Participants preferentially sampled items that were less diagnostic of the correct answer ("inlying" elements; that is, elements closer to the reference value). This preference to sample inlying items was linked to decisions, enhancing the tendency to give more weight to inlying elements in the final choice ("robust averaging"). These findings contrast with a large body of evidence indicating that gaze is directed preferentially to deviant information during natural scene viewing and visual search, and suggest that humans may sample information "robustly" with their eyes during perceptual decision-making.

A distributed, hierarchical and recurrent framework for reward-based choice

Many accounts of reward-based choice argue for distinct component processes that are serial and functionally localized. In this Opinion article, we argue for an alternative viewpoint, in which choices emerge from repeated computations that are distributed across many brain regions. We emphasize how several features of neuroanatomy may support the implementation of choice, including mutual inhibition in recurrent neural networks and the hierarchical organization of timescales for information processing across the cortex. This account also suggests that certain correlates of value are emergent rather than represented explicitly in the brain.

Information foraging for perceptual decisions

We tested an information foraging framework to characterize the mechanisms that drive active (visual) sampling behavior in decision problems that involve multiple sources of information. Experiments 1 through 3 involved participants making an absolute judgment about the direction of motion of a single random dot motion pattern. In Experiment 4, participants made a relative comparison between 2 motion patterns that could only be sampled sequentially. Our results show that: (a) Information (about noisy motion information) grows to an asymptotic level that depends on the quality of the information source; (b) The limited growth is attributable to unequal weighting of the incoming sensory evidence, with early samples being weighted more heavily; (c) Little information is lost once a new source of information is being sampled; and (d) The point at which the observer switches from 1 source to another is governed by online monitoring of his or her degree of (un)certainty about the sampled source. These findings demonstrate that the sampling strategy in perceptual decision-making is under some direct control by ongoing cognitive processing. More specifically, participants are able to track a measure of (un)certainty and use this information to guide their sampling behavior. (PsycINFO Database Record

Modeling Trait Anxiety: From Computational Processes to Personality

Computational methods are increasingly being applied to the study of psychiatric disorders. Often, this involves fitting models to the behavior of individuals with subclinical character traits that are known vulnerability factors for the development of psychiatric conditions. Anxiety disorders can be examined with reference to the behavior of individuals high in "trait" anxiety, which is a known vulnerability factor for the development of anxiety and mood disorders. However, it is not clear how this self-report measure relates to neural and behavioral processes captured by computational models. This paper reviews emerging computational approaches to the study of trait anxiety, specifying how interacting processes susceptible to analysis using computational models could drive a tendency to experience frequent anxious states and promote vulnerability to the development of clinical disorders. Existing computational studies are described in the light of this perspective and appropriate targets for future studies are discussed.

Innovative Techniques for Evaluating Behavioral Nutrition Interventions

Assessing outcomes and the impact from behavioral nutrition interventions has remained challenging because of the lack of methods available beyond traditional nutrition assessment tools and techniques. With the current high global obesity and related chronic disease rates, novel methods to evaluate the impact of behavioral nutrition-based interventions are much needed. The objective of this narrative review is to describe and review the current status of knowledge as it relates to 4 different innovative methods or tools to assess behavioral nutrition interventions. Methods reviewed include 1) the assessment of stress and stress responsiveness to enhance the evaluation of nutrition interventions, 2) eye-tracking technology in nutritional interventions, 3) smartphone biosensors to assess nutrition and health-related outcomes, and 4) skin carotenoid measurements to assess fruit and vegetable intake. Specifically, the novel use of functional magnetic resonance imaging, by characterizing the brain's responsiveness to an intervention, can help researchers develop programs with greater efficacy. Similarly, if eye-tracking technology can enable researchers to get a better sense as to how participants view materials, the materials may be better tailored to create an optimal impact. The latter 2 techniques reviewed, smartphone biosensors and methods to detect skin carotenoids, can provide the research community with portable, effective, nonbiased ways to assess dietary intake and quality and more in the field. The information gained from using these types of methodologies can improve the efficacy and assessment of behavior-based nutrition interventions.

Psychopathic individuals exhibit but do not avoid regret during counterfactual decision making

Psychopathy is associated with persistent antisocial behavior and a striking lack of regret for the consequences of that behavior. Although explanatory models for psychopathy have largely focused on deficits in affective responsiveness, recent work indicates that aberrant value-based decision making may also play a role. On that basis, some have suggested that psychopathic individuals may be unable to effectively use prospective simulations to update action value estimates during cost-benefit decision making. However, the specific mechanisms linking valuation, affective deficits, and maladaptive decision making in psychopathy remain unclear. Using a counterfactual decision-making paradigm, we found that individuals who scored high on a measure of psychopathy were as or more likely than individuals low on psychopathy to report negative affect in response to regret-inducing counterfactual outcomes. However, despite exhibiting intact affective regret sensitivity, they did not use prospective regret signals to guide choice behavior. In turn, diminished behavioral regret sensitivity predicted a higher number of prior incarcerations, and moderated the relationship between psychopathy and incarceration history. These findings raise the possibility that maladaptive decision making in psychopathic individuals is not a consequence of their inability to generate or experience negative emotions. Rather, antisocial behavior in psychopathy may be driven by a deficit in the generation of forward models that integrate information about rules, costs, and goals with stimulus value representations to promote adaptive behavior.

How prior preferences determine decision-making frames and biases in the human brain

Understanding how option values are compared when making a choice is a key objective for decision neuroscience. In natural situations, agents may have a priori on their preferences that create default policies and shape the neural comparison process. We asked participants to make choices between items belonging to different categories (e.g., jazz vs. rock music). Behavioral data confirmed that the items taken from the preferred category were chosen more often and more rapidly, which qualified them as default options. FMRI data showed that baseline activity in classical brain valuation regions, such as the ventromedial Prefrontal Cortex (vmPFC), reflected the strength of prior preferences. In addition, evoked activity in the same regions scaled with the default option value, irrespective of the eventual choice. We therefore suggest that in the brain valuation system, choices are framed as comparisons between default and alternative options, which might save some resource but induce a decision bias.

DOI:http://dx.doi.org/10.7554/eLife.20317.001

If you had the choice of listening to a piece of music by either the singer Céline Dion or jazz pianist Keith Jarrett, which would you pick? When choosing between two mutually exclusive options, the brain first assigns a value to each. An area called the ventromedial prefrontal cortex (vmPFC) compares these two values and calculates the difference between them. The vmPFC then relays this difference to other brain regions that trigger the movements required to obtain the selected option.

But what exactly is the vmPFC comparing? A reasonable assumption is that we approach the decision with an existing preference for one of the options based on our previous experience. Lopez-Persem et al. set out to determine whether and how the vmPFC uses this existing preference – for example, for pop music over jazz – to drive the decision-making process.

For the experiments, volunteers were asked to rate how much they liked individual musicians spanning a range of different genres. While lying inside a brain scanner, the subjects then had to choose their favorite from pairs of musicians selected from the list. When making such decisions, volunteers must consider both the overall category (do I prefer jazz or pop?) but also the individual examples (a pop music fan might choose jazz if the pop option is Britney Spears). Lopez-Persem et al. found that the volunteer’s decisions were biased towards their prior preference. Pop music fans chose Céline Dion or Britney Spears more often than would be expected based on the likability ratings they had given the individual artists in the study.

Brain imaging revealed that the vmPFC represents choices as ‘default minus alternative’, where the default is any member of the previously preferred category (e.g. any pop artist for a pop music fan) and the alternative is from a different category (e.g. a jazz artist). Baseline vmPFC activity is higher for members of the preferred category, giving these options a head start over the alternatives. Asking volunteers to choose between other types of objects, including food and magazines, produced similar results. The brain thus uses a general strategy for decision-making that saves time and effort, but which also introduces bias. The next step is to work out how downstream brain regions use the vmPFC signal to select the preferred option.

DOI:http://dx.doi.org/10.7554/eLife.20317.002

Incubation of Cue-Induced Craving in Adults Addicted to Cocaine Measured by Electroencephalography

IMPORTANCE

A common trigger for relapse in drug addiction is the experience of craving via exposure to cues previously associated with drug use. Preclinical studies have consistently demonstrated incubation of cue-induced drug-seeking during the initial phase of abstinence, followed by a decline over time. In humans, the incubation effect has been shown for alcohol, nicotine, and methamphetamine addictions, but not for heroin or cocaine addiction. Understanding the trajectory of cue-induced craving during abstinence in humans is of importance for addiction medicine.

OBJECTIVE

To assess cue-induced craving for cocaine in humans using both subjective and objective indices of cue-elicited responses.

DESIGN, SETTING, AND PARTICIPANTS

Seventy-six individuals addicted to cocaine with varying durations of abstinence (ie, 2 days, 1 week, 1 month, 6 months, and 1 year) participated in this laboratory-based cross-sectional study from June 19, 2007, to November 26, 2012. The late positive potential component of electroencephalography, a recognized marker of incentive salience, was used to track motivated attention to drug cues across these self-selected groups. Participants also completed subjective ratings of craving for cocaine before presentation of a cue, and ratings of cocaine "liking" (hedonic feelings toward cocaine) and "wanting" (craving for cocaine) after presentation of cocaine-related pictures. Data analysis was conducted from June 5, 2015, to March 30, 2016.

MAIN OUTCOMES AND MEASURES

The late positive potential amplitudes and ratings of liking and wanting cocaine in response to cocaine-related pictures were quantified and compared across groups.

RESULTS

Among the 76 individuals addicted to cocaine, 19 (25%) were abstinent for 2 days, 20 (26%) were abstinent for 1 week, 15 (20%) were abstinent for 1 month, 12 (16%) were abstinent for 6 months, and 10 (13%) were abstinent for 1 year. In response to drug cues, the mean (SD) late positive potential amplitudes showed a parabolic trajectory that was higher at 1 (1.26 [1.36] µV) and 6 (1.17 [1.19] µV) months of abstinence and lower at 2 days (0.17 [1.09] µV), 1 week (0.36 [1.26] µV), and 1 year (-0.27 [1.74] µV) of abstinence (P = .02, partial η2 = 0.16). In contrast, the subjective assessment of baseline craving (mean [SD] rating: 2 days, 26.05 [9.85]; 1 week, 18.70 [11.01]; 1 month, 10.87 [10.70]; 6 months, 6.92 [8.47]; and 1 year, 3.00 [3.77]) and cue-induced liking (mean [SD] rating: 2 days, 3.06 [2.34]; 1 week, 2.33 [2.87]; 1 month, 1.15 [2.03]; 6 months, 1.00 [2.24]; and 1 year, 1.00 [1.26]) and wanting (mean [SD] rating: 2 days, 3.44 [2.62]; 1 week, 2.72 [2.87]; 1 month, 1.46 [2.33]; 6 months, 1.00 [2.16]; and 1 year, 1.00 [1.55]) of cocaine showed a linear decline from 2 days to 1 year of abstinence (P ≤ .001, partial η2 > 0.26).

CONCLUSIONS AND RELEVANCE

The late positive potential responses to drug cues, indicative of motivated attention, showed a trajectory similar to that reported in animal models. In contrast, we did not detect incubation of subjective cue-induced craving. Thus, the objective electroencephalographic measure may possibly be a better indicator of vulnerability to cue-induced relapse than subjective reports of craving, although this hypothesis must be empirically tested. These results suggest the importance of deploying intervention between 1 month and 6 months of abstinence, when addicted individuals may be most vulnerable to, and perhaps least cognizant of, risk of relapse.

Approach-Induced Biases in Human Information Sampling

Information sampling is often biased towards seeking evidence that confirms one's prior beliefs. Despite such biases being a pervasive feature of human behavior, their underlying causes remain unclear. Many accounts of these biases appeal to limitations of human hypothesis testing and cognition, de facto evoking notions of bounded rationality, but neglect more basic aspects of behavioral control. Here, we investigated a potential role for Pavlovian approach in biasing which information humans will choose to sample. We collected a large novel dataset from 32,445 human subjects, making over 3 million decisions, who played a gambling task designed to measure the latent causes and extent of information-sampling biases. We identified three novel approach-related biases, formalized by comparing subject behavior to a dynamic programming model of optimal information gathering. These biases reflected the amount of information sampled ("positive evidence approach"), the selection of which information to sample ("sampling the favorite"), and the interaction between information sampling and subsequent choices ("rejecting unsampled options"). The prevalence of all three biases was related to a Pavlovian approach-avoid parameter quantified within an entirely independent economic decision task. Our large dataset also revealed that individual differences in the amount of information gathered are a stable trait across multiple gameplays and can be related to demographic measures, including age and educational attainment. As well as revealing limitations in cognitive processing, our findings suggest information sampling biases reflect the expression of primitive, yet potentially ecologically adaptive, behavioral repertoires. One such behavior is sampling from options that will eventually be chosen, even when other sources of information are more pertinent for guiding future action.

Implications of Visual Attention Phenomena for Models of Preferential Choice

We use computational modeling to examine the ability of evidence accumulation models to produce the reaction time (RT) distributions and attentional biases found in behavioral and eye-tracking research. We focus on simulating RTs and attention in binary choice with particular emphasis on whether different models can predict the late onset bias (LOB), commonly found in eye movements during choice (sometimes called the gaze cascade). The first finding is that this bias is predicted by models even when attention is entirely random and independent of the choice process. This shows that the LOB is not evidence of a feedback loop between evidence accumulation and attention. Second, we examine models with a relative evidence decision rule and an absolute evidence rule. In the relative models a decision is made once the difference in evidence accumulated for 2 items reaches a threshold. In the absolute models, a decision is made once 1 item accumulates a certain amount of evidence, independently of how much is accumulated for a competitor. Our core result is simple-the existence of the late onset gaze bias to the option ultimately chosen, together with a positively skewed RT distribution means that the stopping rule must be relative not absolute. A large scale grid search of parameter space shows that absolute threshold models struggle to predict these phenomena even when incorporating evidence decay and assumptions of either mutual inhibition or feedforward inhibition.

Salient nutrition labels increase the integration of health attributes in food decision-making

Every day, people struggle to make healthy eating decisions. Nutrition labels have been used to help people properly balance the tradeoff between healthiness and taste, but research suggests that these labels vary in their effectiveness. Here, we investigated the cognitive mechanism underlying value-based decisions with nutrition labels as modulators of value.

More specifically, we used a binary decision task between products along with two different nutrition labels to examine how salient, color-coded labels, compared to purely information-based labels, alter the choice process. Using drift-diffusion modeling, we investigated whether color-coded labels alter the valuation process, or whether they induce a simple stimulus-response association consistent with the traffic-light colors irrespective of the features of the item, which would manifest in a starting point bias in the model. We show that color-coded labels significantly increased healthy choices by increasing the rate of preference formation (drift rate) towards healthier options without altering the starting point. Salient labels increased the sensitivity to health and decreased the weight on taste, indicating that the integration of health and taste attributes during the choice process is sensitive to how information is displayed. Salient labels proved to be more effective in altering the valuation process towards healthier, goal-directed decisions.

Optimal policy for value-based decision-making

For decades now, normative theories of perceptual decisions, and their implementation as drift diffusion models, have driven and significantly improved our understanding of human and animal behaviour and the underlying neural processes. While similar processes seem to govern value-based decisions, we still lack the theoretical understanding of why this ought to be the case. Here, we show that, similar to perceptual decisions, drift diffusion models implement the optimal strategy for value-based decisions. Such optimal decisions require the models' decision boundaries to collapse over time, and to depend on the a priori knowledge about reward contingencies. Diffusion models only implement the optimal strategy under specific task assumptions, and cease to be optimal once we start relaxing these assumptions, by, for example, using non-linear utility functions. Our findings thus provide the much-needed theory for value-based decisions, explain the apparent similarity to perceptual decisions, and predict conditions under which this similarity should break down.

Drift diffusion models (DDM) are fundamental to our understanding of perceptual decision-making. Here, the authors show that DDM can implement optimal choice strategies in value-based decisions but require sufficient knowledge of reward contingencies and collapsing decision boundaries with time.

Learning the opportunity cost of time in a patch-foraging task

Although most decision research concerns choice between simultaneously presented options, in many situations options are encountered serially, and the decision is whether to exploit an option or search for a better one. Such problems have a rich history in animal foraging, but we know little about the psychological processes involved. In particular, it is unknown whether learning in these problems is supported by the well-studied neurocomputational mechanisms involved in more conventional tasks. We investigated how humans learn in a foraging task, which requires deciding whether to harvest a depleting resource or switch to a replenished one. The optimal choice (given by the marginal value theorem; MVT) requires comparing the immediate return from harvesting to the opportunity cost of time, which is given by the long-run average reward. In two experiments, we varied opportunity cost across blocks, and subjects adjusted their behavior to blockwise changes in environmental characteristics. We examined how subjects learned their choice strategies by comparing choice adjustments to a learning rule suggested by the MVT (in which the opportunity cost threshold is estimated as an average over previous rewards) and to the predominant incremental-learning theory in neuroscience, temporal-difference learning (TD). Trial-by-trial decisions were explained better by the MVT threshold-learning rule. These findings expand on the foraging literature, which has focused on steady-state behavior, by elucidating a computational mechanism for learning in switching tasks that is distinct from those used in traditional tasks, and suggest connections to research on average reward rates in other domains of neuroscience.

Gaze data reveal distinct choice processes underlying model-based and model-free reinforcement learning

Organisms appear to learn and make decisions using different strategies known as model-free and model-based learning; the former is mere reinforcement of previously rewarded actions and the latter is a forward-looking strategy that involves evaluation of action-state transition probabilities. Prior work has used neural data to argue that both model-based and model-free learners implement a value comparison process at trial onset, but model-based learners assign more weight to forward-looking computations. Here using eye-tracking, we report evidence for a different interpretation of prior results: model-based subjects make their choices prior to trial onset. In contrast, model-free subjects tend to ignore model-based aspects of the task and instead seem to treat the decision problem as a simple comparison process between two differentially valued items, consistent with previous work on sequential-sampling models of decision making. These findings illustrate a problem with assuming that experimental subjects make their decisions at the same prescribed time.

Comparing fixed and collapsing boundary versions of the diffusion model

Optimality studies and studies of decision-making in monkeys have been used to support a model in which the decision boundaries used to evaluate evidence collapse over time. This article investigates whether a diffusion model with collapsing boundaries provides a better account of human data than a model with fixed boundaries. We compared the models using data from four new numerosity discrimination experiments and two previously published motion discrimination experiments. When model selection was based on BIC values, the fixed boundary model was preferred over the collapsing boundary model for all of the experiments. When model selection was carried out using a parametric bootstrap cross-fitting method (PBCM), which takes into account the flexibility of the alternative models and the ability of one model to account for data from another model, data from 5 of 6 experiments favored either fixed boundaries or boundaries with only negligible collapse. We found that the collapsing boundary model produces response times distributions with the same shape as those produced by the fixed boundary model and that its parameters were not well-identified and were difficult to recover from data. Furthermore, the estimated boundaries of the best-fitting collapsing boundary model were relatively flat and very similar to those of the fixed-boundary model. Overall, a diffusion model with decision boundaries that converge over time does not provide an improvement over the standard diffusion model for our tasks with human data.