Normalized value coding explains dynamic adaptation in the human valuation process

Irrational choices via a curvilinear representational geometry for value

We make decisions by comparing values, but it is not yet clear how value is represented in the brain. Many models assume, if only implicitly, that the representational geometry of value is linear. However, in part due to a historical focus on noisy single neurons, rather than neuronal populations, this hypothesis has not been rigorously tested. Here, we examine the representational geometry of value in the ventromedial prefrontal cortex (vmPFC), a part of the brain linked to economic decision-making, in two male rhesus macaques. We find that values are encoded along a curved manifold in vmPFC. This curvilinear geometry predicts a specific pattern of irrational decision-making: that decision-makers will make worse choices when an irrelevant, decoy option is worse in value, compared to when it is better. We observe this type of irrational choices in behavior. Together, these results not only suggest that the representational geometry of value is nonlinear, but that this nonlinearity could impose bounds on rational decision-making.

Beta and theta oscillations track effort and previous reward in the human basal ganglia and prefrontal cortex during decision making

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we report an exploratory investigation into this with chronic intracranial recordings from the prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12 to 20 Hz) oscillations tracking effort on a single-trial basis and PFC theta (4 to 7 Hz) signaling previous trial reward, with no effects of net subjective value. Stimulation of PFC increased overall acceptance of offers and sensitivity to reward while decreasing the impact of effort on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, supports a causal role of PFC for such choices, and seeds hypotheses for future studies.

Valuations of target items are drawn towards unavailable decoy items due to prior expectations

When people make choices, the items they consider are often embedded in a context (of other items). How this context affects the valuation of the specific item is an important question. High-value context might make items appear less attractive because of contrast-the tendency to normalize perception of an object relative to its background-or more attractive because of assimilation-the tendency to group objects together. Alternatively, a high-value context might increase prior expectations about the item's value. Here, we investigated these possibilities. We examined how unavailable context items affect choices between two target items, as well as the willingness-to-pay for single targets. Participants viewed sets of three items for several seconds before the target(s) were highlighted. In both tasks, we found a significant assimilation-like effect where participants were more likely to choose or place a higher value on a target when it was surrounded by higher-value context. However, these context effects were only significant for participants' fastest choices. Using variants of a drift-diffusion model, we established that the unavailable context shifted participants' prior expectations towards the average values of the sets but had an inconclusive effect on their evaluations of the targets during the decision (i.e. drift rates). In summary, we find that people use context to inform their initial valuations. This can improve efficiency by allowing people to get a head start on their decision. However, it also means that the valuation of an item can change depending on the context.

Curriculum learning inspired by behavioral shaping trains neural networks to adopt animal-like decision making strategies

Recurrent neural networks (RNN) are ubiquitously used in neuroscience to capture both neural dynamics and behaviors of living systems. However, when it comes to complex cognitive tasks, traditional methods for training RNNs can fall short in capturing crucial aspects of animal behavior. To address this challenge, we take inspiration from a commonly used (though rarely appreciated) approach from the experimental neuroscientist's toolkit: behavioral shaping. Our solution leverages task compositionality and models the animal's relevant learning experiences prior to the task. Taking as target a temporal wagering task previously studied in rats, we designed a pretraining curriculum of simpler cognitive tasks that are prerequisites for performing it well. These pretraining tasks are not just simplified versions of the temporal wagering task, but reflect relevant sub-computations. We show that this approach is required for RNNs to adopt similar strategies as rats, including long-timescale inference of latent states, which conventional pretraining approaches fail to capture. Mechanistically, our pretraining supports the development of key dynamical systems features needed for implementing both inference and value-based decision making. Overall, our approach addresses a gap in neural network model training by incorporating inductive biases of animals, which is important when modeling complex behaviors that rely on computational abilities acquired from past experiences.

Multiple timescales of temporal context in risky choice: Behavioral identification and relationships to physiological arousal

Context-dependence is fundamental to risky monetary decision-making. A growing body of evidence suggests that temporal context, or recent events, alters risk-taking at a minimum of three timescales: immediate (e.g. trial-by-trial), neighborhood (e.g. a group of consecutive trials), and global (e.g. task-level). To examine context effects, we created a novel monetary choice set with intentional temporal structure in which option values shifted between multiple levels of value magnitude ("contexts") several times over the course of the task. This structure allowed us to examine whether effects of each timescale were simultaneously present in risky choice behavior and the potential mechanistic role of arousal, an established correlate of risk-taking, in context-dependency. We found that risk-taking was sensitive to immediate, neighborhood, and global timescales: risk-taking decreased following large (vs. small) outcome amounts, increased following large positive (but not negative) shifts in context, and increased when cumulative earnings exceeded expectations. We quantified arousal with skin conductance responses, which were related to the global timescale, increasing with cumulative earnings, suggesting that physiological arousal captures a task-level assessment of performance. Our results both replicate and extend prior research by demonstrating that risky decision-making is consistently dynamic at multiple timescales and that the role of arousal in risk-taking extends to some, but not all timescales of context-dependence.

Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex

Evidence from monkeys and humans suggests that the orbitofrontal cortex (OFC) encodes the subjective value of options under consideration during choice. Data from non-human primates suggests that these value signals are context-dependent, representing subjective value in a way influenced by the decision makers' recent experience. Using electrodes distributed throughout cortical and subcortical structures, human epilepsy patients performed an auction task where they repeatedly reported the subjective values they placed on snack food items. High-gamma activity in many cortical and subcortical sites including the OFC positively correlated with subjective value. Other OFC sites showed signals contextually modulated by the subjective value of previously offered goods-a context dependency predicted by theory but not previously observed in humans. These results suggest that value and value-context signals are simultaneously present but separately represented in human frontal cortical activity.

Distinct value computations support rapid sequential decisions

The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or cache values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors.

Distinct value computations support rapid sequential decisions

The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or "cache" values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors.

Similar functional networks predict performance in both perceptual and value-based decision tasks

There are numerous commonalities between perceptual and preferential decision processes. For instance, previous studies have shown that both of these decision types are influenced by context. Also, the same computational models can explain both. However, the neural processes and functional connections that underlie these similarities between perceptual and value-based decisions are still unclear. Hence, in the current study, we examine whether perceptual and preferential processes can be explained by similar functional networks utilizing data from the Human Connectome Project. We used resting-state functional magnetic resonance imaging data to predict performance of 2 different decision-making tasks: a value-related task (the delay discounting task) and a perceptual task (the flanker task). We then examined the existence of shared predictive-network features across these 2 decision tasks. Interestingly, we found a significant positive correlation between the functional networks, which predicted the value-based and perceptual tasks. In addition, a larger functional connectivity between visual and frontal decision brain areas was a critical feature in the prediction of both tasks. These results demonstrate that functional connections between perceptual and value-related areas in the brain are inherently related to decision-making processes across domains.

Intraparietal stimulation disrupts negative distractor effects in human multi-alternative decision-making

There has been debate about whether addition of an irrelevant distractor option to an otherwise binary decision influences which of the two choices is taken. We show that disparate views on this question are reconciled if distractors exert two opposing but not mutually exclusive effects. Each effect predominates in a different part of decision space: (1) a positive distractor effect predicts high-value distractors improve decision-making; (2) a negative distractor effect, of the type associated with divisive normalisation models, entails decreased accuracy with increased distractor values. Here, we demonstrate both distractor effects coexist in human decision making but in different parts of a decision space defined by the choice values. We show disruption of the medial intraparietal area (MIP) by transcranial magnetic stimulation (TMS) increases positive distractor effects at the expense of negative distractor effects. Furthermore, individuals with larger MIP volumes are also less susceptible to the disruption induced by TMS. These findings also demonstrate a causal link between MIP and the impact of distractors on decision-making via divisive normalisation.

The composition of the choice set modulates probability weighting in risky decisions

Probability distortion-the tendency to underweight larger probabilities and overweight smaller ones-is a robust empirical phenomenon and an important driver of suboptimal choices. We reveal a novel contextual effect on probability distortion that depends on the composition of the choice set. Probability distortion was larger in a magnitude-diverse choice set (in which participants encountered more unique magnitudes than probabilities) but declined, resulting in more veridical weighting, in a probability-diverse choice set (more unique probabilities than magnitudes). This effect was consistent in two, large, independent datasets (N = 481, N = 100) and held for a subset of lotteries that were identical in the two contexts. It also developed gradually as a function of exposure to the choice set, was independent of attentional biases to probability versus magnitude information, and was specific to probability weighting, leaving risk attitudes unaffected. The results highlight the importance of context when processing probabilistic information.

Training diversity promotes absolute-value-guided choice

Many decision-making studies have demonstrated that humans learn either expected values or relative preferences among choice options, yet little is known about what environmental conditions promote one strategy over the other. Here, we test the novel hypothesis that humans adapt the degree to which they form absolute values to the diversity of the learning environment. Since absolute values generalize better to new sets of options, we predicted that the more options a person learns about the more likely they would be to form absolute values. To test this, we designed a multi-day learning experiment comprising twenty learning sessions in which subjects chose among pairs of images each associated with a different probability of reward. We assessed the degree to which subjects formed absolute values and relative preferences by asking them to choose between images they learned about in separate sessions. We found that concurrently learning about more images within a session enhanced absolute-value, and suppressed relative-preference, learning. Conversely, cumulatively pitting each image against a larger number of other images across multiple sessions did not impact the form of learning. These results show that the way humans encode preferences is adapted to the diversity of experiences offered by the immediate learning context.

Human value learning and representation reflect rational adaptation to task demands

Humans and other animals routinely make choices between goods of different values. Choices are often made within identifiable contexts, such that an efficient learner may represent values relative to their local context. However, if goods occur across multiple contexts, a relative value code can lead to irrational choices. In this case, an absolute context-independent value is preferable to a relative code. Here we test the hypothesis that value representation is not fixed but rationally adapted to context expectations. In two experiments, we manipulated participants' expectations about whether item values learned within local contexts would need to be subsequently compared across contexts. Despite identical learning experiences, the group whose expectations included choices across local contexts went on to learn more absolute-like representation than the group whose expectations covered only fixed local contexts. Human value representation is thus neither relative nor absolute but efficiently and rationally tuned to task demands.

Asymmetric and adaptive reward coding via normalized reinforcement learning

Learning is widely modeled in psychology, neuroscience, and computer science by prediction error-guided reinforcement learning (RL) algorithms. While standard RL assumes linear reward functions, reward-related neural activity is a saturating, nonlinear function of reward; however, the computational and behavioral implications of nonlinear RL are unknown. Here, we show that nonlinear RL incorporating the canonical divisive normalization computation introduces an intrinsic and tunable asymmetry in prediction error coding. At the behavioral level, this asymmetry explains empirical variability in risk preferences typically attributed to asymmetric learning rates. At the neural level, diversity in asymmetries provides a computational mechanism for recently proposed theories of distributional RL, allowing the brain to learn the full probability distribution of future rewards. This behavioral and computational flexibility argues for an incorporation of biologically valid value functions in computational models of learning and decision-making.

Reinforcement learning models are widely used to characterize reward-driven learning in biological and computational agents. Standard reinforcement learning models use linear value functions, despite strong empirical evidence that biological value representations are nonlinear functions of external rewards. Here, we examine the properties of a biologically-based nonlinear reinforcement learning algorithm employing the canonical divisive normalization function, a neural computation commonly found in sensory, cognitive, and reward coding. We show that this normalized reinforcement learning algorithm implements a simple but powerful control of how reward learning reflects relative gains and losses. This property explains diverse behavioral and neural phenomena, and suggests the importance of using biologically valid value functions in computational models of learning and decision-making.

Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges

In the past decade, decision neuroscience and neuroeconomics have developed many new insights in the study of decision making. This review provides an overarching update on how the field has advanced in this time period. Although our initial review a decade ago outlined several theoretical, conceptual, methodological, empirical, and practical challenges, there has only been limited progress in resolving these challenges. We summarize significant trends in decision neuroscience through the lens of the challenges outlined for the field and review examples where the field has had significant, direct, and applicable impacts across economics and psychology. First, we review progress on topics including reward learning, explore-exploit decisions, risk and ambiguity, intertemporal choice, and valuation. Next, we assess the impacts of emotion, social rewards, and social context on decision making. Then, we follow up with how individual differences impact choices and new exciting developments in the prediction and neuroforecasting of future decisions. Finally, we consider how trends in decision-neuroscience research reflect progress toward resolving past challenges, discuss new and exciting applications of recent research, and identify new challenges for the field. This article is categorized under: Psychology > Reasoning and Decision Making Psychology > Emotion and Motivation.

Efficiently irrational: deciphering the riddle of human choice

For the past half-century, cognitive and social scientists have struggled with the irrationalities of human choice behavior; people consistently make choices that are logically inconsistent. Is human choice behavior evolutionarily adaptive or is it an inefficient patchwork of competing mechanisms? In this review, I present an interdisciplinary synthesis arguing for a novel interpretation: choice is efficiently irrational. Connecting findings across disciplines suggests that observed choice behavior reflects a precise optimization of the trade-off between the costs of increasing the precision of the choice mechanism and the declining benefits that come as precision increases. Under these constraints, a rationally imprecise strategy emerges that works toward optimal efficiency rather than toward optimal rationality. This approach rationalizes many of the puzzling inconsistencies of human choice behavior, explaining why these inconsistencies arise as an optimizing solution in biological choosers.

High-value decisions are fast and accurate, inconsistent with diminishing value sensitivity

It is a widely held belief that people's choices are less sensitive to changes in value as value increases. For example, the subjective difference between $11 and $12 is believed to be smaller than between $1 and $2. This idea is consistent with applications of the Weber-Fechner Law and divisive normalization to value-based choice and with psychological interpretations of diminishing marginal utility. According to random utility theory in economics, smaller subjective differences predict less accurate choices. Meanwhile, in the context of sequential sampling models in psychology, smaller subjective differences also predict longer response times. Based on these models, we would predict decisions between high-value options to be slower and less accurate. In contrast, some have argued on normative grounds that choices between high-value options should be made with less caution, leading to faster and less accurate choices. Here, we model the dynamics of the choice process across three different choice domains, accounting for both discriminability and response caution. Contrary to predictions, we mostly observe faster and more accurate decisions (i.e., higher drift rates) between high-value options. We also observe that when participants are alerted about incoming high-value decisions, they exert more caution and not less. We rule out several explanations for these results, using tasks with both subjective and objective values. These results cast doubt on the notion that increasing value reduces discriminability.

Advances in modeling learning and decision-making in neuroscience

An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.

Rational regulation of water-seeking effort in rodents

We confirm that rats can act as rational economic agents, making choices about how much work to do to obtain a reward in a way that optimally trades off the value of the reward against the cost of the effort. Contrary to the notion that bigger rewards are more motivating, rats worked harder in economies where rewards were small, ensuring a sufficient minimum income of water. But they chose to earn and consume more water per day when water was “cheap” (available for little work). We present a mathematical model explaining why rats work when they do (surprisingly, not just when they are thirsty) and suggesting where in the brain animals might compute the current value of working for water.

In the laboratory, animals’ motivation to work tends to be positively correlated with reward magnitude. But in nature, rewards earned by work are essential to survival (e.g., working to find water), and the payoff of that work can vary on long timescales (e.g., seasonally). Under these constraints, the strategy of working less when rewards are small could be fatal. We found that instead, rats in a closed economy did more work for water rewards when the rewards were stably smaller, a phenomenon also observed in human labor supply curves. Like human consumers, rats showed elasticity of demand, consuming far more water per day when its price in effort was lower. The neural mechanisms underlying such “rational” market behaviors remain largely unexplored. We propose a dynamic utility maximization model that can account for the dependence of rat labor supply (trials/day) on the wage rate (milliliter/trial) and also predict the temporal dynamics of when rats work. Based on data from mice, we hypothesize that glutamatergic neurons in the subfornical organ in lamina terminalis continuously compute the instantaneous marginal utility of voluntary work for water reward and causally determine the amount and timing of work.

The aversive value of pain in human decision-making

BACKGROUND

In order to decide between avoiding pain or pursuing competing rewards, pain must be assigned an abstract value that can be traded against that of competing goods. To assess the relationship between subjectively perceived pain and its value, we conducted an experiment where participants had to accept or decline different intensities of painful electric shocks in exchange of monetary rewards.

METHODS

Participants (n = 90) were divided into three groups that were exposed to different distributions of monetary rewards. Monetary offers ranged linearly from $0 to $5 or $10 in groups 1 and 2, respectively, and exponentially from $0 to $5 in group 3. Pain offers ranged from pain detection to pain tolerance thresholds. The value of pain was assessed by identifying the indifference points corresponding to a 50% chance of accepting a certain level of pain for a given monetary reward.

RESULTS

The value of pain increased quadratically as a function of the anticipated pain intensity and was found to be relative to the mean and standard deviation of monetary offers. Moreover, decision times increased as a function of the intensity of accepted painful stimulations. Finally, inter-individual differences in psychological traits related to harm avoidance and persistence influenced the value of pain.

CONCLUSIONS

This is the first demonstration that the value of pain follows a curvilinear function and is relative to the mean and standard deviation of competing monetary rewards. These new observations significantly contribute to our understanding of how pain is assigned value when making decisions between avoiding pain and obtaining rewards.

SIGNIFICANCE

This work provides a description of the pain value function indicating how much people are willing to pay to avoid different intensities of pain. We found that the function was curvilinear, suggesting that the same unit of subjective pain has more value in the high vs. low pain range. Moreover, the pain value was influenced by the experimental manipulation of the rewards distribution and of the inter-individual differences in harm avoidance and persistence. Altogether, the present study provides a detailed account of how subjectively experienced pain is assigned value.

Punishments and rewards both modestly impair visuomotor memory retention

While the effects of rewards on memory appear well documented, the effects of punishments remain uncertain. Based on neuroimaging data, this study tested the hypothesis that, as compared to a neutral condition, a context allowing successful punishment avoidance would enhance memory to a similar extent as rewards. In a fully within-subject and counter-balanced design, participants (n = 18) took part in 3 distinct learning sessions during which the delivery of performance-contingent monetary punishments and rewards was manipulated. Specifically, participants had to reach towards visual targets while compensating for a gradually introduced visual deviation. Accuracy at achieving targets was either punished (Hit: "+0$"; Miss: "-0.5$), rewarded (Hit: "+0.5$"; Miss: "-0$"), or associated with neutral binary feedback (Hit: "Hit"; Miss: "Miss"). Retention was assessed through reach aftereffects both immediately and 24 h after initial acquisition. The results disconfirmed the hypothesis by showing that the punishment and reward learning sessions both impaired retention as compared to the neutral session, suggesting that both types of incentives similarly impaired memory formation and consolidation. Two alternative but complementary interpretations are discussed. One interpretation is that the presence of punishments and rewards induced a negative learning context, which - based on neurobiological data - could have been sufficient to interfere with memory formation and consolidation. Another interpretation is that punishments and rewards emphasized the disrupting effects of target hits on implicit learning processes, therefore yielding retention impairments. Altogether, these results suggest that incentives can have counter-productive effects on memory.

Optimal policy for attention-modulated decisions explains human fixation behavior

Traditional accumulation-to-bound decision-making models assume that all choice options are processed with equal attention. In real life decisions, however, humans alternate their visual fixation between individual items to efficiently gather relevant information (Yang et al., 2016). These fixations also causally affect one's choices, biasing them toward the longer-fixated item (Krajbich et al., 2010). We derive a normative decision-making model in which attention enhances the reliability of information, consistent with neurophysiological findings (Cohen and Maunsell, 2009). Furthermore, our model actively controls fixation changes to optimize information gathering. We show that the optimal model reproduces fixation-related choice biases seen in humans and provides a Bayesian computational rationale for this phenomenon. This insight led to additional predictions that we could confirm in human data. Finally, by varying the relative cognitive advantage conferred by attention, we show that decision performance is benefited by a balanced spread of resources between the attended and unattended items.

Excitatory-inhibitory tone shapes decision strategies in a hierarchical neural network model of multi-attribute choice

We are constantly faced with decisions between alternatives defined by multiple attributes, necessitating an evaluation and integration of different information sources. Time-varying signals in multiple brain areas are implicated in decision-making; but we lack a rigorous biophysical description of how basic circuit properties, such as excitatory-inhibitory (E/I) tone and cascading nonlinearities, shape attribute processing and choice behavior. Furthermore, how such properties govern choice performance under varying levels of environmental uncertainty is unknown. We investigated two-attribute, two-alternative decision-making in a dynamical, cascading nonlinear neural network with three layers: an input layer encoding choice alternative attribute values; an intermediate layer of modules processing separate attributes; and a final layer producing the decision. Depending on intermediate layer E/I tone, the network displays distinct regimes characterized by linear (I), convex (II) or concave (III) choice indifference curves. In regimes I and II, each option's attribute information is additively integrated. In regime III, time-varying nonlinear operations amplify the separation between offer distributions by selectively attending to the attribute with the larger differences in input values. At low environmental uncertainty, a linear combination most consistently selects higher valued alternatives. However, at high environmental uncertainty, regime III is more likely than a linear operation to select alternatives with higher value. Furthermore, there are conditions where readout from the intermediate layer could be experimentally indistinguishable from the final layer. Finally, these principles are used to examine multi-attribute decisions in systems with reduced inhibitory tone, leading to predictions of different choice patterns and overall performance between those with restrictions on inhibitory tone and neurotypicals.

Attraction to similar options: The Gestalt law of proximity is related to the attraction effect

Previous studies have suggested that there are common mechanisms between perceptual and value-based processes. For instance, both perceptual and value-based choices are highly influenced by the context in which the choices are made. However, the mechanisms which allow context to influence our choice process as well as the extent of the similarity between the perceptual and preferential processes are still unclear. In this study, we examine a within-subject relation between the attraction effect, which is a well-known effect of context on preferential choice, and the Gestalt law of proximity. Then, we aim to use this link to better understand the mechanisms underlying the attraction effect. We conducted one study followed by an additional pre-registered replication study, where subjects performed a Gestalt-psychophysical task and a decoy task. Comparing the behavioral sensitivity of each subject in both tasks, we found that the more susceptible a subject is to the proximity law, the more she displayed the attraction effect. These results demonstrate a within-subject relation between a perceptual phenomenon (proximity law) and a value-based bias (attraction effect) which further strengthens the notion of common rules between perceptual and value-based processing. Moreover, this suggests that the mechanism underlying the attraction effect is related to grouping by proximity with attention as a mediator.

Quantifying the immediate computational effects of preceding outcomes on subsequent risky choices

Forty years ago, prospect theory introduced the notion that risky options are evaluated relative to their recent context, causing a significant shift in the study of risky monetary decision-making in psychology, economics, and neuroscience. Despite the central role of past experiences, it remains unclear whether, how, and how much past experiences quantitatively influence risky monetary choices moment-to-moment in a nominally learning-free setting. We analyzed a large dataset of risky monetary choices with trial-by-trial feedback to quantify how past experiences, or recent events, influence risky choice behavior and the underlying processes. We found larger recent outcomes both negatively influence subsequent risk-taking and positively influence the weight put on potential losses. Using a hierarchical Bayesian framework to fit a modified version of prospect theory, we demonstrated that the same risks will be evaluated differently given different past experiences. The computations underlying risky decision-making are fundamentally dynamic, even if the environment is not.

Value-based attention but not divisive normalization influences decisions with multiple alternatives

Violations of economic rationality principles in choices between three or more options are critical for understanding the neural and cognitive mechanisms of decision-making. A recent study reported that the relative choice accuracy between two options decreases as the value of a third (distractor) option increases and attributed this effect to divisive normalization of neural value representations. In two preregistered experiments, a direct replication and an eye-tracking experiment, we assessed the replicability of this effect and tested an alternative account that assumes value-based attention to mediate the distractor effect. Surprisingly, we could not replicate the distractor effect in our experiments. However, we found a dynamic influence of distractor value on fixations to distractors as predicted by the value-based attention theory. Computationally, we show that extending an established sequential sampling decision-making model by a value-based attention mechanism offers a comprehensive account of the interplay between value, attention, response times and decisions.

Temporal contrast effects in human speech perception are immune to selective attention

Two fundamental properties of perception are selective attention and perceptual contrast, but how these two processes interact remains unknown. Does an attended stimulus history exert a larger contrastive influence on the perception of a following target than unattended stimuli? Dutch listeners categorized target sounds with a reduced prefix "ge-" marking tense (e.g., ambiguous between gegaan-gaan "gone-go"). In 'single talker' Experiments 1-2, participants perceived the reduced syllable (reporting gegaan) when the target was heard after a fast sentence, but not after a slow sentence (reporting gaan). In 'selective attention' Experiments 3-5, participants listened to two simultaneous sentences from two different talkers, followed by the same target sounds, with instructions to attend only one of the two talkers. Critically, the speech rates of attended and unattended talkers were found to equally influence target perception - even when participants could watch the attended talker speak. In fact, participants' target perception in 'selective attention' Experiments 3-5 did not differ from participants who were explicitly instructed to divide their attention equally across the two talkers (Experiment 6). This suggests that contrast effects of speech rate are immune to selective attention, largely operating prior to attentional stream segregation in the auditory processing hierarchy.

Context effects on probability estimation

Many decisions rely on how we evaluate potential outcomes and estimate their corresponding probabilities of occurrence. Outcome evaluation is subjective because it requires consulting internal preferences and is sensitive to context. In contrast, probability estimation requires extracting statistics from the environment and therefore imposes unique challenges to the decision maker. Here, we show that probability estimation, like outcome evaluation, is subject to context effects that bias probability estimates away from other events present in the same context. However, unlike valuation, these context effects appeared to be scaled by estimated uncertainty, which is largest at intermediate probabilities. Blood-oxygen-level-dependent (BOLD) imaging showed that patterns of multivoxel activity in the dorsal anterior cingulate cortex (dACC), ventromedial prefrontal cortex (VMPFC), and intraparietal sulcus (IPS) predicted individual differences in context effects on probability estimates. These results establish VMPFC as the neurocomputational substrate shared between valuation and probability estimation and highlight the additional involvement of dACC and IPS that can be uniquely attributed to probability estimation. Because probability estimation is a required component of computational accounts from sensory inference to higher cognition, the context effects found here may affect a wide array of cognitive computations.

This study shows how probability estimation can be affected by the context of our recent experience, namely, how the presence of multiple events experienced closed in time can influence their respective probability estimates.

Goal congruency dominates reward value in accounting for behavioral and neural correlates of value-based decision-making

When choosing between options, whether menu items or career paths, we can evaluate how rewarding each one will be, or how congruent it is with our current choice goal (e.g., to point out the best option or the worst one.). Past decision-making research interpreted findings through the former lens, but in these experiments the most rewarding option was always most congruent with the task goal (choosing the best option). It is therefore unclear to what extent expected reward vs. goal congruency can account for choice value findings. To deconfound these two variables, we performed three behavioral studies and an fMRI study in which the task goal varied between identifying the best vs. the worst option. Contrary to prevailing accounts, we find that goal congruency dominates choice behavior and neural activity. We separately identify dissociable signals of expected reward. Our findings call for a reinterpretation of previous research on value-based choice.

Decision-making research has confounded the reward value of options with their goal-congruency, as the task goal was always to pick the most rewarding option. Here, authors separately asked participants to select the least rewarding of a set of options, revealing a dominant role for goal congruency.

Choice-theoretic foundations of the divisive normalization model

Recent advances in neuroscience suggest that a utility-like calculation is involved in how the brain makes choices, and that this calculation may use a computation known as divisive normalization. While this tells us how the brain makes choices, it is not immediately evident why the brain uses this computation or exactly what behavior is consistent with it. In this paper, we address both of these questions by proving a three-way equivalence theorem between the normalization model, an information-processing model, and an axiomatic characterization. The information-processing model views behavior as optimally balancing the expected value of the chosen object against the entropic cost of reducing stochasticity in choice. This provides an optimality rationale for why the brain may have evolved to use normalization-type models. The axiomatic characterization gives a set of testable behavioral statements equivalent to the normalization model. This answers what behavior arises from normalization. Our equivalence result unifies these three models into a single theory that answers the "how", "why", and "what" of choice behavior.

Rewards interact with repetition-dependent learning to enhance long-term retention of motor memories

The combination of behavioral experiences that enhance long-term retention remains largely unknown. Informed by neurophysiological lines of work, this study tested the hypothesis that performance-contingent monetary rewards potentiate repetition-dependent forms of learning, as induced by extensive practice at asymptote, to enhance long-term retention of motor memories. To this end, six groups of 14 participants (n = 84) acquired novel motor behaviors by adapting to a gradual visuomotor rotation while these factors were manipulated. Retention was assessed 24 h later. While all groups similarly acquired the novel motor behaviors, results from the retention session revealed an interaction indicating that rewards enhanced long-term retention, but only when practice was extended to asymptote. Specifically, the interaction indicated that this effect selectively occurred when rewards were intermittently available (i.e., 50%), but not when they were absent (i.e., 0%) or continuously available (i.e., 100%) during acquisition. This suggests that the influence of rewards on extensive practice and long-term retention is nonlinear, as continuous rewards did not further enhance retention as compared with intermittent rewards. One possibility is that rewards' intermittent availability allows to maintain their subjective value during acquisition, which may be key to potentiate long-term retention.

An Analysis of Decision under Risk in Rats

In 1979, Daniel Kahneman and Amos Tversky published a ground-breaking paper titled "Prospect Theory: An Analysis of Decision under Risk," which presented a behavioral economic theory that accounted for the ways in which humans deviate from economists' normative workhorse model, Expected Utility Theory [1, 2]. For example, people exhibit probability distortion (they overweight low probabilities), loss aversion (losses loom larger than gains), and reference dependence (outcomes are evaluated as gains or losses relative to an internal reference point). We found that rats exhibited many of these same biases, using a task in which rats chose between guaranteed and probabilistic rewards. However, prospect theory assumes stable preferences in the absence of learning, an assumption at odds with alternative frameworks such as animal learning theory and reinforcement learning [3-7]. Rats also exhibited trial history effects, consistent with ongoing learning. A reinforcement learning model in which state-action values were updated by the subjective value of outcomes according to prospect theory reproduced rats' nonlinear utility and probability weighting functions and also captured trial-by-trial learning dynamics.

Behavioral Paradigms to Probe Individual Mouse Differences in Value-Based Decision Making

Value-based decision making relies on distributed neural systems that weigh the benefits of actions against the cost required to obtain a given outcome. Perturbations of these systems are thought to underlie abnormalities in action selection seen across many neuropsychiatric disorders. Genetic tools in mice provide a promising opportunity to explore the cellular components of these systems and their molecular foundations. However, few tasks have been designed that robustly characterize how individual mice integrate differential reward benefits and cost in their selection of actions. Here we present a forced-choice, two-alternative task in which each option is associated with a specific reward outcome, and unique operant contingency. We employed global and individual trial measures to assess the choice patterns and behavioral flexibility of mice in response to differing "choice benefits" (modeled as varying reward magnitude ratios) and different modalities of "choice cost" (modeled as either increasing repetitive motor output to obtain reward or increased delay to reward delivery). We demonstrate that (1) mouse choice is highly sensitive to the relative benefit of outcomes; (2) choice costs are heavily discounted in environments with large discrepancies in relative reward; (3) divergent cost modalities are differentially integrated into action selection; (4) individual mouse sensitivity to reward benefit is correlated with sensitivity to reward costs. These paradigms reveal stable individual animal differences in value-based action selection, thereby providing a foundation for interrogating the neural circuit and molecular pathophysiology of goal-directed dysfunction.

Time preferences are reliable across time-horizons and verbal versus experiential tasks

Individual differences in delay-discounting correlate with important real world outcomes, for example education, income, drug use, and criminality. As such, delay-discounting has been extensively studied by economists, psychologists and neuroscientists to reveal its behavioral and biological mechanisms in both human and non-human animal models. However, two major methodological differences hinder comparing results across species. Human studies present long time-horizon options verbally, whereas animal studies employ experiential cues and short delays. To bridge these divides, we developed a novel language-free experiential task inspired by animal decision-making studies. We found that the ranks of subjects' time-preferences were reliable across both verbal/experiential and second/day differences. Yet, discount factors scaled dramatically across the tasks, indicating a strong effect of temporal context. Taken together, this indicates that individuals have a stable, but context-dependent, time-preference that can be reliably assessed using different methods, providing a foundation to bridge studies of time-preferences across species.

Editorial note

This article has been through an editorial process in which the authors decide how to respond to the issues raised during peer review. The Reviewing Editor's assessment is that all the issues have been addressed (see decision letter).

Efficient coding of subjective value

Preference-based decisions are essential for survival, for instance when deciding what we should (not) eat. Despite their importance, preference-based decisions are surprisingly variable and can appear irrational in ways that have defied mechanistic explanations. Here we propose that subjective valuation results from an inference process that accounts for the structure of values in the environment and that maximizes information in value representations in line with demands imposed by limited coding resources. A model of this inference process explains the variability in both subjective value reports and preference-based choices, and predicts a new preference illusion that we validate with empirical data. Interestingly, the same model explains the level of confidence associated with these reports. Our results imply that preference-based decisions reflect information-maximizing transmission and statistically optimal decoding of subjective values by a limited-capacity system. These findings provide a unified account of how humans perceive and valuate the environment to optimally guide behavior.

The evil of banality: When choosing between the mundane feels like choosing between the worst

Our most important decisions often provoke the greatest anxiety, whether we seek the better of two prizes or the lesser of two evils. Yet many of our choices are more mundane, such as selecting from a slate of mediocre but acceptable restaurants. Previous research suggests that choices of decreasing value should provoke decreasing anxiety. Here we show that this is not the case. Across three behavioral studies and one fMRI study, we find that anxiety and its neural correlates demonstrate a U-shaped function of choice set value, greatest when choosing between both the highest value and lowest value sets. Intermediate (moderate-value) choice sets provoke the least anxiety, even when they are just as difficult to select between as the choice sets at the two extremes. We show that these counterintuitive findings are accounted for by decision makers perceiving low-value items as aversive (i.e., negatively motivationally salient) rather than simply unrewarding. Importantly, though, neural signatures of these anxious reactions only appear when participants are required to choose one item from a set and not when simply appraising that set's overall value. Decision makers thus experience anxiety from competing avoidance motivations when forced to select among low-value options, comparable to the competing approach motivations they experience when choosing between high-value items. We further show that a common method of measuring subjective values (willingness to pay) can inadvertently censor a portion of this quadratic pattern, creating the misperception that anxiety simply increases linearly with set value. Collectively, these findings reveal the surprising costs of seemingly banal decisions. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Context Effects in the Judgment of Visual Relative-Frequency: Trial-by-Trial Adaptation and Non-linear Sequential Effect

Humans' judgment of relative-frequency, similar to their use of probability in decision-making, is often distorted as an inverted-S-shape curve-small relative-frequency overestimated and large relative-frequency underestimated. Here we investigated how the judgment of relative-frequency, despite its natural reference points (0 and 1) and stereotyped distortion, may adapt to the environmental statistics. The task was to report the relative-frequency of black (or white) dots in a visual array of black and white dots. We found that participants' judgment was distorted in the typical inverted-S-shape, but the distortion curve was influenced by both the central tendency and spread of the distribution of objective relative-frequencies: the lower the central tendency, the higher the overall judgment (contrast effect); the higher the spread, the more curved the inverted-S-shape (curvature effect). These context effects are in the spirit of efficient coding but opposite to what would be predicted by Bayesian inference. We further modeled the context effects on the level of individual trials, through which we found not only a trial-by-trial adaptation, but also the non-linear sequential effects that were recently reported mainly in circularly distributed visual stimuli.

Testosterone, Cortisol and Financial Risk-Taking

Both testosterone and cortisol have major actions on financial decision-making closely related to their primary biological functions, reproductive success and response to stress, respectively. Financial risk-taking represents a particular example of strategic decisions made in the context of choice under conditions of uncertainty. Such decisions have multiple components, and this article considers how much we know of how either hormone affects risk-appetite, reward value, information processing and estimation of the costs and benefits of potential success or failure, both personal and social. It also considers how far we can map these actions on neural mechanisms underlying risk appetite and decision-making, with particular reference to areas of the brain concerned in either cognitive or emotional functions.

Local Choices: Rationality and the Contextuality of Decision-Making

Rational explanation is ubiquitous in psychology and social sciences, ranging from rational analysis, expectancy-value theories, ideal observer models, mental logic to probabilistic frameworks, rational choice theory, and informal "folk psychological" explanation. However, rational explanation appears to be challenged by apparently systematic irrationality observed in psychological experiments, especially in the field of judgement and decision-making (JDM). Here, it is proposed that the experimental results require not that rational explanation should be rejected, but that rational explanation is local, i.e., within a context. Thus, rational models need to be supplemented with a theory of contextual shifts. We review evidence in JDM that patterns of choices are often consistent within contexts, but unstable between contexts. We also demonstrate that for a limited, though reasonably broad, class of decision-making domains, recent theoretical models can be viewed as providing theories of contextual shifts. It is argued that one particular significant source of global inconsistency arises from a cognitive inability to represent absolute magnitudes, whether for perceptual variables, utilities, payoffs, or probabilities. This overall argument provides a fresh perspective on the scope and limits of human rationality.