Regulation of evidence accumulation by pupil-linked arousal processes

Decomposing loss aversion from a single neural signal

People often display stronger aversion to losses than appetite for equivalent gains, a widespread phenomenon known as loss aversion. The prevailing theory attributes loss aversion to a valuation bias that amplifies losses relative to gains. An alternative account attributes loss aversion to a response bias that avoids choices that might result in loss. By modeling the temporal dynamics of scalp electrical activity during decisions to accept or reject gambles within a sequential sampling framework, we decomposed valuation bias and response bias from a single event-related neural signal, the P3. Specifically, we found valuation bias manifested as larger sensitivity of P3 to losses than gains, which was localizable to reward-related brain regions. By contrast, response bias manifested as larger P3 preceding gamble acceptance than rejection and was localizable to motor cortex. Our study reveals the dissociable neural biomarkers of response bias and valuation bias underpinning loss-averse decisions.

Quantifying decision-making in dynamic, continuously evolving environments

During perceptual decision-making tasks, centroparietal electroencephalographic (EEG) potentials report an evidence accumulation-to-bound process that is time locked to trial onset. However, decisions in real-world environments are rarely confined to discrete trials; they instead unfold continuously, with accumulation of time-varying evidence being recency-weighted towards its immediate past. The neural mechanisms supporting recency-weighted continuous decision-making remain unclear. Here, we use a novel continuous task design to study how the centroparietal positivity (CPP) adapts to different environments that place different constraints on evidence accumulation. We show that adaptations in evidence weighting to these different environments are reflected in changes in the CPP. The CPP becomes more sensitive to fluctuations in sensory evidence when large shifts in evidence are less frequent, and the potential is primarily sensitive to fluctuations in decision-relevant (not decision-irrelevant) sensory input. A complementary triphasic component over occipito-parietal cortex encodes the sum of recently accumulated sensory evidence, and its magnitude covaries with parameters describing how different individuals integrate sensory evidence over time. A computational model based on leaky evidence accumulation suggests that these findings can be accounted for by a shift in decision threshold between different environments, which is also reflected in the magnitude of pre-decision EEG activity. Our findings reveal how adaptations in EEG responses reflect flexibility in evidence accumulation to the statistics of dynamic sensory environments.

Rat movements reflect internal decision dynamics in an evidence accumulation task

Perceptual decision-making involves multiple cognitive processes, including accumulation of sensory evidence, planning, and executing a motor action. How these processes are intertwined is unclear; some models assume that decision-related processes precede motor execution, whereas others propose that movements reflecting on-going decision processes occur before commitment to a choice. Here we develop and apply two complementary methods to study the relationship between decision processes and the movements leading up to a choice. The first is a free response pulse-based evidence accumulation task, in which stimuli continue until choice is reported. The second is a motion-based drift diffusion model (mDDM), in which movement variables from video pose estimation constrain decision parameters on a trial-by-trial basis. We find the mDDM provides a better model fit to rats' decisions in the free response accumulation task than traditional DDM models. Interestingly, on each trial we observed a period of time, prior to choice, that was characterized by head immobility. The length of this period was positively correlated with the rats' decision bounds and stimuli presented during this period had the greatest impact on choice. Together these results support a model in which internal decision dynamics are reflected in movements and demonstrate that inclusion of movement parameters improves the performance of diffusion-to-bound decision models.

Pupillary dynamics reflect the impact of temporal expectation on detection strategy

Everyday life's perceptual decision-making is informed by experience. In particular, temporal expectation can ease the detection of relevant events in noisy sensory streams. Here, we investigated if humans can extract hidden temporal cues from the occurrences of probabilistic targets and utilize them to inform target detection in a complex acoustic stream. To understand what neural mechanisms implement temporal expectation influence on decision-making, we used pupillometry as a proxy for underlying neuromodulatory activity. We found that participants' detection strategy was influenced by the hidden temporal context and correlated with sound-evoked pupil dilation. A model of urgency fitted on false alarms predicted detection reaction time. Altogether, these findings suggest that temporal expectation informs decision-making and could be implemented through neuromodulatory-mediated urgency signals.

Assessing evidence accumulation and rule learning in humans with an online game

Evidence accumulation, an essential component of perception and decision making, is frequently studied with psychophysical tasks involving noisy or ambiguous stimuli. In these tasks, participants typically receive verbal or written instructions that describe the strategy that should be used to guide decisions. Although convenient and effective, explicit instructions can influence learning and decision making strategies and can limit comparisons with animal models, in which behaviors are reinforced through feedback. Here, we developed an online video game and nonverbal training pipeline, inspired by pulse-based tasks for rodents, as an alternative to traditional psychophysical tasks used to study evidence accumulation. Using this game, we collected behavioral data from hundreds of participants trained with an explicit description of the decision rule or with experiential feedback. Participants trained with feedback alone learned the game rules rapidly and used strategies and displayed biases similar to those who received explicit instructions. Finally, by leveraging data across hundreds of participants, we show that perceptual judgments were well described by an accumulation process in which noise scaled nonlinearly with evidence, consistent with previous animal studies but inconsistent with diffusion models widely used to describe perceptual decisions in humans. These results challenge the conventional description of the accumulation process and suggest that online games provide a valuable platform to examine perceptual decision making and learning in humans. In addition, the feedback-based training pipeline developed for this game may be useful for evaluating perceptual decision making in human populations with difficulty following verbal instructions.NEW & NOTEWORTHY Perceptual uncertainty sets critical constraints on our ability to accumulate evidence and make decisions; however, its sources remain unclear. We developed a video game, and feedback-based training pipeline, to study uncertainty during decision making. Leveraging choices from hundreds of subjects, we demonstrate that human choices are inconsistent with popular diffusion models of human decision making and instead are best fit by models in which perceptual uncertainty scales nonlinearly with the strength of sensory evidence.

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals. Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust the ways they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.

Dynamic decision policy reconfiguration under outcome uncertainty

In uncertain or unstable environments, sometimes the best decision is to change your mind. To shed light on this flexibility, we evaluated how the underlying decision policy adapts when the most rewarding action changes. Human participants performed a dynamic two-armed bandit task that manipulated the certainty in relative reward (conflict) and the reliability of action-outcomes (volatility). Continuous estimates of conflict and volatility contributed to shifts in exploratory states by changing both the rate of evidence accumulation (drift rate) and the amount of evidence needed to make a decision (boundary height), respectively. At the trialwise level, following a switch in the optimal choice, the drift rate plummets and the boundary height weakly spikes, leading to a slow exploratory state. We find that the drift rate drives most of this response, with an unreliable contribution of boundary height across experiments. Surprisingly, we find no evidence that pupillary responses associated with decision policy changes. We conclude that humans show a stereotypical shift in their decision policies in response to environmental changes.

Subjective optimality in finite sequential decision-making

Many decisions in life are sequential and constrained by a time window. Although mathematically derived optimal solutions exist, it has been reported that humans often deviate from making optimal choices. Here, we used a secretary problem, a classic example of finite sequential decision-making, and investigated the mechanisms underlying individuals' suboptimal choices. Across three independent experiments, we found that a dynamic programming model comprising subjective value function explains individuals' deviations from optimality and predicts the choice behaviors under fewer and more opportunities. We further identified that pupil dilation reflected the levels of decision difficulty and subsequent choices to accept or reject the stimulus at each opportunity. The value sensitivity, a model-based estimate that characterizes each individual's subjective valuation, correlated with the extent to which individuals' physiological responses tracked stimuli information. Our results provide model-based and physiological evidence for subjective valuation in finite sequential decision-making, rediscovering human suboptimality in subjectively optimal decision-making processes.

Flexible categorization in perceptual decision making

Perceptual decisions rely on accumulating sensory evidence. This computation has been studied using either drift diffusion models or neurobiological network models exhibiting winner-take-all attractor dynamics. Although both models can account for a large amount of data, it remains unclear whether their dynamics are qualitatively equivalent. Here we show that in the attractor model, but not in the drift diffusion model, an increase in the stimulus fluctuations or the stimulus duration promotes transitions between decision states. The increase in the number of transitions leads to a crossover between weighting mostly early evidence (primacy) to weighting late evidence (recency), a prediction we validate with psychophysical data. Between these two limiting cases, we found a novel flexible categorization regime, in which fluctuations can reverse initially-incorrect categorizations. This reversal asymmetry results in a non-monotonic psychometric curve, a distinctive feature of the attractor model. Our findings point to correcting decision reversals as an important feature of perceptual decision making.

Phasic Activation of Dorsal Raphe Serotonergic Neurons Increases Pupil Size

Transient variations in pupil size (PS) under constant luminance are coupled to rapid changes in arousal state,1-3 which have been interpreted as vigilance,4 salience,5 or a surprise signal.6-8 Neural control of such fluctuations presumably involves multiple brain regions5,9-11 and neuromodulatory systems,3,12,13 but it is often associated with phasic activity of the noradrenergic system.9,12,14,15 Serotonin (5-HT), a neuromodulator also implicated in aspects of arousal16 such as sleep-wake transitions,17 motivational state regulation,18 and signaling of unexpected events,19 seems to affect PS,20-24 but these effects have not been investigated in detail. Here we show that phasic 5-HT neuron stimulation causes transient PS changes. We used optogenetic activation of 5-HT neurons in the dorsal raphe nucleus (DRN) of head-fixed mice performing a foraging task. 5-HT-driven modulations of PS were maintained throughout the photostimulation period and sustained for a few seconds after the end of stimulation. We found no evidence that the increase in PS with activation of 5-HT neurons resulted from interactions of photostimulation with behavioral variables, such as locomotion or licking. Furthermore, we observed that the effect of 5-HT on PS depended on the level of environmental uncertainty, consistent with the idea that 5-HT could report a surprise signal.19 These results advance our understanding of the neuromodulatory control of PS, revealing a tight relationship between phasic activation of 5-HT neurons and changes in PS.

Efficient sampling and noisy decisions

Human decisions are based on finite information, which makes them inherently imprecise. But what determines the degree of such imprecision? Here, we develop an efficient coding framework for higher-level cognitive processes in which information is represented by a finite number of discrete samples. We characterize the sampling process that maximizes perceptual accuracy or fitness under the often-adopted assumption that full adaptation to an environmental distribution is possible, and show how the optimal process differs when detailed information about the current contextual distribution is costly. We tested this theory on a numerosity discrimination task, and found that humans efficiently adapt to contextual distributions, but in the way predicted by the model in which people must economize on environmental information. Thus, understanding decision behavior requires that we account for biological restrictions on information coding, challenging the often-adopted assumption of precise prior knowledge in higher-level decision systems.

Path integration in large-scale space and with novel geometries: Comparing vector addition and encoding-error models

Path integration is thought to rely on vestibular and proprioceptive cues yet most studies in humans involve primarily visual input, providing limited insight into their respective contributions. We developed a paradigm involving walking in an omnidirectional treadmill in which participants were guided on two sides of a triangle and then found their back way to origin. In Experiment 1, we tested a range of different triangle types while keeping the distance of the unguided side constant to determine the influence of spatial geometry. Participants overshot the angle they needed to turn and undershot the distance they needed to walk, with no consistent effect of triangle type. In Experiment 2, we manipulated distance while keeping angle constant to determine how path integration operated over both shorter and longer distances. Participants underestimated the distance they needed to walk to the origin, with error increasing as a function of the walked distance. To attempt to account for our findings, we developed configural-based computational models involving vector addition, the second of which included terms for the influence of past trials on the current one. We compared against a previously developed configural model of human path integration, the Encoding-Error model. We found that the vector addition models captured the tendency of participants to under-encode guided sides of the triangles and an influence of past trials on current trials. Together, our findings expand our understanding of body-based contributions to human path integration, further suggesting the value of vector addition models in understanding these important components of human navigation.

How do we remember where we have been? One important mechanism for doing so is called path integration, which refers to the computation of one’s position in space with only self-motion cues. By tracking the direction and distance we have walked, we can create a mental arrow from the current location to the origin, termed the homing vector. Previous studies have shown that the homing vector is subject to systematic distortions depending on previously experienced paths, yet what influences these patterns of errors, particularly in humans, remains uncertain. In this study, we compare two models of path integration based on participants walking two sides of a triangle without vision and then completing the third side based on their estimate of the homing vector. We found no effect of triangle shape on systematic errors, while the systematic errors scaled with path length logarithmically, similar to Weber-Fechner law. While we show that both models captured participants’ behavior, a model based on vector addition best captured the patterns of error in the homing vector. Our study therefore has important implications for how humans track their location, suggesting that vector-based models provide a reasonable and simple explanation for how we do so.

A divisive model of evidence accumulation explains uneven weighting of evidence over time

Divisive normalization has long been used to account for computations in various neural processes and behaviours. The model proposes that inputs into a neural system are divisively normalized by the system’s total activity. More recently, dynamical versions of divisive normalization have been shown to account for how neural activity evolves over time in value-based decision making. Despite its ubiquity, divisive normalization has not been studied in decisions that require evidence to be integrated over time. Such decisions are important when the information is not all available at once. A key feature of such decisions is how evidence is weighted over time, known as the integration kernel. Here, we provide a formal expression for the integration kernel in divisive normalization, and show that divisive normalization quantitatively accounts for 133 human participants’ perceptual decision making behaviour, performing as well as the state-of-the-art Drift Diffusion Model, the predominant model for perceptual evidence accumulation.

Divisive normalization is thought to be a ubiquitous computation in the brain, but has not been studied in decisions that require integrating evidence over time. Here, the authors show in humans that dynamic divisive normalization accounts for the uneven weighting of perceptual evidence over time.