Rats adopt the optimal timescale for evidence integration in a dynamic environment

Brain-wide dynamics linking sensation to action during decision-making

Perceptual decisions rely on learned associations between sensory evidence and appropriate actions, involving the filtering and integration of relevant inputs to prepare and execute timely responses1,2. Despite the distributed nature of task-relevant representations3-10, it remains unclear how transformations between sensory input, evidence integration, motor planning and execution are orchestrated across brain areas and dimensions of neural activity. Here we addressed this question by recording brain-wide neural activity in mice learning to report changes in ambiguous visual input. After learning, evidence integration emerged across most brain areas in sparse neural populations that drive movement-preparatory activity. Visual responses evolved from transient activations in sensory areas to sustained representations in frontal-motor cortex, thalamus, basal ganglia, midbrain and cerebellum, enabling parallel evidence accumulation. In areas that accumulate evidence, shared population activity patterns encode visual evidence and movement preparation, distinct from movement-execution dynamics. Activity in movement-preparatory subspace is driven by neurons integrating evidence, which collapses at movement onset, allowing the integration process to reset. Across premotor regions, evidence-integration timescales were independent of intrinsic regional dynamics, and thus depended on task experience. In summary, learning aligns evidence accumulation to action preparation in activity dynamics across dozens of brain regions. This leads to highly distributed and parallelized sensorimotor transformations during decision-making. Our work unifies concepts from decision-making and motor control fields into a brain-wide framework for understanding how sensory evidence controls actions.

Trial-history biases in evidence accumulation can give rise to apparent lapses in decision-making

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, previous work has suggested that they covary in their prevalence and that their proposed neural substrates overlap. Here we demonstrate that during decision-making, history biases and apparent lapses can both arise from a common cognitive process that is optimal under mistaken beliefs that the world is changing i.e. nonstationary. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct decision-making datasets of male rats, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Just a phase? Causal probing reveals spurious phasic dependence of sustained attention

For over a decade, electrophysiological studies have reported correlations between attention / perception and the phase of spontaneous brain oscillations. To date, these findings have been interpreted as evidence that the brain uses neural oscillations to sample and predict upcoming stimuli. Yet, evidence from simulations have shown that analysis artefacts could also lead to spurious pre-stimulus oscillations that appear to predict future brain responses. To address this discrepancy, we conducted an experiment in which visual stimuli were presented in time to specific phases of spontaneous alpha and theta oscillations. This allowed us to causally probe the role of ongoing neural activity in visual processing independent of the stimulus-evoked dynamics. Our findings did not support a causal link between spontaneous alpha / theta rhythms and behaviour. However, spurious correlations between theta phase and behaviour emerged offline using gold-standard time-frequency analyses. These findings are a reminder that care should be taken when inferring causal relationships between neural activity and behaviour using acausal analysis methods.

Quantifying decision-making in dynamic, continuously evolving environments

During perceptual decision-making tasks, centroparietal electroencephalographic (EEG) potentials report an evidence accumulation-to-bound process that is time locked to trial onset. However, decisions in real-world environments are rarely confined to discrete trials; they instead unfold continuously, with accumulation of time-varying evidence being recency-weighted towards its immediate past. The neural mechanisms supporting recency-weighted continuous decision-making remain unclear. Here, we use a novel continuous task design to study how the centroparietal positivity (CPP) adapts to different environments that place different constraints on evidence accumulation. We show that adaptations in evidence weighting to these different environments are reflected in changes in the CPP. The CPP becomes more sensitive to fluctuations in sensory evidence when large shifts in evidence are less frequent, and the potential is primarily sensitive to fluctuations in decision-relevant (not decision-irrelevant) sensory input. A complementary triphasic component over occipito-parietal cortex encodes the sum of recently accumulated sensory evidence, and its magnitude covaries with parameters describing how different individuals integrate sensory evidence over time. A computational model based on leaky evidence accumulation suggests that these findings can be accounted for by a shift in decision threshold between different environments, which is also reflected in the magnitude of pre-decision EEG activity. Our findings reveal how adaptations in EEG responses reflect flexibility in evidence accumulation to the statistics of dynamic sensory environments.

Rat movements reflect internal decision dynamics in an evidence accumulation task

Perceptual decision-making involves multiple cognitive processes, including accumulation of sensory evidence, planning, and executing a motor action. How these processes are intertwined is unclear; some models assume that decision-related processes precede motor execution, whereas others propose that movements reflecting on-going decision processes occur before commitment to a choice. Here we develop and apply two complementary methods to study the relationship between decision processes and the movements leading up to a choice. The first is a free response pulse-based evidence accumulation task, in which stimuli continue until choice is reported. The second is a motion-based drift diffusion model (mDDM), in which movement variables from video pose estimation constrain decision parameters on a trial-by-trial basis. We find the mDDM provides a better model fit to rats' decisions in the free response accumulation task than traditional DDM models. Interestingly, on each trial we observed a period of time, prior to choice, that was characterized by head immobility. The length of this period was positively correlated with the rats' decision bounds and stimuli presented during this period had the greatest impact on choice. Together these results support a model in which internal decision dynamics are reflected in movements and demonstrate that inclusion of movement parameters improves the performance of diffusion-to-bound decision models.

Bayesian inference in ring attractor networks

Working memories are thought to be held in attractor networks in the brain. These attractors should keep track of the uncertainty associated with each memory, so as to weigh it properly against conflicting new evidence. However, conventional attractors do not represent uncertainty. Here, we show how uncertainty could be incorporated into an attractor, specifically a ring attractor that encodes head direction. First, we introduce a rigorous normative framework (the circular Kalman filter) for benchmarking the performance of a ring attractor under conditions of uncertainty. Next, we show that the recurrent connections within a conventional ring attractor can be retuned to match this benchmark. This allows the amplitude of network activity to grow in response to confirmatory evidence, while shrinking in response to poor-quality or strongly conflicting evidence. This "Bayesian ring attractor" performs near-optimal angular path integration and evidence accumulation. Indeed, we show that a Bayesian ring attractor is consistently more accurate than a conventional ring attractor. Moreover, near-optimal performance can be achieved without exact tuning of the network connections. Finally, we use large-scale connectome data to show that the network can achieve near-optimal performance even after we incorporate biological constraints. Our work demonstrates how attractors can implement a dynamic Bayesian inference algorithm in a biologically plausible manner, and it makes testable predictions with direct relevance to the head direction system as well as any neural system that tracks direction, orientation, or periodic rhythms.

Flexible sensory-motor mapping rules manifest in correlated variability of stimulus and action codes across the brain

Humans and non-human primates can flexibly switch between different arbitrary mappings from sensation to action to solve a cognitive task. It has remained unknown how the brain implements such flexible sensory-motor mapping rules. Here, we uncovered a dynamic reconfiguration of task-specific correlated variability between sensory and motor brain regions. Human participants switched between two rules for reporting visual orientation judgments during fMRI recordings. Rule switches were either signaled explicitly or inferred by the participants from ambiguous cues. We used behavioral modeling to reconstruct the time course of their belief about the active rule. In both contexts, the patterns of correlations between ongoing fluctuations in stimulus- and action-selective activity across visual- and action-related brain regions tracked participants' belief about the active rule. The rule-specific correlation patterns broke down around the time of behavioral errors. We conclude that internal beliefs about task state are instantiated in brain-wide, selective patterns of correlated variability.

Trial-history biases in evidence accumulation can give rise to apparent lapses

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, several hints in the literature suggest that they covary in their prevalence and that their proposed neural substrates overlap - what could underlie these links? Here we demonstrate that history biases and apparent lapses can both arise from a common cognitive process that is normative under misbeliefs about non-stationarity in the world. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct rat decision-making datasets, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Stable choice coding in rat frontal orienting fields across model-predicted changes of mind

During decision making in a changing environment, evidence that may guide the decision accumulates until the point of action. In the rat, provisional choice is thought to be represented in frontal orienting fields (FOF), but this has only been tested in static environments where provisional and final decisions are not easily dissociated. Here, we characterize the representation of accumulated evidence in the FOF of rats performing a recently developed dynamic evidence accumulation task, which induces changes in the provisional decision, referred to as “changes of mind”. We find that FOF encodes evidence throughout decision formation with a temporal gain modulation that rises until the period when the animal may need to act. Furthermore, reversals in FOF firing rates can be accounted for by changes of mind predicted using a model of the decision process fit only to behavioral data. Our results suggest that the FOF represents provisional decisions even in dynamic, uncertain environments, allowing for rapid motor execution when it is time to act.

A leaky accumulation model can predict rats’ changes of mind during decision making in a dynamic environment explaining reversals in frontal cortical activity and demonstrating a stable choice code despite environmental uncertainty.

Dynamics of Visual Perceptual Decision-Making in Freely Behaving Mice

The temporal dynamics of perceptual decisions offer a key window into the cognitive processes contributing to decision-making. Investigating perceptual dynamics in a genetically tractable animal model can facilitate the subsequent unpacking of the underlying neural mechanisms. Here, we investigated the time course as well as fundamental psychophysical constants governing visual perceptual decision-making in freely behaving mice. We did so by analyzing response accuracy against reaction time (RT), i.e., conditional accuracy, in a series of two-alternative forced choice (2-AFC) orientation discrimination tasks in which we varied target size, luminance, duration, and presence of a foil. Our results quantified two distinct stages in the time course of mouse visual decision-making: a "sensory encoding" stage in which conditional accuracy exhibits a classic trade-off with response speed, and a subsequent "short-term memory (STM)-dependent" stage in which conditional accuracy exhibits a classic asymptotic decay following stimulus offset. We estimated the duration of visual sensory encoding as 200-320 ms across tasks, the lower bound of the duration of STM as ∼1700 ms, and the briefest duration of visual stimulus input that is informative as ≤50 ms. Separately, by varying stimulus onset delay, we demonstrated that the conditional accuracy function (CAF) and RT distribution can be independently modulated, and found that the duration for which mice naturally withhold from responding is a quantitative metric of impulsivity. Taken together, our results establish a quantitative foundation for investigating the neural circuit bases of visual decision dynamics in mice.

Dynamics of Visual Perceptual Decision-Making in Freely Behaving Mice

The temporal dynamics of perceptual decisions offer a key window into the cognitive processes contributing to decision-making. Investigating perceptual dynamics in a genetically tractable animal model can facilitate the subsequent unpacking of the underlying neural mechanisms. Here, we investigated the time course as well as fundamental psychophysical constants governing visual perceptual decision-making in freely behaving mice. We did so by analyzing response accuracy against reaction time (RT), i.e., conditional accuracy, in a series of two-alternative forced choice (2-AFC) orientation discrimination tasks in which we varied target size, luminance, duration, and presence of a foil. Our results quantified two distinct stages in the time course of mouse visual decision-making: a “sensory encoding” stage in which conditional accuracy exhibits a classic trade-off with response speed, and a subsequent “short-term memory (STM)-dependent” stage in which conditional accuracy exhibits a classic asymptotic decay following stimulus offset. We estimated the duration of visual sensory encoding as 200–320 ms across tasks, the lower bound of the duration of STM as ∼1700 ms, and the briefest duration of visual stimulus input that is informative as ≤50 ms. Separately, by varying stimulus onset delay, we demonstrated that the conditional accuracy function (CAF) and RT distribution can be independently modulated, and found that the duration for which mice naturally withhold from responding is a quantitative metric of impulsivity. Taken together, our results establish a quantitative foundation for investigating the neural circuit bases of visual decision dynamics in mice.

A confirmation bias in perceptual decision-making due to hierarchical approximate inference

Making good decisions requires updating beliefs according to new evidence. This is a dynamical process that is prone to biases: in some cases, beliefs become entrenched and resistant to new evidence (leading to primacy effects), while in other cases, beliefs fade over time and rely primarily on later evidence (leading to recency effects). How and why either type of bias dominates in a given context is an important open question. Here, we study this question in classic perceptual decision-making tasks, where, puzzlingly, previous empirical studies differ in the kinds of biases they observe, ranging from primacy to recency, despite seemingly equivalent tasks. We present a new model, based on hierarchical approximate inference and derived from normative principles, that not only explains both primacy and recency effects in existing studies, but also predicts how the type of bias should depend on the statistics of stimuli in a given task. We verify this prediction in a novel visual discrimination task with human observers, finding that each observer's temporal bias changed as the result of changing the key stimulus statistics identified by our model. The key dynamic that leads to a primacy bias in our model is an overweighting of new sensory information that agrees with the observer's existing belief-a type of 'confirmation bias'. By fitting an extended drift-diffusion model to our data we rule out an alternative explanation for primacy effects due to bounded integration. Taken together, our results resolve a major discrepancy among existing perceptual decision-making studies, and suggest that a key source of bias in human decision-making is approximate hierarchical inference.

The importance of urgency in decision making based on dynamic information

A standard view in the literature is that decisions are the result of a process that accumulates evidence in favor of each alternative until such accumulation reaches a threshold and a decision is made. However, this view has been recently questioned by an alternative proposal that suggests that, instead of accumulated, evidence is combined with an urgency signal. Both theories have been mathematically formalized and supported by a variety of decision-making tasks with constant information. However, recently, tasks with changing information have shown to be more effective to study the dynamics of decision making. Recent research using one of such tasks, the tokens task, has shown that decisions are better described by an urgency mechanism than by an accumulation one. However, the results of that study could depend on a task where all fundamental information was noiseless and always present, favoring a mechanism of non-integration, such as the urgency one. Here, we wanted to address whether the same conclusions were also supported by an experimental paradigm in which sensory evidence was removed shortly after it was provided, making working memory necessary to properly perform the task. Here, we show that, under such condition, participants' behavior could be explained by an urgency-gating mechanism that low-pass filters the mnemonic information and combines it with an urgency signal that grows with time but not by an accumulation process that integrates the same mnemonic information. Thus, our study supports the idea that, under certain situations with dynamic sensory information, decisions are better explained by an urgency-gating mechanism than by an accumulation one.

Adaptive circuit dynamics across human cortex during evidence accumulation in changing environments

Many decisions under uncertainty entail the temporal accumulation of evidence that informs about the state of the environment. When environments are subject to hidden changes in their state, maximizing accuracy and reward requires non-linear accumulation of evidence. How this adaptive, non-linear computation is realized in the brain is unknown. We analyzed human behavior and cortical population activity (measured with magnetoencephalography) recorded during visual evidence accumulation in a changing environment. Behavior and decision-related activity in cortical regions involved in action planning exhibited hallmarks of adaptive evidence accumulation, which could also be implemented by a recurrent cortical microcircuit. Decision dynamics in action-encoding parietal and frontal regions were mirrored in a frequency-specific modulation of the state of the visual cortex that depended on pupil-linked arousal and the expected probability of change. These findings link normative decision computations to recurrent cortical circuit dynamics and highlight the adaptive nature of decision-related feedback to the sensory cortex.

Change point detection with multiple alternatives reveals parallel evaluation of the same stream of evidence along distinct timescales

In order to behave appropriately in a rapidly changing world, individuals must be able to detect when changes occur in that environment. However, at any given moment, there are a multitude of potential changes of behavioral significance that could occur. Here we investigate how knowledge about the space of possible changes affects human change point detection. We used a stochastic auditory change point detection task that allowed model-free and model-based characterization of the decision process people employ. We found that subjects can simultaneously apply distinct timescales of evidence evaluation to the same stream of evidence when there are multiple types of changes possible. Informative cues that specified the nature of the change led to improved accuracy for change point detection through mechanisms involving both the timescales of evidence evaluation and adjustments of decision bounds. These results establish three important capacities of information processing for decision making that any proposed neural mechanism of evidence evaluation must be able to support: the ability to simultaneously employ multiple timescales of evidence evaluation, the ability to rapidly adjust those timescales, and the ability to modify the amount of information required to make a decision in the context of flexible timescales.

Sex-specific speed-accuracy trade-offs shape neural processing of acoustic signals in a grasshopper

Speed-accuracy trade-offs-being fast at the risk of being wrong-are fundamental to many decisions and natural selection is expected to resolve these trade-offs according to the costs and benefits of behaviour. We here test the prediction that females and males should integrate information from courtship signals differently because they experience different pay-offs along the speed-accuracy continuum. We fitted a neural model of decision making (a drift-diffusion model of integration to threshold) to behavioural data from the grasshopper Chorthippus biguttulus to determine the parameters of temporal integration of acoustic directional information used by male grasshoppers to locate receptive females. The model revealed that males had a low threshold for initiating a turning response, yet a large integration time constant enabled them to continue to gather information when cues were weak. This contrasts with parameters estimated for females of the same species when evaluating potential mates, in which response thresholds were much higher and behaviour was strongly influenced by unattractive stimuli. Our results reveal differences in neural integration consistent with the sex-specific costs of mate search: males often face competition and need to be fast, while females often pay high error costs and need to be deliberate.

Extracting the dynamics of behavior in sensory decision-making experiments

Decision-making strategies evolve during training and can continue to vary even in well-trained animals. However, studies of sensory decision-making tend to characterize behavior in terms of a fixed psychometric function that is fit only after training is complete. Here, we present PsyTrack, a flexible method for inferring the trajectory of sensory decision-making strategies from choice data. We apply PsyTrack to training data from mice, rats, and human subjects learning to perform auditory and visual decision-making tasks. We show that it successfully captures trial-to-trial fluctuations in the weighting of sensory stimuli, bias, and task-irrelevant covariates such as choice and stimulus history. This analysis reveals dramatic differences in learning across mice and rapid adaptation to changes in task statistics. PsyTrack scales easily to large datasets and offers a powerful tool for quantifying time-varying behavior in a wide variety of animals and tasks.

A Diversity of Intrinsic Timescales Underlie Neural Computations

Neural processing occurs across a range of temporal scales. To facilitate this, the brain uses fast-changing representations reflecting momentary sensory input alongside more temporally extended representations, which integrate across both short and long temporal windows. The temporal flexibility of these representations allows animals to behave adaptively. Short temporal windows facilitate adaptive responding in dynamic environments, while longer temporal windows promote the gradual integration of information across time. In the cognitive and motor domains, the brain sets overarching goals to be achieved within a long temporal window, which must be broken down into sequences of actions and precise movement control processed across much shorter temporal windows. Previous human neuroimaging studies and large-scale artificial network models have ascribed different processing timescales to different cortical regions, linking this to each region's position in an anatomical hierarchy determined by patterns of inter-regional connectivity. However, even within cortical regions, there is variability in responses when studied with single-neuron electrophysiology. Here, we review a series of recent electrophysiology experiments that demonstrate the heterogeneity of temporal receptive fields at the level of single neurons within a cortical region. This heterogeneity appears functionally relevant for the computations that neurons perform during decision-making and working memory. We consider anatomical and biophysical mechanisms that may give rise to a heterogeneity of timescales, including recurrent connectivity, cortical layer distribution, and neurotransmitter receptor expression. Finally, we reflect on the computational relevance of each brain region possessing a heterogeneity of neuronal timescales. We argue that this architecture is of particular importance for sensory, motor, and cognitive computations.

Robust parallel decision-making in neural circuits with nonlinear inhibition

Animals frequently need to choose the best alternative from a set of possibilities, whether it is which direction to swim in or which food source to favor. How long should a network of neurons take to choose the best of N options? Theoretical results suggest that the optimal time grows as log(N), if the values of each option are imperfectly perceived. However, standard self-terminating neural network models of decision-making cannot achieve this optimal behavior. We show how using certain additional nonlinear response properties in neurons, which are ignored in standard models, results in a decision-making architecture that both achieves the optimal scaling of decision time and accounts for multiple experimentally observed features of neural decision-making.

An elemental computation in the brain is to identify the best in a set of options and report its value. It is required for inference, decision-making, optimization, action selection, consensus, and foraging. Neural computing is considered powerful because of its parallelism; however, it is unclear whether neurons can perform this max-finding operation in a way that improves upon the prohibitively slow optimal serial max-finding computation (which takes ∼N⁡log(N) time for N noisy candidate options) by a factor of N, the benchmark for parallel computation. Biologically plausible architectures for this task are winner-take-all (WTA) networks, where individual neurons inhibit each other so only those with the largest input remain active. We show that conventional WTA networks fail the parallelism benchmark and, worse, in the presence of noise, altogether fail to produce a winner when N is large. We introduce the nWTA network, in which neurons are equipped with a second nonlinearity that prevents weakly active neurons from contributing inhibition. Without parameter fine-tuning or rescaling as N varies, the nWTA network achieves the parallelism benchmark. The network reproduces experimentally observed phenomena like Hick’s law without needing an additional readout stage or adaptive N-dependent thresholds. Our work bridges scales by linking cellular nonlinearities to circuit-level decision-making, establishes that distributed computation saturating the parallelism benchmark is possible in networks of noisy, finite-memory neurons, and shows that Hick’s law may be a symptom of near-optimal parallel decision-making with noisy input.

Dynamics of odor sampling strategies in mice

Mammalian olfactory receptor neurons in the nasal cavity are stimulated by odorants carried by the inhaled air and their activation is therefore tied to and driven by the breathing or sniffing frequency. Sniffing frequency can be deliberately modulated to alter how odorants stimulate olfactory receptor neurons, giving the animal control over the frequency of odorant exposure to potentially aid odorant detection and discrimination. We monitored sniffing behaviors and odorant discrimination ability of freely-moving mice while they sampled either decreasing concentrations of target odorants or sampled a fixed target odorant concentration in the presence of a background of increasing odorant concentrations, using a Go-NoGo behavioral paradigm. This allowed us to ask how mice alter their odorant sampling duration and sampling (sniffing) frequency depending on the demands of the task and its difficulty. Mice showed an anticipatory increase in sniffing rate prior to odorant exposure and chose to sample for longer durations when exposed to odorants as compared to the solvent control odorant. Similarly, mice also took more odorant sampling sniffs when exposed to target odorants compared to the solvent control odorant. In general, odorant sampling strategies became more similar the more difficult the task was, e.g. the lower the target odorant concentration or the lower the target odorant contrast relative to the background odorant, suggesting that sniffing patterns are not preset, but are dynamically modulated by the particular task and its difficulty.

Timescales of Evidence Evaluation for Decision Making and Associated Confidence Judgments Are Adapted to Task Demands

Decision making often involves choosing actions based on relevant evidence. This can benefit from focussing evidence evaluation on the timescale of greatest relevance based on the situation. Here, we use an auditory change detection task to determine how people adjust their timescale of evidence evaluation depending on task demands for detecting changes in their environment and assessing their internal confidence in those decisions. We confirm previous results that people adopt shorter timescales of evidence evaluation for detecting changes in contexts with shorter signal durations, while bolstering those results with model-free analyses not previously used and extending the results to the auditory domain. We also extend these results to show that in contexts with shorter signal durations, people also adopt correspondingly shorter timescales of evidence evaluation for assessing confidence in their decision about detecting a change. These results provide important insights into adaptability and flexible control of evidence evaluation for decision making.

A divisive model of evidence accumulation explains uneven weighting of evidence over time

Divisive normalization has long been used to account for computations in various neural processes and behaviours. The model proposes that inputs into a neural system are divisively normalized by the system’s total activity. More recently, dynamical versions of divisive normalization have been shown to account for how neural activity evolves over time in value-based decision making. Despite its ubiquity, divisive normalization has not been studied in decisions that require evidence to be integrated over time. Such decisions are important when the information is not all available at once. A key feature of such decisions is how evidence is weighted over time, known as the integration kernel. Here, we provide a formal expression for the integration kernel in divisive normalization, and show that divisive normalization quantitatively accounts for 133 human participants’ perceptual decision making behaviour, performing as well as the state-of-the-art Drift Diffusion Model, the predominant model for perceptual evidence accumulation.

Divisive normalization is thought to be a ubiquitous computation in the brain, but has not been studied in decisions that require integrating evidence over time. Here, the authors show in humans that dynamic divisive normalization accounts for the uneven weighting of perceptual evidence over time.

Analyzing dynamic decision-making models using Chapman-Kolmogorov equations

Decision-making in dynamic environments typically requires adaptive evidence accumulation that weights new evidence more heavily than old observations. Recent experimental studies of dynamic decision tasks require subjects to make decisions for which the correct choice switches stochastically throughout a single trial. In such cases, an ideal observer's belief is described by an evolution equation that is doubly stochastic, reflecting stochasticity in the both observations and environmental changes. In these contexts, we show that the probability density of the belief can be represented using differential Chapman-Kolmogorov equations, allowing efficient computation of ensemble statistics. This allows us to reliably compare normative models to near-normative approximations using, as model performance metrics, decision response accuracy and Kullback-Leibler divergence of the belief distributions. Such belief distributions could be obtained empirically from subjects by asking them to report their decision confidence. We also study how response accuracy is affected by additional internal noise, showing optimality requires longer integration timescales as more noise is added. Lastly, we demonstrate that our method can be applied to tasks in which evidence arrives in a discrete, pulsatile fashion, rather than continuously.

Kalman-like Self-Tuned Sensitivity in Biophysical Sensing

Living organisms need to be sensitive to a changing environment while also ignoring uninformative environmental fluctuations. Here, we argue that living cells can navigate these conflicting demands by dynamically tuning their environmental sensitivity. We analyze the circadian clock in Synechococcus elongatus, showing that clock-metabolism coupling can detect mismatch between clock predictions and the day-night light cycle, temporarily raise the clock's sensitivity to light changes, and thus re-entraining faster. We find analogous behavior in recent experiments on switching between slow and fast osmotic-stress-response pathways in yeast. In both cases, cells can raise their sensitivity to new external information in epochs of frequent challenging stress, much like a Kalman filter with adaptive gain in signal processing. Our work suggests a new class of experiments that probe the history dependence of environmental sensitivity in biophysical sensing mechanisms.

Performance of normative and approximate evidence accumulation on the dynamic clicks task

The aim of a number of psychophysics tasks is to uncover how mammals make decisions in a world that is in flux. Here we examine the characteristics of ideal and near-ideal observers in a task of this type. We ask when and how performance depends on task parameters and design, and, in turn, what observer performance tells us about their decision-making process. In the dynamic clicks task subjects hear two streams (left and right) of Poisson clicks with different rates. Subjects are rewarded when they correctly identify the side with the higher rate, as this side switches unpredictably. We show that a reduced set of task parameters defines regions in parameter space in which optimal, but not near-optimal observers, maintain constant response accuracy. We also show that for a range of task parameters an approximate normative model must be finely tuned to reach near-optimal performance, illustrating a potential way to distinguish between normative models and their approximations. In addition, we show that using the negative log-likelihood and the 0/1-loss functions to fit these types of models is not equivalent: the 0/1-loss leads to a bias in parameter recovery that increases with sensory noise. These findings suggest ways to tease apart models that are hard to distinguish when tuned exactly, and point to general pitfalls in experimental design, model fitting, and interpretation of the resulting data.

Optimal models of decision-making in dynamic environments

Nature is in constant flux, so animals must account for changes in their environment when making decisions. How animals learn the timescale of such changes and adapt their decision strategies accordingly is not well understood. Recent psychophysical experiments have shown humans and other animals can achieve near-optimal performance at two alternative forced choice (2AFC) tasks in dynamically changing environments. Characterization of performance requires the derivation and analysis of computational models of optimal decision-making policies on such tasks. We review recent theoretical work in this area, and discuss how models compare with subjects' behavior in tasks where the correct choice or evidence quality changes in dynamic, but predictable, ways.

Foraging as an evidence accumulation process

The patch-leaving problem is a canonical foraging task, in which a forager must decide to leave a current resource in search for another. Theoretical work has derived optimal strategies for when to leave a patch, and experiments have tested for conditions where animals do or do not follow an optimal strategy. Nevertheless, models of patch-leaving decisions do not consider the imperfect and noisy sampling process through which an animal gathers information, and how this process is constrained by neurobiological mechanisms. In this theoretical study, we formulate an evidence accumulation model of patch-leaving decisions where the animal averages over noisy measurements to estimate the state of the current patch and the overall environment. We solve the model for conditions where foraging decisions are optimal and equivalent to the marginal value theorem, and perform simulations to analyze deviations from optimal when these conditions are not met. By adjusting the drift rate and decision threshold, the model can represent different “strategies”, for example an incremental, decremental, or counting strategy. These strategies yield identical decisions in the limiting case but differ in how patch residence times adapt when the foraging environment is uncertain. To describe sub-optimal decisions, we introduce an energy-dependent marginal utility function that predicts longer than optimal patch residence times when food is plentiful. Our model provides a quantitative connection between ecological models of foraging behavior and evidence accumulation models of decision making. Moreover, it provides a theoretical framework for potential experiments which seek to identify neural circuits underlying patch-leaving decisions.

Foraging is a ubiquitous animal behavior, performed by organisms as different as worms, birds, rats, and humans. Although the behavior has been extensively studied, it is not known how the brain processes information obtained during foraging activity to make subsequent foraging decisions. We form an evidence accumulation model of foraging decisions that describes the process through which an animal gathers information and uses it to make foraging decisions. By building on studies of the neural decision mechanisms within systems neuroscience, this model connects the foraging decision process with ecological models of patch-leaving decisions, such as the marginal value theorem. The model suggests the existence of different foraging strategies, which optimize for different environmental conditions and their potential implementation by neural decision making circuits. The model also shows how state-dependence, such as satiation level, can affect evidence accumulation to lead to sub-optimal foraging decisions. Our model provides a framework for future experimental studies which seek to elucidate how neural decision making mechanisms have been shaped by evolutionary forces in an animal’s surrounding environment.

Flexibility of Timescales of Evidence Evaluation for Decision Making

To understand the neural mechanisms that support decision making, it is critical to characterize the timescale of evidence evaluation. Recent work has shown that subjects can adaptively adjust the timescale of evidence evaluation across blocks of trials depending on context [1]. However, it's currently unknown if adjustments to evidence evaluation occur online during deliberations based on a single stream of evidence. To examine this question, we employed a change-detection task in which subjects report their level of confidence in judging whether there has been a change in a stochastic auditory stimulus. Using a combination of psychophysical reverse correlation analyses and single-trial behavioral modeling, we compared the time period over which sensory information has leverage on detection report choices versus confidence. We demonstrate that the length of this period differs on separate sets of trials based on what's being reported. Surprisingly, confidence judgments on trials with no detection report are influenced by evidence occurring earlier than the time period of influence for detection reports. Our findings call into question models of decision formation involving static parameters that yield a singular timescale of evidence evaluation and instead suggest that the brain represents and utilizes multiple timescales of evidence evaluation during deliberation.

How a well-adapting immune system remembers

The adaptive immune system is able to protect us from a large variety of pathogens, even ones it has not seen yet. Can predicting the future pathogen distribution help in protection? We find that a combination of probabilistic forecasting and occasional sampling of the current environment reduces infection costs—a scheme easily implemented by the memory repertoire. The proposed theoretical framework offers a modular recipe for updating the memory repertoire, which quantitatively predicts the strength of the immune response in flu-vaccination experiments, unlike other update schemes. It also links the observed early life dynamics of the memory pool to the sparseness properties of the pathogen distribution and competitive receptor dynamics for pathogens.

An adaptive agent predicting the future state of an environment must weigh trust in new observations against prior experiences. In this light, we propose a view of the adaptive immune system as a dynamic Bayesian machinery that updates its memory repertoire by balancing evidence from new pathogen encounters against past experience of infection to predict and prepare for future threats. This framework links the observed initial rapid increase of the memory pool early in life followed by a midlife plateau to the ease of learning salient features of sparse environments. We also derive a modulated memory pool update rule in agreement with current vaccine-response experiments. Our results suggest that pathogenic environments are sparse and that memory repertoires significantly decrease infection costs, even with moderate sampling. The predicted optimal update scheme maps onto commonly considered competitive dynamics for antigen receptors.