An Accumulation-of-Evidence Task Using Visual Pulses for Mice Navigating in Virtual Reality

A cross-species framework for investigating perceptual evidence accumulation

Cross-species studies are important for a comprehensive understanding of brain functions. However, direct quantitative comparison of behaviors across species presents a significant challenge. To enable such comparisons in perceptual decision-making, we developed a synchronized evidence accumulation task for human and non-human animals, by aligning mechanics, stimuli, and training. The task was readily learned by rats, mice and humans, with each species exhibiting qualitatively similar performance. Quantitative model comparison revealed that all three species employed an evidence accumulation strategy, but differed in speed, accuracy, and key decision parameters. Human performance prioritized accuracy, whereas rodent performance was limited by internal time-pressure. Rats optimized reward rate, while mice appeared to switch between evidence accumulation and other strategies trial-to-trial. Together, these results reveal striking similarities and species-specific priorities in decision-making. Furthermore, the synchronized behavioral framework we present may facilitate future studies involving cross-species comparisons, such as evaluating the face validity of animal models of neuropsychiatric disorders.

Highlights

Development of an evidence accumulation task for rats and miceSynchronized video game allows direct comparisons with humansRat, mouse and human behavior are well fit by the same decision modelsModel parameters reveal species-specific priorities in accumulation strategy.

Pre-existing visual responses in a projection-defined dopamine population explain individual learning trajectories

A key challenge of learning a new task is that the environment is high dimensional-there are many different sensory features and possible actions, with typically only a small reward-relevant subset. Although animals can learn to perform complex tasks that involve arbitrary associations between stimuli, actions, and rewards,1,2,3,4,5,6 a consistent and striking result across varied experimental paradigms is that in initially acquiring such tasks, large differences between individuals are apparent in the learning process.7,8,9,10,11,12 What neural mechanisms contribute to initial task acquisition, and why do some individuals learn a new task much more quickly than others? To address these questions, we recorded longitudinally from dopaminergic (DA) axon terminals in mice learning a visual decision-making task.7 Across striatum, DA responses tracked idiosyncratic and side-specific learning trajectories, consistent with widespread reward prediction error coding across DA terminals. However, even before any rewards were delivered, contralateral-side-specific visual responses were present in DA terminals, primarily in the dorsomedial striatum (DMS). These pre-existing responses predicted the extent of learning for contralateral stimuli. Moreover, activation of these terminals improved contralateral performance. Thus, the initial conditions of a projection-specific and feature-specific DA signal help explain individual learning trajectories. More broadly, this work suggests that functional heterogeneity across DA projections may serve to bias target regions toward learning about different subsets of task features, providing a potential mechanism to address the dimensionality of the initial task learning problem.

Rat movements reflect internal decision dynamics in an evidence accumulation task

Perceptual decision-making involves multiple cognitive processes, including accumulation of sensory evidence, planning, and executing a motor action. How these processes are intertwined is unclear; some models assume that decision-related processes precede motor execution, whereas others propose that movements reflecting ongoing decision processes occur before commitment to a choice. Here we combine two complementary methods to study the relationship between decision processes and the movements leading up to a choice. The first is a free-response pulse-based evidence accumulation task, in which stimuli continue until choice is reported, and the second is a motion-based drift diffusion model (mDDM), in which movement variables from video pose estimation constrain decision parameters on a trial-by-trial basis. We find that the mDDM provides a better fit to rats' decisions in the free-response accumulation task than traditional drift diffusion models. Interestingly, on each trial we observed a period, before choice, that was characterized by head immobility. The length of this period was positively correlated with the rats' decision bounds, and stimuli presented during this period had the greatest impact on choice. Together these results support a model in which internal decision dynamics are reflected in movements and demonstrate that inclusion of movement parameters improves the performance of diffusion-to-bound decision models.NEW & NOTEWORTHY In this study we combine a novel pulse-based evidence accumulation task with a newly developed motion-based drift diffusion model (mDDM). In this model, we incorporate movement parameters derived from high-resolution video data to estimate parameters of the model on a trial-by-trial basis. We find that this new model is an improved description of animal choice behavior.

Brain-wide dynamics linking sensation to action during decision-making

Perceptual decisions rely on learned associations between sensory evidence and appropriate actions, involving the filtering and integration of relevant inputs to prepare and execute timely responses1,2. Despite the distributed nature of task-relevant representations3-10, it remains unclear how transformations between sensory input, evidence integration, motor planning and execution are orchestrated across brain areas and dimensions of neural activity. Here we addressed this question by recording brain-wide neural activity in mice learning to report changes in ambiguous visual input. After learning, evidence integration emerged across most brain areas in sparse neural populations that drive movement-preparatory activity. Visual responses evolved from transient activations in sensory areas to sustained representations in frontal-motor cortex, thalamus, basal ganglia, midbrain and cerebellum, enabling parallel evidence accumulation. In areas that accumulate evidence, shared population activity patterns encode visual evidence and movement preparation, distinct from movement-execution dynamics. Activity in movement-preparatory subspace is driven by neurons integrating evidence, which collapses at movement onset, allowing the integration process to reset. Across premotor regions, evidence-integration timescales were independent of intrinsic regional dynamics, and thus depended on task experience. In summary, learning aligns evidence accumulation to action preparation in activity dynamics across dozens of brain regions. This leads to highly distributed and parallelized sensorimotor transformations during decision-making. Our work unifies concepts from decision-making and motor control fields into a brain-wide framework for understanding how sensory evidence controls actions.

Spatiotemporal dynamics of locomotor decisions in Drosophila melanogaster

Decision-making in animals often involves choosing actions while navigating the environment, a process markedly different from static decision paradigms commonly studied in laboratory settings. Even in decision-making assays in which animals can freely locomote, decision outcomes are often interpreted as happening at single points in space and single moments in time, a simplification that potentially glosses over important spatiotemporal dynamics. We investigated locomotor decision-making in Drosophila melanogaster in Y-shaped mazes, measuring the extent to which their future choices could be predicted through space and time. We demonstrate that turn-decisions can be reliably predicted from flies' locomotor dynamics, with distinct predictability phases emerging as flies progress through maze regions. We show that these predictability dynamics are not merely the result of maze geometry or wall-following tendencies, but instead reflect the capacity of flies to move in ways that depend on sustained locomotor signatures, suggesting an active, working memory-like process. Additionally, we demonstrate that fly mutants known to have sensory and information-processing deficits exhibit altered spatial predictability patterns, highlighting the role of visual, mechanosensory, and dopaminergic signaling in locomotor decision-making. Finally, highlighting the broad applicability of our analyses, we generalize our findings to other species and tasks. We show that human participants in a virtual Y-maze exhibited similar decision predictability dynamics as flies. This study advances our understanding of decision-making processes, emphasizing the importance of spatial and temporal dynamics of locomotor behavior in the lead-up to discrete choice outcomes.

Temporal regularities shape perceptual decisions and striatal dopamine signals

Perceptual decisions should depend on sensory evidence. However, such decisions are also influenced by past choices and outcomes. These choice history biases may reflect advantageous strategies to exploit temporal regularities of natural environments. However, it is unclear whether and how observers can adapt their choice history biases to different temporal regularities, to exploit the multitude of temporal correlations that exist in nature. Here, we show that male mice adapt their perceptual choice history biases to different temporal regularities of visual stimuli. This adaptation was slow, evolving over hundreds of trials across several days. It occurred alongside a fast non-adaptive choice history bias, limited to a few trials. Both fast and slow trial history effects are well captured by a normative reinforcement learning algorithm with multi-trial belief states, comprising both current trial sensory and previous trial memory states. We demonstrate that dorsal striatal dopamine tracks predictions of the model and behavior, suggesting that striatal dopamine reports reward predictions associated with adaptive choice history biases. Our results reveal the adaptive nature of perceptual choice history biases and shed light on their underlying computational principles and neural correlates.

A feature-specific prediction error model explains dopaminergic heterogeneity

The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary 'feature-specific RPE' model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal's moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.

Active Learning for Discrete Latent Variable Models

Active learning seeks to reduce the amount of data required to fit the parameters of a model, thus forming an important class of techniques in modern machine learning. However, past work on active learning has largely overlooked latent variable models, which play a vital role in neuroscience, psychology, and a variety of other engineering and scientific disciplines. Here we address this gap by proposing a novel framework for maximum-mutual-information input selection for discrete latent variable regression models. We first apply our method to a class of models known as mixtures of linear regressions (MLR). While it is well known that active learning confers no advantage for linear-gaussian regression models, we use Fisher information to show analytically that active learning can nevertheless achieve large gains for mixtures of such models, and we validate this improvement using both simulations and real-world data. We then consider a powerful class of temporally structured latent variable models given by a hidden Markov model (HMM) with generalized linear model (GLM) observations, which has recently been used to identify discrete states from animal decision-making data. We show that our method substantially reduces the amount of data needed to fit GLM-HMMs and outperforms a variety of approximate methods based on variational and amortized inference. Infomax learning for latent variable models thus offers a powerful approach for characterizing temporally structured latent states, with a wide variety of applications in neuroscience and beyond.

Trial-history biases in evidence accumulation can give rise to apparent lapses in decision-making

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, previous work has suggested that they covary in their prevalence and that their proposed neural substrates overlap. Here we demonstrate that during decision-making, history biases and apparent lapses can both arise from a common cognitive process that is optimal under mistaken beliefs that the world is changing i.e. nonstationary. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct decision-making datasets of male rats, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Neural circuit models for evidence accumulation through choice-selective sequences

Decision making is traditionally thought to be mediated by populations of neurons whose firing rates persistently accumulate evidence across time. However, recent decision-making experiments in rodents have observed neurons across the brain that fire sequentially as a function of spatial position or time, rather than persistently, with the subset of neurons in the sequence depending on the animal's choice. We develop two new candidate circuit models, in which evidence is encoded either in the relative firing rates of two competing chains of neurons or in the network location of a stereotyped pattern ("bump") of neural activity. Encoded evidence is then faithfully transferred between neuronal populations representing different positions or times. Neural recordings from four different brain regions during a decision-making task showed that, during the evidence accumulation period, different brain regions displayed tuning curves consistent with different candidate models for evidence accumulation. This work provides mechanistic models and potential neural substrates for how graded-value information may be precisely accumulated within and transferred between neural populations, a set of computations fundamental to many cognitive operations.

Adaptive algorithms for shaping behavior

Dogs and laboratory mice are commonly trained to perform complex tasks by guiding them through a curriculum of simpler tasks ('shaping'). What are the principles behind effective shaping strategies? Here, we propose a machine learning framework for shaping animal behavior, where an autonomous teacher agent decides its student's task based on the student's transcript of successes and failures on previously assigned tasks. Using autonomous teachers that plan a curriculum in a common sequence learning task, we show that near-optimal shaping algorithms adaptively alternate between simpler and harder tasks to carefully balance reinforcement and extinction. Based on this intuition, we derive an adaptive shaping heuristic with minimal parameters, which we show is near-optimal on the sequence learning task and robustly trains deep reinforcement learning agents on navigation tasks that involve sparse, delayed rewards. Extensions to continuous curricula are explored. Our work provides a starting point towards a general computational framework for shaping animal behavior.

New information triggers prospective codes to adapt for flexible navigation

Navigating a dynamic world requires rapidly updating choices by integrating past experiences with new information. In hippocampus and prefrontal cortex, neural activity representing future goals is theorized to support planning. However, it remains unknown how prospective goal representations incorporate new, pivotal information. Accordingly, we designed a novel task that precisely introduces new information using virtual reality, and we recorded neural activity as mice flexibly adapted their planned destinations. We found that new information triggered increased hippocampal prospective representations of both possible goals; while in prefrontal cortex, new information caused prospective representations of choices to rapidly shift to the new choice. When mice did not flexibly adapt, prefrontal choice codes failed to switch, despite relatively intact hippocampal goal representations. Prospective code updating depended on the commitment to the initial choice and degree of adaptation needed. Thus, we show how prospective codes update with new information to flexibly adapt ongoing navigational plans.

Rat movements reflect internal decision dynamics in an evidence accumulation task

Perceptual decision-making involves multiple cognitive processes, including accumulation of sensory evidence, planning, and executing a motor action. How these processes are intertwined is unclear; some models assume that decision-related processes precede motor execution, whereas others propose that movements reflecting on-going decision processes occur before commitment to a choice. Here we develop and apply two complementary methods to study the relationship between decision processes and the movements leading up to a choice. The first is a free response pulse-based evidence accumulation task, in which stimuli continue until choice is reported. The second is a motion-based drift diffusion model (mDDM), in which movement variables from video pose estimation constrain decision parameters on a trial-by-trial basis. We find the mDDM provides a better model fit to rats' decisions in the free response accumulation task than traditional DDM models. Interestingly, on each trial we observed a period of time, prior to choice, that was characterized by head immobility. The length of this period was positively correlated with the rats' decision bounds and stimuli presented during this period had the greatest impact on choice. Together these results support a model in which internal decision dynamics are reflected in movements and demonstrate that inclusion of movement parameters improves the performance of diffusion-to-bound decision models.

Mixtures of strategies underlie rodent behavior during reversal learning

In reversal learning tasks, the behavior of humans and animals is often assumed to be uniform within single experimental sessions to facilitate data analysis and model fitting. However, behavior of agents can display substantial variability in single experimental sessions, as they execute different blocks of trials with different transition dynamics. Here, we observed that in a deterministic reversal learning task, mice display noisy and sub-optimal choice transitions even at the expert stages of learning. We investigated two sources of the sub-optimality in the behavior. First, we found that mice exhibit a high lapse rate during task execution, as they reverted to unrewarded directions after choice transitions. Second, we unexpectedly found that a majority of mice did not execute a uniform strategy, but rather mixed between several behavioral modes with different transition dynamics. We quantified the use of such mixtures with a state-space model, block Hidden Markov Model (block HMM), to dissociate the mixtures of dynamic choice transitions in individual blocks of trials. Additionally, we found that blockHMM transition modes in rodent behavior can be accounted for by two different types of behavioral algorithms, model-free or inference-based learning, that might be used to solve the task. Combining these approaches, we found that mice used a mixture of both exploratory, model-free strategies and deterministic, inference-based behavior in the task, explaining their overall noisy choice sequences. Together, our combined computational approach highlights intrinsic sources of noise in rodent reversal learning behavior and provides a richer description of behavior than conventional techniques, while uncovering the hidden states that underlie the block-by-block transitions.

Antipsychotic drug efficacy correlates with the modulation of D1 rather than D2 receptor-expressing striatal projection neurons

Elevated dopamine transmission in psychosis is assumed to unbalance striatal output through D1- and D2-receptor-expressing spiny-projection neurons (SPNs). Antipsychotic drugs are thought to re-balance this output by blocking D2 receptors (D2Rs). In this study, we found that amphetamine-driven dopamine release unbalanced D1-SPN and D2-SPN Ca2+ activity in mice, but that antipsychotic efficacy was associated with the reversal of abnormal D1-SPN, rather than D2-SPN, dynamics, even for drugs that are D2R selective or lacking any dopamine receptor affinity. By contrast, a clinically ineffective drug normalized D2-SPN dynamics but exacerbated D1-SPN dynamics under hyperdopaminergic conditions. Consistent with antipsychotic effect, selective D1-SPN inhibition attenuated amphetamine-driven changes in locomotion, sensorimotor gating and hallucination-like perception. Notably, antipsychotic efficacy correlated with the selective inhibition of D1-SPNs only under hyperdopaminergic conditions-a dopamine-state-dependence exhibited by D1R partial agonism but not non-antipsychotic D1R antagonists. Our findings provide new insights into antipsychotic drug mechanism and reveal an important role for D1-SPN modulation.

A distributed and efficient population code of mixed selectivity neurons for flexible navigation decisions

Decision-making requires flexibility to rapidly switch one's actions in response to sensory stimuli depending on information stored in memory. We identified cortical areas and neural activity patterns underlying this flexibility during virtual navigation, where mice switched navigation toward or away from a visual cue depending on its match to a remembered cue. Optogenetics screening identified V1, posterior parietal cortex (PPC), and retrosplenial cortex (RSC) as necessary for accurate decisions. Calcium imaging revealed neurons that can mediate rapid navigation switches by encoding a mixture of a current and remembered visual cue. These mixed selectivity neurons emerged through task learning and predicted the mouse's choices by forming efficient population codes before correct, but not incorrect, choices. They were distributed across posterior cortex, even V1, and were densest in RSC and sparsest in PPC. We propose flexibility in navigation decisions arises from neurons that mix visual and memory information within a visual-parietal-retrosplenial network.

CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

Reinforcement learning (RL) has become a popular paradigm for modeling animal behavior, analyzing neuronal representations, and studying their emergence during learning. This development has been fueled by advances in understanding the role of RL in both the brain and artificial intelligence. However, while in machine learning a set of tools and standardized benchmarks facilitate the development of new methods and their comparison to existing ones, in neuroscience, the software infrastructure is much more fragmented. Even if sharing theoretical principles, computational studies rarely share software frameworks, thereby impeding the integration or comparison of different results. Machine learning tools are also difficult to port to computational neuroscience since the experimental requirements are usually not well aligned. To address these challenges we introduce CoBeL-RL, a closed-loop simulator of complex behavior and learning based on RL and deep neural networks. It provides a neuroscience-oriented framework for efficiently setting up and running simulations. CoBeL-RL offers a set of virtual environments, e.g., T-maze and Morris water maze, which can be simulated at different levels of abstraction, e.g., a simple gridworld or a 3D environment with complex visual stimuli, and set up using intuitive GUI tools. A range of RL algorithms, e.g., Dyna-Q and deep Q-network algorithms, is provided and can be easily extended. CoBeL-RL provides tools for monitoring and analyzing behavior and unit activity, and allows for fine-grained control of the simulation via interfaces to relevant points in its closed-loop. In summary, CoBeL-RL fills an important gap in the software toolbox of computational neuroscience.

Strategically managing learning during perceptual decision making

Making optimal decisions in the face of noise requires balancing short-term speed and accuracy. But a theory of optimality should account for the fact that short-term speed can influence long-term accuracy through learning. Here, we demonstrate that long-term learning is an important dynamical dimension of the speed-accuracy trade-off. We study learning trajectories in rats and formally characterize these dynamics in a theory expressed as both a recurrent neural network and an analytical extension of the drift-diffusion model that learns over time. The model reveals that choosing suboptimal response times to learn faster sacrifices immediate reward, but can lead to greater total reward. We empirically verify predictions of the theory, including a relationship between stimulus exposure and learning speed, and a modulation of reaction time by future learning prospects. We find that rats' strategies approximately maximize total reward over the full learning epoch, suggesting cognitive control over the learning process.

Trial-history biases in evidence accumulation can give rise to apparent lapses

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, several hints in the literature suggest that they covary in their prevalence and that their proposed neural substrates overlap - what could underlie these links? Here we demonstrate that history biases and apparent lapses can both arise from a common cognitive process that is normative under misbeliefs about non-stationarity in the world. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct rat decision-making datasets, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Assessing evidence accumulation and rule learning in humans with an online game

Evidence accumulation, an essential component of perception and decision making, is frequently studied with psychophysical tasks involving noisy or ambiguous stimuli. In these tasks, participants typically receive verbal or written instructions that describe the strategy that should be used to guide decisions. Although convenient and effective, explicit instructions can influence learning and decision making strategies and can limit comparisons with animal models, in which behaviors are reinforced through feedback. Here, we developed an online video game and nonverbal training pipeline, inspired by pulse-based tasks for rodents, as an alternative to traditional psychophysical tasks used to study evidence accumulation. Using this game, we collected behavioral data from hundreds of participants trained with an explicit description of the decision rule or with experiential feedback. Participants trained with feedback alone learned the game rules rapidly and used strategies and displayed biases similar to those who received explicit instructions. Finally, by leveraging data across hundreds of participants, we show that perceptual judgments were well described by an accumulation process in which noise scaled nonlinearly with evidence, consistent with previous animal studies but inconsistent with diffusion models widely used to describe perceptual decisions in humans. These results challenge the conventional description of the accumulation process and suggest that online games provide a valuable platform to examine perceptual decision making and learning in humans. In addition, the feedback-based training pipeline developed for this game may be useful for evaluating perceptual decision making in human populations with difficulty following verbal instructions.NEW & NOTEWORTHY Perceptual uncertainty sets critical constraints on our ability to accumulate evidence and make decisions; however, its sources remain unclear. We developed a video game, and feedback-based training pipeline, to study uncertainty during decision making. Leveraging choices from hundreds of subjects, we demonstrate that human choices are inconsistent with popular diffusion models of human decision making and instead are best fit by models in which perceptual uncertainty scales nonlinearly with the strength of sensory evidence.

Refinements to rodent head fixation and fluid/food control for neuroscience

The use of head fixation in mice is increasingly common in research, its use having initially been restricted to the field of sensory neuroscience. Head restraint has often been combined with fluid control, rather than food restriction, to motivate behaviour, but this too is now in use for both restrained and non-restrained animals. Despite this, there is little guidance on how best to employ these techniques to optimise both scientific outcomes and animal welfare. This article summarises current practices and provides recommendations to improve animal wellbeing and data quality, based on a survey of the community, literature reviews, and the expert opinion and practical experience of an international working group convened by the UK's National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs). Topics covered include head fixation surgery and post-operative care, habituation to restraint, and the use of fluid/food control to motivate performance. We also discuss some recent developments that may offer alternative ways to collect data from large numbers of behavioural trials without the need for restraint. The aim is to provide support for researchers at all levels, animal care staff, and ethics committees to refine procedures and practices in line with the refinement principle of the 3Rs.

Shared and specialized coding across posterior cortical areas for dynamic navigation decisions

Animals adaptively integrate sensation, planning, and action to navigate toward goal locations in ever-changing environments, but the functional organization of cortex supporting these processes remains unclear. We characterized encoding in approximately 90,000 neurons across the mouse posterior cortex during a virtual navigation task with rule switching. The encoding of task and behavioral variables was highly distributed across cortical areas but differed in magnitude, resulting in three spatial gradients for visual cue, spatial position plus dynamics of choice formation, and locomotion, with peaks respectively in visual, retrosplenial, and parietal cortices. Surprisingly, the conjunctive encoding of these variables in single neurons was similar throughout the posterior cortex, creating high-dimensional representations in all areas instead of revealing computations specialized for each area. We propose that, for guiding navigation decisions, the posterior cortex operates in parallel rather than hierarchically, and collectively generates a state representation of the behavior and environment, with each area specialized in handling distinct information modalities.

Sensory Evidence Accumulation Using Optic Flow in a Naturalistic Navigation Task

Sensory evidence accumulation is considered a hallmark of decision-making in noisy environments. Integration of sensory inputs has been traditionally studied using passive stimuli, segregating perception from action. Lessons learned from this approach, however, may not generalize to ethological behaviors like navigation, where there is an active interplay between perception and action. We designed a sensory-based sequential decision task in virtual reality in which humans and monkeys navigated to a memorized location by integrating optic flow generated by their own joystick movements. A major challenge in such closed-loop tasks is that subjects' actions will determine future sensory input, causing ambiguity about whether they rely on sensory input rather than expectations based solely on a learned model of the dynamics. To test whether subjects integrated optic flow over time, we used three independent experimental manipulations, unpredictable optic flow perturbations, which pushed subjects off their trajectory; gain manipulation of the joystick controller, which changed the consequences of actions; and manipulation of the optic flow density, which changed the information borne by sensory evidence. Our results suggest that both macaques (male) and humans (female/male) relied heavily on optic flow, thereby demonstrating a critical role for sensory evidence accumulation during naturalistic action-perception closed-loop tasks.SIGNIFICANCE STATEMENT The temporal integration of evidence is a fundamental component of mammalian intelligence. Yet, it has traditionally been studied using experimental paradigms that fail to capture the closed-loop interaction between actions and sensations inherent in real-world continuous behaviors. These conventional paradigms use binary decision tasks and passive stimuli with statistics that remain stationary over time. Instead, we developed a naturalistic visuomotor visual navigation paradigm that mimics the causal structure of real-world sensorimotor interactions and probed the extent to which participants integrate sensory evidence by adding task manipulations that reveal complementary aspects of the computation.

Multiple timescales of sensory-evidence accumulation across the dorsal cortex

Cortical areas seem to form a hierarchy of intrinsic timescales, but the relevance of this organization for cognitive behavior remains unknown. In particular, decisions requiring the gradual accrual of sensory evidence over time recruit widespread areas across this hierarchy. Here, we tested the hypothesis that this recruitment is related to the intrinsic integration timescales of these widespread areas. We trained mice to accumulate evidence over seconds while navigating in virtual reality and optogenetically silenced the activity of many cortical areas during different brief trial epochs. We found that the inactivation of all tested areas affected the evidence-accumulation computation. Specifically, we observed distinct changes in the weighting of sensory evidence occurring during and before silencing, such that frontal inactivations led to stronger deficits on long timescales than posterior cortical ones. Inactivation of a subset of frontal areas also led to moderate effects on behavioral processes beyond evidence accumulation. Moreover, large-scale cortical Ca2+ activity during task performance displayed different temporal integration windows. Our findings suggest that the intrinsic timescale hierarchy of distributed cortical areas is an important component of evidence-accumulation mechanisms.

Opponent control of behavior by dorsomedial striatal pathways depends on task demands and internal state

A classic view of the striatum holds that activity in direct and indirect pathways oppositely modulates motor output. Whether this involves direct control of movement, or reflects a cognitive process underlying movement, remains unresolved. Here we find that strong, opponent control of behavior by the two pathways of the dorsomedial striatum depends on the cognitive requirements of a task. Furthermore, a latent state model (a hidden Markov model with generalized linear model observations) reveals that-even within a single task-the contribution of the two pathways to behavior is state dependent. Specifically, the two pathways have large contributions in one of two states associated with a strategy of evidence accumulation, compared to a state associated with a strategy of repeating previous choices. Thus, both the demands imposed by a task, as well as the internal state of mice when performing a task, determine whether dorsomedial striatum pathways provide strong and opponent control of behavior.

Computational validity: using computation to translate behaviours across species

We propose a new conceptual framework (computational validity) for translation across species and populations based on the computational similarity between the information processing underlying parallel tasks. Translating between species depends not on the superficial similarity of the tasks presented, but rather on the computational similarity of the strategies and mechanisms that underlie those behaviours. Computational validity goes beyond construct validity by directly addressing questions of information processing. Computational validity interacts with circuit validity as computation depends on circuits, but similar computations could be accomplished by different circuits. Because different individuals may use different computations to accomplish a given task, computational validity suggests that behaviour should be understood through the subject's point of view; thus, behaviour should be characterized on an individual level rather than a task level. Tasks can constrain the computational algorithms available to a subject and the observed subtleties of that behaviour can provide information about the computations used by each individual. Computational validity has especially high relevance for the study of psychiatric disorders, given the new views of psychiatry as identifying and mediating information processing dysfunctions that may show high inter-individual variability, as well as for animal models investigating aspects of human psychiatric disorders. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Mice alternate between discrete strategies during perceptual decision-making

Classical models of perceptual decision-making assume that subjects use a single, consistent strategy to form decisions, or that decision-making strategies evolve slowly over time. Here we present new analyses suggesting that this common view is incorrect. We analyzed data from mouse and human decision-making experiments and found that choice behavior relies on an interplay among multiple interleaved strategies. These strategies, characterized by states in a hidden Markov model, persist for tens to hundreds of trials before switching, and often switch multiple times within a session. The identified decision-making strategies were highly consistent across mice and comprised a single 'engaged' state, in which decisions relied heavily on the sensory stimulus, and several biased states in which errors frequently occurred. These results provide a powerful alternate explanation for 'lapses' often observed in rodent behavioral experiments, and suggest that standard measures of performance mask the presence of major changes in strategy across trials.

Sequential and efficient neural-population coding of complex task information

Recent work has highlighted that many types of variables are represented in each neocortical area. How can these many neural representations be organized together without interference and coherently maintained/updated through time? We recorded from excitatory neural populations in posterior cortices as mice performed a complex, dynamic task involving multiple interrelated variables. The neural encoding implied that highly correlated task variables were represented by less-correlated neural population modes, while pairs of neurons exhibited a spectrum of signal correlations. This finding relates to principles of efficient coding, but notably utilizes neural population modes as the encoding unit and suggests partial whitening of task-specific information where different variables are represented with different signal-to-noise levels. Remarkably, this encoding function was multiplexed with sequential neural dynamics yet reliably followed changes in task-variable correlations throughout the trial. We suggest that neural circuits can implement time-dependent encodings in a simple way using random sequential dynamics as a temporal scaffold.

Protean Kinematics: A Blended Model of VR Physics

Avatar research largely focuses on the effects of the appearance and external characteristics of avatars, but may also warrant further consideration of the effects of avatar movement characteristics. With Protean kinematics, we offer an expansion the avatar-user appearances-based effects of the Proteus Effect to a systematic exploration into the role of movement in affecting social perceptions (about others) and idealized perceptions (about self). This work presents both a theoretical (typology) and methodological (physics-based measurement) approach to understanding the complex blend of physical inputs and virtual outputs that occur in the perceptual experience of VR, particularly in consideration of the collection of hippocampal (e.g., place cells, grid cells) and entorhinal neurons (e.g., speed cells) that fire topologically relative to physical movement in physical space. Offered is a novel method that distills the blend of physical and virtual kinematics to contribute to modern understandings of human-agent interaction and cognitive psychology.

Probabilistic discrimination of relative stimulus features in mice

During perceptual decision-making, the brain encodes the upcoming decision and the stimulus information in a mixed representation. Paradigms suitable for studying decision computations in isolation rely on stimulus comparisons, with choices depending on relative rather than absolute properties of the stimuli. The adoption of tasks requiring relative perceptual judgments in mice would be advantageous in view of the powerful tools available for the dissection of brain circuits. However, whether and how mice can perform a relative visual discrimination task has not yet been fully established. Here, we show that mice can solve a complex orientation discrimination task in which the choices are decoupled from the orientation of individual stimuli. Moreover, we demonstrate a typical discrimination acuity of 9°, challenging the common belief that mice are poor visual discriminators. We reached these conclusions by introducing a probabilistic choice model that explained behavioral strategies in 40 mice and demonstrated that the circularity of the stimulus space is an additional source of choice variability for trials with fixed difficulty. Furthermore, history biases in the model changed with task engagement, demonstrating behavioral sensitivity to the availability of cognitive resources. In conclusion, our results reveal that mice adopt a diverse set of strategies in a task that decouples decision-relevant information from stimulus-specific information, thus demonstrating their usefulness as an animal model for studying neural representations of relative categories in perceptual decision-making research.

Geometry of abstract learned knowledge in the hippocampus

Hippocampal neurons encode physical variables^1-7 such as space¹ or auditory frequency⁶ in cognitive maps⁸. In addition, functional magnetic resonance imaging studies in humans have shown that the hippocampus can also encode more abstract, learned variables^9-11. However, their integration into existing neural representations of physical variables^12,13 is unknown. Here, using two-photon calcium imaging, we show that individual neurons in the dorsal hippocampus jointly encode accumulated evidence with spatial position in mice performing a decision-making task in virtual reality^14-16. Nonlinear dimensionality reduction¹³ showed that population activity was well-described by approximately four to six latent variables, which suggests that neural activity is constrained to a low-dimensional manifold. Within this low-dimensional space, both physical and abstract variables were jointly mapped in an orderly manner, creating a geometric representation that we show is similar across mice. The existence of conjoined cognitive maps suggests that the hippocampus performs a general computation-the creation of task-specific low-dimensional manifolds that contain a geometric representation of learned knowledge.

Standardized and reproducible measurement of decision-making in mice

Progress in science requires standardized assays whose results can be readily shared, compared, and reproduced across laboratories. Reproducibility, however, has been a concern in neuroscience, particularly for measurements of mouse behavior. Here, we show that a standardized task to probe decision-making in mice produces reproducible results across multiple laboratories. We adopted a task for head-fixed mice that assays perceptual and value-based decision making, and we standardized training protocol and experimental hardware, software, and procedures. We trained 140 mice across seven laboratories in three countries, and we collected 5 million mouse choices into a publicly available database. Learning speed was variable across mice and laboratories, but once training was complete there were no significant differences in behavior across laboratories. Mice in different laboratories adopted similar reliance on visual stimuli, on past successes and failures, and on estimates of stimulus prior probability to guide their choices. These results reveal that a complex mouse behavior can be reproduced across multiple laboratories. They establish a standard for reproducible rodent behavior, and provide an unprecedented dataset and open-access tools to study decision-making in mice. More generally, they indicate a path toward achieving reproducibility in neuroscience through collaborative open-science approaches.

Standardized and reproducible measurement of decision-making in mice

Progress in science requires standardized assays whose results can be readily shared, compared, and reproduced across laboratories. Reproducibility, however, has been a concern in neuroscience, particularly for measurements of mouse behavior. Here, we show that a standardized task to probe decision-making in mice produces reproducible results across multiple laboratories. We adopted a task for head-fixed mice that assays perceptual and value-based decision making, and we standardized training protocol and experimental hardware, software, and procedures. We trained 140 mice across seven laboratories in three countries, and we collected 5 million mouse choices into a publicly available database. Learning speed was variable across mice and laboratories, but once training was complete there were no significant differences in behavior across laboratories. Mice in different laboratories adopted similar reliance on visual stimuli, on past successes and failures, and on estimates of stimulus prior probability to guide their choices. These results reveal that a complex mouse behavior can be reproduced across multiple laboratories. They establish a standard for reproducible rodent behavior, and provide an unprecedented dataset and open-access tools to study decision-making in mice. More generally, they indicate a path toward achieving reproducibility in neuroscience through collaborative open-science approaches.

Discrimination of Regular and Irregular Rhythms Explained by a Time Difference Accumulation Model

Perceiving the temporal regularity in a sequence of repetitive sensory events facilitates the preparation and execution of relevant behaviors with tight temporal constraints. How we estimate temporal regularity from repeating patterns of sensory stimuli is not completely understood. We developed a decision-making task in which participants had to decide whether a train of visual, auditory, or tactile pulses, had a regular or an irregular temporal pattern. We tested the hypothesis that subjects categorize stimuli as irregular by accumulating the time differences between the predicted and observed times of sensory pulses defining a temporal rhythm. Results suggest that instead of waiting for a single large temporal deviation, participants accumulate timing-error signals and judge a pattern as irregular when the amount of evidence reaches a decision threshold. Model fits of bounded integration showed that this accumulation occurs with negligible leak of evidence. Consistent with previous findings, we show that participants perform better when evaluating the regularity of auditory pulses, as compared with visual or tactile stimuli. Our results suggest that temporal regularity is estimated by comparing expected and measured pulse onset times, and that each prediction error is accumulated towards a threshold to generate a behavioral choice.

Lapses in perceptual decisions reflect exploration

Perceptual decision-makers often display a constant rate of errors independent of evidence strength. These ‘lapses’ are treated as a nuisance arising from noise tangential to the decision, e.g. inattention or motor errors. Here, we use a multisensory decision task in rats to demonstrate that these explanations cannot account for lapses’ stimulus dependence. We propose a novel explanation: lapses reflect a strategic trade-off between exploiting known rewarding actions and exploring uncertain ones. We tested this model’s predictions by selectively manipulating one action’s reward magnitude or probability. As uniquely predicted by this model, changes were restricted to lapses associated with that action. Finally, we show that lapses are a powerful tool for assigning decision-related computations to neural structures based on disruption experiments (here, posterior striatum and secondary motor cortex). These results suggest that lapses reflect an integral component of decision-making and are informative about action values in normal and disrupted brain states.

Amplitude modulations of cortical sensory responses in pulsatile evidence accumulation

How does the brain internally represent a sequence of sensory information that jointly drives a decision-making behavior? Studies of perceptual decision-making have often assumed that sensory cortices provide noisy but otherwise veridical sensory inputs to downstream processes that accumulate and drive decisions. However, sensory processing in even the earliest sensory cortices can be systematically modified by various external and internal contexts. We recorded from neuronal populations across posterior cortex as mice performed a navigational decision-making task based on accumulating randomly timed pulses of visual evidence. Even in V1, only a small fraction of active neurons had sensory-like responses time-locked to each pulse. Here, we focus on how these 'cue-locked' neurons exhibited a variety of amplitude modulations from sensory to cognitive, notably by choice and accumulated evidence. These task-related modulations affected a large fraction of cue-locked neurons across posterior cortex, suggesting that future models of behavior should account for such influences.

Theoretical principles for illuminating sensorimotor processing with brain-wide neuronal recordings

Modern recording techniques now permit brain-wide sensorimotor circuits to be observed at single neuron resolution in small animals. Extracting theoretical understanding from these recordings requires principles that organize findings and guide future experiments. Here we review theoretical principles that shed light onto brain-wide sensorimotor processing. We begin with an analogy that conceptualizes principles as streetlamps that illuminate the empirical terrain, and we illustrate the analogy by showing how two familiar principles apply in new ways to brain-wide phenomena. We then focus the bulk of the review on describing three more principles that have wide utility for mapping brain-wide neural activity, making testable predictions from highly parameterized mechanistic models, and investigating the computational determinants of neuronal response patterns across the brain.

Brain-wide, scale-wide physiology underlying behavioral flexibility in zebrafish

The brain is tasked with choosing actions that maximize an animal's chances of survival and reproduction. These choices must be flexible and informed by the current state of the environment, the needs of the body, and the outcomes of past actions. This information is physiologically encoded and processed across different brain regions on a wide range of spatial scales, from molecules in single synapses to networks of brain areas. Uncovering these spatially distributed neural interactions underlying behavior requires investigations that span a similar range of spatial scales. Larval zebrafish, given their small size, transparency, and ease of genetic access, are a good model organism for such investigations, allowing the use of modern microscopy, molecular biology, and computational techniques. These approaches are yielding new insights into the mechanistic basis of behavioral states, which we review here and compare to related studies in mammalian species.

Novel Virtual Reality System for Auditory Tasks in Head-fixed Mice

An emerging corpus of research seeks to use virtual realities (VRs) to understand the neural mechanisms underlying spatial navigation and decision making in rodents. These studies have primarily used visual stimuli to represent the virtual world. However, auditory cues play an important role in navigation for animals, especially when the visual system cannot detect objects or predators. We have developed a virtual reality environment defined exclusively by free-field acoustic landmarks for head-fixed mice. We trained animals to run in a virtual environment with 3 acoustic landmarks. We present evidence that they can learn to navigate in our context: we observed anticipatory licking and modest anticipatory slowing preceding the reward region. Furthermore, we found that animals were highly aware of changes in landmark cues: licking behavior changed dramatically when the familiar virtual environment was switched to a novel one, and then rapidly reverted to normal when the familiar virtual environment was re-introduced, all within the same session. Finally, while animals executed the task, we performed in-vivo calcium imaging in the CA1 region of the hippocampus using a modified Miniscope.org system. Our experiments point to a future in which auditory virtual reality can be used to expand our understanding of the neural bases of audition in locomoting animals and the variety of sensory cues which anchor spatial representations in a new virtual environment.

A flexible framework for simulating and fitting generalized drift-diffusion models

The drift-diffusion model (DDM) is an important decision-making model in cognitive neuroscience. However, innovations in model form have been limited by methodological challenges. Here, we introduce the generalized drift-diffusion model (GDDM) framework for building and fitting DDM extensions, and provide a software package which implements the framework. The GDDM framework augments traditional DDM parameters through arbitrary user-defined functions. Models are solved numerically by directly solving the Fokker-Planck equation using efficient numerical methods, yielding a 100-fold or greater speedup over standard methodology. This speed allows GDDMs to be fit to data using maximum likelihood on the full response time (RT) distribution. We demonstrate fitting of GDDMs within our framework to both animal and human datasets from perceptual decision-making tasks, with better accuracy and fewer parameters than several DDMs implemented using the latest methodology, to test hypothesized decision-making mechanisms. Overall, our framework will allow for decision-making model innovation and novel experimental designs.

Sources of path integration error in young and aging humans

Path integration plays a vital role in navigation: it enables the continuous tracking of one's position in space by integrating self-motion cues. Path integration abilities vary widely across individuals, and tend to deteriorate in old age. The specific causes of path integration errors, however, remain poorly characterized. Here, we combine tests of path integration performance in participants of different ages with an analysis based on the Langevin equation for diffusive dynamics, which allows us to decompose errors into distinct causes that can corrupt path integration computations. We show that, across age groups, the dominant error source is unbiased noise that accumulates with travel distance not elapsed time, suggesting that the noise originates in the velocity input rather than within the integrator. Age-related declines are primarily traced to a growth in this noise. These findings shed light on the contributors to path integration error and the mechanisms underlying age-related navigational deficits.

Tracking the Mind's Eye: Primate Gaze Behavior during Virtual Visuomotor Navigation Reflects Belief Dynamics

To take the best actions, we often need to maintain and update beliefs about variables that cannot be directly observed. To understand the principles underlying such belief updates, we need tools to uncover subjects' belief dynamics from natural behavior. We tested whether eye movements could be used to infer subjects' beliefs about latent variables using a naturalistic navigation task. Humans and monkeys navigated to a remembered goal location in a virtual environment that provided optic flow but lacked explicit position cues. We observed eye movements that appeared to continuously track the goal location even when no visible target was present there. Accurate goal tracking was associated with improved task performance, and inhibiting eye movements in humans impaired navigation precision. These results suggest that gaze dynamics play a key role in action selection during challenging visuomotor behaviors and may possibly serve as a window into the subject's dynamically evolving internal beliefs.

Differentiating between integration and non-integration strategies in perceptual decision making

Many tasks used to study decision-making encourage subjects to integrate evidence over time. Such tasks are useful to understand how the brain operates on multiple samples of information over prolonged timescales, but only if subjects actually integrate evidence to form their decisions. We explored the behavioral observations that corroborate evidence-integration in a number of task-designs. Several commonly accepted signs of integration were also predicted by non-integration strategies. Furthermore, an integration model could fit data generated by non-integration models. We identified the features of non-integration models that allowed them to mimic integration and used these insights to design a motion discrimination task that disentangled the models. In human subjects performing the task, we falsified a non-integration strategy in each and confirmed prolonged integration in all but one subject. The findings illustrate the difficulty of identifying a decision-maker’s strategy and support solutions to achieve this goal.

Current approaches to modeling the virtual reality in rodents for the assessment of brain plasticity and behavior

Virtual reality (VR) and augmented reality (AR) have become valuable tools to study brains and behaviors resulting in development of new methods of diagnostics and treatment. Neurodegenerаtion is one of the best examples demonstrating efficacy of VR/АR technologies in modern neurology. Development of novel VR systems for rodents and combination of VR tools with up-to-date imaging techniques (i.e. MRI, imaging of neural networks etc.), brain electrophysiology (EEG, patch-clamp), precise analytics (microdialysis) allowed implementing of VR protocols into the animal neurobiology to study brain plasticity, sensorimotor integration, spatial navigation, memory, and decision-making. VR/AR for rodents is а young field of experimental neuroscience and has already provided more consistent testing conditions, less human-animal interaction, opportunities to use a wider variety of experimental parameters. Here we discuss present and future perspectives of using VR/AR to assess brain plasticity, neurogenesis and complex behavior in rodent and human study, and their advantages for translational neuroscience.

Somatic and Dendritic Encoding of Spatial Variables in Retrosplenial Cortex Differs during 2D Navigation

Active amplification of organized synaptic inputs in dendrites can endow individual neurons with the ability to perform complex computations. However, whether dendrites in behaving animals perform independent local computations is not known. Such activity may be particularly important for complex behavior, where neurons integrate multiple streams of information. Head-restrained imaging has yielded important insights into cellular and circuit function, but this approach limits behavior and the underlying computations. We describe a method for full-featured 2-photon imaging in awake mice during free locomotion with volitional head rotation. We examine head direction and position encoding in simultaneously imaged apical tuft dendrites and their respective cell bodies in retrosplenial cortex, an area that encodes multi-modal navigational information. Activity in dendrites was not determined solely by somatic activity but reflected distinct navigational variables, fulfilling the requirements for dendritic computation. Our approach provides a foundation for studying sub-cellular processes during complex behaviors.

Inferring learning rules from animal decision-making

How do animals learn? This remains an elusive question in neuroscience. Whereas reinforcement learning often focuses on the design of algorithms that enable artificial agents to efficiently learn new tasks, here we develop a modeling framework to directly infer the empirical learning rules that animals use to acquire new behaviors. Our method efficiently infers the trial-to-trial changes in an animal's policy, and decomposes those changes into a learning component and a noise component. Specifically, this allows us to: (i) compare different learning rules and objective functions that an animal may be using to update its policy; (ii) estimate distinct learning rates for different parameters of an animal's policy; (iii) identify variations in learning across cohorts of animals; and (iv) uncover trial-to-trial changes that are not captured by normative learning rules. After validating our framework on simulated choice data, we applied our model to data from rats and mice learning perceptual decision-making tasks. We found that certain learning rules were far more capable of explaining trial-to-trial changes in an animal's policy. Whereas the average contribution of the conventional REINFORCE learning rule to the policy update for mice learning the International Brain Laboratory's task was just 30%, we found that adding baseline parameters allowed the learning rule to explain 92% of the animals' policy updates under our model. Intriguingly, the best-fitting learning rates and baseline values indicate that an animal's policy update, at each trial, does not occur in the direction that maximizes expected reward. Understanding how an animal transitions from chance-level to high-accuracy performance when learning a new task not only provides neuroscientists with insight into their animals, but also provides concrete examples of biological learning algorithms to the machine learning community.

Task-Dependent Changes in the Large-Scale Dynamics and Necessity of Cortical Regions

Neural activity throughout the cortex is correlated with perceptual decisions, but inactivation studies suggest that only a small number of areas are necessary for these behaviors. Here we show that the number of required cortical areas and their dynamics vary across related tasks with different cognitive computations. In a visually guided virtual T-maze task, bilateral inactivation of only a few dorsal cortical regions impaired performance. In contrast, in tasks requiring evidence accumulation and/or post-stimulus memory, performance was impaired by inactivation of widespread cortical areas with diverse patterns of behavioral deficits across areas and tasks. Wide-field imaging revealed widespread ramps of Ca2+ activity during the accumulation and visually guided tasks. Additionally, during accumulation, different regions had more diverse activity profiles, leading to reduced inter-area correlations. Using a modular recurrent neural network model trained to perform analogous tasks, we argue that differences in computational strategies alone could explain these findings.

Cerebellar disruption impairs working memory during evidence accumulation

To select actions based on sensory evidence, animals must create and manipulate representations of stimulus information in memory. Here we report that during accumulation of somatosensory evidence, optogenetic manipulation of cerebellar Purkinje cells reduces the accuracy of subsequent memory-guided decisions and causes mice to downweight prior information. Behavioral deficits are consistent with the addition of noise and leak to the evidence accumulation process. We conclude that the cerebellum can influence the accurate maintenance of working memory.

Glia Accumulate Evidence that Actions Are Futile and Suppress Unsuccessful Behavior

When a behavior repeatedly fails to achieve its goal, animals often give up and become passive, which can be strategic for preserving energy or regrouping between attempts. It is unknown how the brain identifies behavioral failures and mediates this behavioral-state switch. In larval zebrafish swimming in virtual reality, visual feedback can be withheld so that swim attempts fail to trigger expected visual flow. After tens of seconds of such motor futility, animals became passive for similar durations. Whole-brain calcium imaging revealed noradrenergic neurons that responded specifically to failed swim attempts and radial astrocytes whose calcium levels accumulated with increasing numbers of failed attempts. Using cell ablation and optogenetic or chemogenetic activation, we found that noradrenergic neurons progressively activated brainstem radial astrocytes, which then suppressed swimming. Thus, radial astrocytes perform a computation critical for behavior: they accumulate evidence that current actions are ineffective and consequently drive changes in behavioral states. VIDEO ABSTRACT.