Magnetic voluntary head-fixation in transgenic rats enables lifespan imaging of hippocampal neurons

The precise neural mechanisms within the brain that contribute to the remarkable lifetime persistence of memory are not fully understood. Two-photon calcium imaging allows the activity of individual cells to be followed across long periods, but conventional approaches require head-fixation, which limits the type of behavior that can be studied. We present a magnetic voluntary head-fixation system that provides stable optical access to the brain during complex behavior. Compared to previous systems that used mechanical restraint, there are no moving parts and animals can engage and disengage entirely at will. This system is failsafe, easy for animals to use and reliable enough to allow long-term experiments to be routinely performed. Animals completed hundreds of trials per session of an odor discrimination task that required 2-4 s fixations. Together with a reflectance fluorescence collection scheme that increases two-photon signal and a transgenic Thy1-GCaMP6f rat line, we are able to reliably image the cellular activity in the hippocampus during behavior over long periods (median 6 months), allowing us track the same neurons over a large fraction of animals' lives (up to 19 months).

Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning

Different brain systems have been hypothesized to subserve multiple "experts" that compete to generate behavior. In reinforcement learning, two general processes, one model-free (MF) and one model-based (MB), are often modeled as a mixture of agents (MoA) and hypothesized to capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by a static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from a set of agents and the temporal dynamics of underlying "hidden" states that capture shifts in agent contributions over time. Applying this model to a multi-step,reward-guided task in rats reveals a progression of within-session strategies: a shift from initial MB exploration to MB exploitation, and finally to reduced engagement. The inferred states predict changes in both response time and OFC neural encoding during the task, suggesting that these states are capturing real shifts in dynamics.

Chronic brain functional ultrasound imaging in freely moving rodents performing cognitive tasks

BACKGROUND

Functional ultrasound imaging (fUS) is an emerging imaging technique that indirectly measures neural activity via changes in blood volume. Chronic fUS imaging during cognitive tasks in freely moving animals faces multiple exceptional challenges: performing large durable craniotomies with chronic implants, designing behavioral experiments matching the hemodynamic timescale, stabilizing the ultrasound probe during freely moving behavior, accurately assessing motion artifacts, and validating that the animal can perform cognitive tasks while tethered.

NEW METHOD

We provide validated solutions for those technical challenges. In addition, we present standardized step-by-step reproducible protocols, procedures, and data processing pipelines. Finally, we present proof-of-concept analysis of brain dynamics during a decision making task.

RESULTS

We obtain stable recordings from which we can robustly decode task variables from fUS data over multiple months. Moreover, we find that brain wide imaging through hemodynamic response is nonlinearly related to cognitive variables, such as task difficulty, as compared to sensory responses previously explored.

COMPARISON WITH EXISTING METHODS

Computational pipelines in fUS are nascent and we present an initial development of a full processing pathway to correct and segment fUS data.

CONCLUSIONS

Our methods provide stable imaging and analysis of behavior with fUS that will enable new experimental paradigms in understanding brain-wide dynamics in naturalistic behaviors.

Trial-history biases in evidence accumulation can give rise to apparent lapses in decision-making

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, previous work has suggested that they covary in their prevalence and that their proposed neural substrates overlap. Here we demonstrate that during decision-making, history biases and apparent lapses can both arise from a common cognitive process that is optimal under mistaken beliefs that the world is changing i.e. nonstationary. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct decision-making datasets of male rats, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Neural circuit models for evidence accumulation through choice-selective sequences

Decision making is traditionally thought to be mediated by populations of neurons whose firing rates persistently accumulate evidence across time. However, recent decision-making experiments in rodents have observed neurons across the brain that fire sequentially as a function of spatial position or time, rather than persistently, with the subset of neurons in the sequence depending on the animal's choice. We develop two new candidate circuit models, in which evidence is encoded either in the relative firing rates of two competing chains of neurons or in the network location of a stereotyped pattern ("bump") of neural activity. Encoded evidence is then faithfully transferred between neuronal populations representing different positions or times. Neural recordings from four different brain regions during a decision-making task showed that, during the evidence accumulation period, different brain regions displayed tuning curves consistent with different candidate models for evidence accumulation. This work provides mechanistic models and potential neural substrates for how graded-value information may be precisely accumulated within and transferred between neural populations, a set of computations fundamental to many cognitive operations.

Transitions in dynamical regime and neural mode underlie perceptual decision-making

Perceptual decision-making is the process by which an animal uses sensory stimuli to choose an action or mental proposition. This process is thought to be mediated by neurons organized as attractor networks 1,2 . However, whether attractor dynamics underlie decision behavior and the complex neuronal responses remains unclear. Here we use an unsupervised, deep learning-based method to discover decision-related dynamics from the simultaneous activity of neurons in frontal cortex and striatum of rats while they accumulate pulsatile auditory evidence. We show that contrary to prevailing hypotheses, attractors play a role only after a transition from a regime in the dynamics that is strongly driven by inputs to one dominated by the intrinsic dynamics. The initial regime mediates evidence accumulation, and the subsequent intrinsic-dominant regime subserves decision commitment. This regime transition is coupled to a rapid reorganization in the representation of the decision process in the neural population (a change in the "neural mode" along which the process develops). A simplified model approximating the coupled transition in the dynamics and neural mode allows inferring, from each trial's neural activity, the internal decision commitment time in that trial, and captures diverse and complex single-neuron temporal profiles, such as ramping and stepping 3-5 . It also captures trial-averaged curved trajectories 6-8 , and reveals distinctions between brain regions. Our results show that the formation of a perceptual choice involves a rapid, coordinated transition in both the dynamical regime and the neural mode of the decision process, and suggest pairing deep learning and parsimonious models as a promising approach for understanding complex data.

Flow-field inference from neural data using deep recurrent networks

Computations involved in processes such as decision-making, working memory, and motor control are thought to emerge from the dynamics governing the collective activity of neurons in large populations. But the estimation of these dynamics remains a significant challenge. Here we introduce Flow-field Inference from Neural Data using deep Recurrent networks (FINDR), an unsupervised deep learning method that can infer low-dimensional nonlinear stochastic dynamics underlying neural population activity. Using population spike train data from frontal brain regions of rats performing an auditory decision-making task, we demonstrate that FINDR outperforms existing methods in capturing the heterogeneous responses of individual neurons. We further show that FINDR can discover interpretable low-dimensional dynamics when it is trained to disentangle task-relevant and irrelevant components of the neural population activity. Importantly, the low-dimensional nature of the learned dynamics allows for explicit visualization of flow fields and attractor structures. We suggest FINDR as a powerful method for revealing the low-dimensional task-relevant dynamics of neural populations and their associated computations.

Princeton RAtlas: A Common Coordinate Framework for Fully cleared, Whole Rattus norvegicus Brains

Whole-brain clearing and imaging methods are becoming more common in mice but have yet to become standard in rats, at least partially due to inadequate clearing from most available protocols. Here, we build on recent mouse-tissue clearing and light-sheet imaging methods and develop and adapt them to rats. We first used cleared rat brains to create an open-source, 3D rat atlas at 25 μm resolution. We then registered and imported other existing labeled volumes and made all of the code and data available for the community (https://github.com/emilyjanedennis/PRA) to further enable modern, whole-brain neuroscience in the rat. Key features • This protocol adapts iDISCO (Renier et al., 2014) and uDISCO (Pan et al., 2016) tissue-clearing techniques to consistently clear rat brains. • This protocol also decreases the number of working hours per day to fit in an 8 h workday. Graphical overview.

Trial-history biases in evidence accumulation can give rise to apparent lapses

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, several hints in the literature suggest that they covary in their prevalence and that their proposed neural substrates overlap - what could underlie these links? Here we demonstrate that history biases and apparent lapses can both arise from a common cognitive process that is normative under misbeliefs about non-stationarity in the world. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct rat decision-making datasets, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Value representations in the rodent orbitofrontal cortex drive learning, not choice

Humans and animals make predictions about the rewards they expect to receive in different situations. In formal models of behavior, these predictions are known as value representations, and they play two very different roles. Firstly, they drive choice: the expected values of available options are compared to one another, and the best option is selected. Secondly, they support learning: expected values are compared to rewards actually received, and future expectations are updated accordingly. Whether these different functions are mediated by different neural representations remains an open question. Here, we employ a recently developed multi-step task for rats that computationally separates learning from choosing. We investigate the role of value representations in the rodent orbitofrontal cortex, a key structure for value-based cognition. Electrophysiological recordings and optogenetic perturbations indicate that these representations do not directly drive choice. Instead, they signal expected reward information to a learning process elsewhere in the brain that updates choice mechanisms.

Multiple timescales of sensory-evidence accumulation across the dorsal cortex

Cortical areas seem to form a hierarchy of intrinsic timescales, but the relevance of this organization for cognitive behavior remains unknown. In particular, decisions requiring the gradual accrual of sensory evidence over time recruit widespread areas across this hierarchy. Here, we tested the hypothesis that this recruitment is related to the intrinsic integration timescales of these widespread areas. We trained mice to accumulate evidence over seconds while navigating in virtual reality and optogenetically silenced the activity of many cortical areas during different brief trial epochs. We found that the inactivation of all tested areas affected the evidence-accumulation computation. Specifically, we observed distinct changes in the weighting of sensory evidence occurring during and before silencing, such that frontal inactivations led to stronger deficits on long timescales than posterior cortical ones. Inactivation of a subset of frontal areas also led to moderate effects on behavioral processes beyond evidence accumulation. Moreover, large-scale cortical Ca2+ activity during task performance displayed different temporal integration windows. Our findings suggest that the intrinsic timescale hierarchy of distributed cortical areas is an important component of evidence-accumulation mechanisms.

Stable choice coding in rat frontal orienting fields across model-predicted changes of mind

During decision making in a changing environment, evidence that may guide the decision accumulates until the point of action. In the rat, provisional choice is thought to be represented in frontal orienting fields (FOF), but this has only been tested in static environments where provisional and final decisions are not easily dissociated. Here, we characterize the representation of accumulated evidence in the FOF of rats performing a recently developed dynamic evidence accumulation task, which induces changes in the provisional decision, referred to as “changes of mind”. We find that FOF encodes evidence throughout decision formation with a temporal gain modulation that rises until the period when the animal may need to act. Furthermore, reversals in FOF firing rates can be accounted for by changes of mind predicted using a model of the decision process fit only to behavioral data. Our results suggest that the FOF represents provisional decisions even in dynamic, uncertain environments, allowing for rapid motor execution when it is time to act.

A leaky accumulation model can predict rats’ changes of mind during decision making in a dynamic environment explaining reversals in frontal cortical activity and demonstrating a stable choice code despite environmental uncertainty.

Sequential and efficient neural-population coding of complex task information

Recent work has highlighted that many types of variables are represented in each neocortical area. How can these many neural representations be organized together without interference and coherently maintained/updated through time? We recorded from excitatory neural populations in posterior cortices as mice performed a complex, dynamic task involving multiple interrelated variables. The neural encoding implied that highly correlated task variables were represented by less-correlated neural population modes, while pairs of neurons exhibited a spectrum of signal correlations. This finding relates to principles of efficient coding, but notably utilizes neural population modes as the encoding unit and suggests partial whitening of task-specific information where different variables are represented with different signal-to-noise levels. Remarkably, this encoding function was multiplexed with sequential neural dynamics yet reliably followed changes in task-variable correlations throughout the trial. We suggest that neural circuits can implement time-dependent encodings in a simple way using random sequential dynamics as a temporal scaffold.

Subpopulations of neurons in lOFC encode previous and current rewards at time of choice

Studies of neural dynamics in lateral orbitofrontal cortex (lOFC) have shown that subsets of neurons that encode distinct aspects of behavior, such as value, may project to common downstream targets. However, it is unclear whether reward history, which may subserve lOFC's well-documented role in learning, is represented by functional subpopulations in lOFC. Previously, we analyzed neural recordings from rats performing a value-based decision-making task, and we documented trial-by-trial learning that required lOFC (Constantinople et al., 2019). Here, we characterize functional subpopulations of lOFC neurons during behavior, including their encoding of task variables. We found five distinct clusters of lOFC neurons, either based on clustering of their trial-averaged peristimulus time histograms (PSTHs), or a feature space defined by their average conditional firing rates aligned to different task variables. We observed weak encoding of reward attributes, but stronger encoding of reward history, the animal's left or right choice, and reward receipt across all clusters. Only one cluster, however, encoded the animal's reward history at the time shortly preceding the choice, suggesting a possible role in integrating previous and current trial outcomes at the time of choice. This cluster also exhibits qualitatively similar responses to identified corticostriatal projection neurons in a recent study (Hirokawa et al., 2019), and suggests a possible role for subpopulations of lOFC neurons in mediating trial-by-trial learning.

Interrogating theoretical models of neural computation with emergent property inference

A cornerstone of theoretical neuroscience is the circuit model: a system of equations that captures a hypothesized neural mechanism. Such models are valuable when they give rise to an experimentally observed phenomenon -- whether behavioral or a pattern of neural activity -- and thus can offer insights into neural computation. The operation of these circuits, like all models, critically depends on the choice of model parameters. A key step is then to identify the model parameters consistent with observed phenomena: to solve the inverse problem. In this work, we present a novel technique, emergent property inference (EPI), that brings the modern probabilistic modeling toolkit to theoretical neuroscience. When theorizing circuit models, theoreticians predominantly focus on reproducing computational properties rather than a particular dataset. Our method uses deep neural networks to learn parameter distributions with these computational properties. This methodology is introduced through a motivational example of parameter inference in the stomatogastric ganglion. EPI is then shown to allow precise control over the behavior of inferred parameters and to scale in parameter dimension better than alternative techniques. In the remainder of this work, we present novel theoretical findings in models of primary visual cortex and superior colliculus, which were gained through the examination of complex parametric structure captured by EPI. Beyond its scientific contribution, this work illustrates the variety of analyses possible once deep learning is harnessed towards solving theoretical inverse problems.

Geometry of abstract learned knowledge in the hippocampus

Hippocampal neurons encode physical variables^1-7 such as space¹ or auditory frequency⁶ in cognitive maps⁸. In addition, functional magnetic resonance imaging studies in humans have shown that the hippocampus can also encode more abstract, learned variables^9-11. However, their integration into existing neural representations of physical variables^12,13 is unknown. Here, using two-photon calcium imaging, we show that individual neurons in the dorsal hippocampus jointly encode accumulated evidence with spatial position in mice performing a decision-making task in virtual reality^14-16. Nonlinear dimensionality reduction¹³ showed that population activity was well-described by approximately four to six latent variables, which suggests that neural activity is constrained to a low-dimensional manifold. Within this low-dimensional space, both physical and abstract variables were jointly mapped in an orderly manner, creating a geometric representation that we show is similar across mice. The existence of conjoined cognitive maps suggests that the hippocampus performs a general computation-the creation of task-specific low-dimensional manifolds that contain a geometric representation of learned knowledge.

Extracting the dynamics of behavior in sensory decision-making experiments

Decision-making strategies evolve during training and can continue to vary even in well-trained animals. However, studies of sensory decision-making tend to characterize behavior in terms of a fixed psychometric function that is fit only after training is complete. Here, we present PsyTrack, a flexible method for inferring the trajectory of sensory decision-making strategies from choice data. We apply PsyTrack to training data from mice, rats, and human subjects learning to perform auditory and visual decision-making tasks. We show that it successfully captures trial-to-trial fluctuations in the weighting of sensory stimuli, bias, and task-irrelevant covariates such as choice and stimulus history. This analysis reveals dramatic differences in learning across mice and rapid adaptation to changes in task statistics. PsyTrack scales easily to large datasets and offers a powerful tool for quantifying time-varying behavior in a wide variety of animals and tasks.

Amplitude modulations of cortical sensory responses in pulsatile evidence accumulation

How does the brain internally represent a sequence of sensory information that jointly drives a decision-making behavior? Studies of perceptual decision-making have often assumed that sensory cortices provide noisy but otherwise veridical sensory inputs to downstream processes that accumulate and drive decisions. However, sensory processing in even the earliest sensory cortices can be systematically modified by various external and internal contexts. We recorded from neuronal populations across posterior cortex as mice performed a navigational decision-making task based on accumulating randomly timed pulses of visual evidence. Even in V1, only a small fraction of active neurons had sensory-like responses time-locked to each pulse. Here, we focus on how these 'cue-locked' neurons exhibited a variety of amplitude modulations from sensory to cognitive, notably by choice and accumulated evidence. These task-related modulations affected a large fraction of cue-locked neurons across posterior cortex, suggesting that future models of behavior should account for such influences.

An approach for long-term, multi-probe Neuropixels recordings in unrestrained rats

The use of Neuropixels probes for chronic neural recordings is in its infancy and initial studies leave questions about long-term stability and probe reusability unaddressed. Here, we demonstrate a new approach for chronic Neuropixels recordings over a period of months in freely moving rats. Our approach allows multiple probes per rat and multiple cycles of probe reuse. We found that hundreds of units could be recorded for multiple months, but that yields depended systematically on anatomical position. Explanted probes displayed a small increase in noise compared to unimplanted probes, but this was insufficient to impair future single-unit recordings. We conclude that cost-effective, multi-region, and multi-probe Neuropixels recordings can be carried out with high yields over multiple months in rats or other similarly sized animals. Our methods and observations may facilitate the standardization of chronic recording from Neuropixels probes in freely moving animals.

Task-Dependent Changes in the Large-Scale Dynamics and Necessity of Cortical Regions

Neural activity throughout the cortex is correlated with perceptual decisions, but inactivation studies suggest that only a small number of areas are necessary for these behaviors. Here we show that the number of required cortical areas and their dynamics vary across related tasks with different cognitive computations. In a visually guided virtual T-maze task, bilateral inactivation of only a few dorsal cortical regions impaired performance. In contrast, in tasks requiring evidence accumulation and/or post-stimulus memory, performance was impaired by inactivation of widespread cortical areas with diverse patterns of behavioral deficits across areas and tasks. Wide-field imaging revealed widespread ramps of Ca2+ activity during the accumulation and visually guided tasks. Additionally, during accumulation, different regions had more diverse activity profiles, leading to reduced inter-area correlations. Using a modular recurrent neural network model trained to perform analogous tasks, we argue that differences in computational strategies alone could explain these findings.

Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases

Individual choices are not made in isolation but are embedded in a series of past experiences, decisions, and outcomes. The effects of past experiences on choices, often called sequential biases, are ubiquitous in perceptual and value-based decision-making, but their neural substrates are unclear. We trained rats to choose between cued guaranteed and probabilistic rewards in a task in which outcomes on each trial were independent. Behavioral variability often reflected sequential effects, including increased willingness to take risks following risky wins, and spatial 'win-stay/lose-shift' biases. Recordings from lateral orbitofrontal cortex (lOFC) revealed encoding of reward history and receipt, and optogenetic inhibition of lOFC eliminated rats' increased preference for risk following risky wins, but spared other sequential effects. Our data show that different sequential biases are neurally dissociable, and the lOFC's role in adaptive behavior promotes learning of more abstract biases (here, biases for the risky option), but not spatial ones.

Coarse Graining, Fixed Points, and Scaling in a Large Population of Neurons

We develop a phenomenological coarse-graining procedure for activity in a large network of neurons, and apply this to recordings from a population of 1000+ cells in the hippocampus. Distributions of coarse-grained variables seem to approach a fixed non-Gaussian form, and we see evidence of scaling in both static and dynamic quantities. These results suggest that the collective behavior of the network is described by a nontrivial fixed point.

An Analysis of Decision under Risk in Rats

In 1979, Daniel Kahneman and Amos Tversky published a ground-breaking paper titled "Prospect Theory: An Analysis of Decision under Risk," which presented a behavioral economic theory that accounted for the ways in which humans deviate from economists' normative workhorse model, Expected Utility Theory [1, 2]. For example, people exhibit probability distortion (they overweight low probabilities), loss aversion (losses loom larger than gains), and reference dependence (outcomes are evaluated as gains or losses relative to an internal reference point). We found that rats exhibited many of these same biases, using a task in which rats chose between guaranteed and probabilistic rewards. However, prospect theory assumes stable preferences in the absence of learning, an assumption at odds with alternative frameworks such as animal learning theory and reinforcement learning [3-7]. Rats also exhibited trial history effects, consistent with ongoing learning. A reinforcement learning model in which state-action values were updated by the subjective value of outcomes according to prospect theory reproduced rats' nonlinear utility and probability weighting functions and also captured trial-by-trial learning dynamics.

Efficient inference for time-varying behavior during learning

The process of learning new behaviors over time is a problem of great interest in both neuroscience and artificial intelligence. However, most standard analyses of animal training data either treat behavior as fixed or track only coarse performance statistics (e.g., accuracy, bias), providing limited insight into the evolution of the policies governing behavior. To overcome these limitations, we propose a dynamic psychophysical model that efficiently tracks trial-to-trial changes in behavior over the course of training. Our model consists of a dynamic logistic regression model, parametrized by a set of time-varying weights that express dependence on sensory stimuli as well as task-irrelevant covariates, such as stimulus, choice, and answer history. Our implementation scales to large behavioral datasets, allowing us to infer 500K parameters (e.g., 10 weights over 50K trials) in minutes on a desktop computer. We optimize hyperparameters governing how rapidly each weight evolves over time using the decoupled Laplace approximation, an efficient method for maximizing marginal likelihood in non-conjugate models. To illustrate performance, we apply our method to psychophysical data from both rats and human subjects learning a delayed sensory discrimination task. The model successfully tracks the psychophysical weights of rats over the course of training, capturing day-to-day and trial-to-trial fluctuations that underlie changes in performance, choice bias, and dependencies on task history. Finally, we investigate why rats frequently make mistakes on easy trials, and suggest that apparent lapses can be explained by sub-optimal weighting of known task covariates.

Rats adopt the optimal timescale for evidence integration in a dynamic environment

Decision making in dynamic environments requires discounting old evidence that may no longer inform the current state of the world. Previous work found that humans discount old evidence in a dynamic environment, but do not discount at the optimal rate. Here we investigated whether rats can optimally discount evidence in a dynamic environment by adapting the timescale over which they accumulate evidence. Using discrete evidence pulses, we exactly compute the optimal inference process. We show that the optimal timescale for evidence discounting depends on both the stimulus statistics and noise in sensory processing. When both of these components are taken into account, rats accumulate and discount evidence with the optimal timescale. Finally, by changing the volatility of the environment, we demonstrate experimental control over the rats' accumulation timescale. The mechanisms supporting integration are a subject of extensive study, and experimental control over these timescales may open new avenues of investigation.

Causal contribution and dynamical encoding in the striatum during evidence accumulation

A broad range of decision-making processes involve gradual accumulation of evidence over time, but the neural circuits responsible for this computation are not yet established. Recent data indicate that cortical regions that are prominently associated with accumulating evidence, such as the posterior parietal cortex and the frontal orienting fields, may not be directly involved in this computation. Which, then, are the regions involved? Regions that are directly involved in evidence accumulation should directly influence the accumulation-based decision-making behavior, have a graded neural encoding of accumulated evidence and contribute throughout the accumulation process. Here, we investigated the role of the anterior dorsal striatum (ADS) in a rodent auditory evidence accumulation task using a combination of behavioral, pharmacological, optogenetic, electrophysiological and computational approaches. We find that the ADS is the first brain region known to satisfy the three criteria. Thus, the ADS may be the first identified node in the network responsible for evidence accumulation.

Author Correction: Dorsal hippocampus contributes to model-based planning

In the version of this article initially published, the green label in Fig. 1c read "rightward choices" instead of "leftward choices." The error has been corrected in the HTML and PDF versions of the article.

An Accumulation-of-Evidence Task Using Visual Pulses for Mice Navigating in Virtual Reality

The gradual accumulation of sensory evidence is a crucial component of perceptual decision making, but its neural mechanisms are still poorly understood. Given the wide availability of genetic and optical tools for mice, they can be useful model organisms for the study of these phenomena; however, behavioral tools are largely lacking. Here, we describe a new evidence-accumulation task for head-fixed mice navigating in a virtual reality (VR) environment. As they navigate down the stem of a virtual T-maze, they see brief pulses of visual evidence on either side, and retrieve a reward on the arm with the highest number of pulses. The pulses occur randomly with Poisson statistics, yielding a diverse yet well-controlled stimulus set, making the data conducive to a variety of computational approaches. A large number of mice of different genotypes were able to learn and consistently perform the task, at levels similar to rats in analogous tasks. They are sensitive to side differences of a single pulse, and their memory of the cues is stable over time. Moreover, using non-parametric as well as modeling approaches, we show that the mice indeed accumulate evidence: they use multiple pulses of evidence from throughout the cue region of the maze to make their decision, albeit with a small overweighting of earlier cues, and their performance is affected by the magnitude but not the duration of evidence. Additionally, analysis of the mice's running patterns revealed that trajectories are fairly stereotyped yet modulated by the amount of sensory evidence, suggesting that the navigational component of this task may provide a continuous readout correlated to the underlying cognitive variables. Our task, which can be readily integrated with state-of-the-art techniques, is thus a valuable tool to study the circuit mechanisms and dynamics underlying perceptual decision making, particularly under more complex behavioral contexts.

Dorsal hippocampus contributes to model-based planning

Planning can be defined as action selection that leverages an internal model of the outcomes likely to follow each possible action. Its neural mechanisms remain poorly understood. Here we adapt recent advances from human research for rats, presenting for the first time an animal task that produces many trials of planned behavior per session, making multitrial rodent experimental tools available to study planning. We use part of this toolkit to address a perennially controversial issue in planning: the role of the dorsal hippocampus. Although prospective hippocampal representations have been proposed to support planning, intact planning in animals with damaged hippocampi has been repeatedly observed. Combining formal algorithmic behavioral analysis with muscimol inactivation, we provide causal evidence directly linking dorsal hippocampus with planning behavior. Our results and methods open the door to new and more detailed investigations of the neural mechanisms of planning in the hippocampus and throughout the brain.

Rat Prefrontal Cortex Inactivations during Decision Making Are Explained by Bistable Attractor Dynamics

Two-node attractor networks are flexible models for neural activity during decision making. Depending on the network configuration, these networks can model distinct aspects of decisions including evidence integration, evidence categorization, and decision memory. Here, we use attractor networks to model recent causal perturbations of the frontal orienting fields (FOF) in rat cortex during a perceptual decision-making task (Erlich, Brunton, Duan, Hanks, & Brody, 2015 ). We focus on a striking feature of the perturbation results. Pharmacological silencing of the FOF resulted in a stimulus-independent bias. We fit several models to test whether integration, categorization, or decision memory could account for this bias and found that only the memory configuration successfully accounts for it. This memory model naturally accounts for optogenetic perturbations of FOF in the same task and correctly predicts a memory-duration-dependent deficit caused by silencing FOF in a different task. Our results provide mechanistic support for a "postcategorization" memory role of the FOF in upcoming choices.

Neural underpinnings of the evidence accumulator

Gradual accumulation of evidence favoring one or another choice is considered a core component of many different types of decisions, and has been the subject of many neurophysiological studies in non-human primates. But its neural circuit mechanisms remain mysterious. Investigating it in rodents has recently become possible, facilitating perturbation experiments to delineate the relevant causal circuit, as well as the application of other tools more readily available in rodents. In addition, advances in stimulus design and analysis have aided studying the relevant neural encoding. In complement to ongoing non-human primate studies, these newly available model systems and tools place the field at an exciting time that suggests that the dynamical circuit mechanisms underlying accumulation of evidence could soon be revealed.

Sources of noise during accumulation of evidence in unrestrained and voluntarily head-restrained rats

Decision-making behavior is often characterized by substantial variability, but its source remains unclear. We developed a visual accumulation of evidence task designed to quantify sources of noise and to be performed during voluntary head restraint, enabling cellular resolution imaging in future studies. Rats accumulated discrete numbers of flashes presented to the left and right visual hemifields and indicated the side that had the greater number of flashes. Using a signal-detection theory-based model, we found that the standard deviation in their internal estimate of flash number scaled linearly with the number of flashes. This indicates a major source of noise that, surprisingly, is not consistent with the widely used 'drift-diffusion modeling' (DDM) approach but is instead closely related to proposed models of numerical cognition and counting. We speculate that this form of noise could be important in accumulation of evidence tasks generally.

DOI:http://dx.doi.org/10.7554/eLife.11308.001

Perceptual decision-making, i.e. making choices based on observed evidence, is rarely perfect. Humans and other animals tend to respond correctly on some trials and incorrectly on others. For over a century, this variability has been used to study the basis of decision-making. Most behavioral models assume that random fluctuations or 'noise' in the decision-making process is the primary source of variability and errors. However, the nature of this noise is unclear and the subject of intense scrutiny.

To investigate the sources of the behavioral variability during decision-making, Scott, Constantinople et al. trained rats to perform a visual 'accumulation of evidence' task. The animals counted flashes of light that appeared on either their left or their right. Up to 15 flashes occurred on each side, in a random order, and the rats then received a reward if they selected the side that the greatest number of flashes had occurred on. The rats chose correctly on many occasions but not on every single one.

Using a computer-controlled rat training facility or 'rat academy', Scott, Constantinople et al. collected hundreds of thousands of behavioral trials from over a dozen rats. This large dataset provided the statistical power necessary to test the assumptions of leading models of behavioral variability during decision-making, and revealed that noise grew more rapidly with the number of flashes than previously predicted. This finding explained patterns of behavior that previous models struggled with, most notably the fact that individuals make errors even on the easiest trials. The analysis also revealed that animals maintain two separate running totals – one of stimuli on the left and another of stimuli on the right – rather than a single tally of the difference between the two.

Scott, Constantinople et al. further demonstrated that rats could be trained to perform this task using a new system that enables functional brain imaging. The next step is to repeat these experiments while simultaneously recording brain activity to study the neural circuits that underlie decision-making and its variability.

DOI:http://dx.doi.org/10.7554/eLife.11308.002

Cortical and Subcortical Contributions to Short-Term Memory for Orienting Movements

Neural activity in frontal cortical areas has been causally linked to short-term memory (STM), but whether this activity is necessary for forming, maintaining, or reading out STM remains unclear. In rats performing a memory-guided orienting task, the frontal orienting fields in cortex (FOF) are considered critical for STM maintenance, and during each trial display a monotonically increasing neural encoding for STM. Here, we transiently inactivated either the FOF or the superior colliculus and found that the resulting impairments in memory-guided orienting performance followed a monotonically decreasing time course, surprisingly opposite to the neural encoding. A dynamical attractor model in which STM relies equally on cortical and subcortical regions reconciled the encoding and inactivation data. We confirmed key predictions of the model, including a time-dependent relationship between trial difficulty and perturbability, and substantial, supralinear, impairment following simultaneous inactivation of the FOF and superior colliculus during memory maintenance.

Distinct effects of prefrontal and parietal cortex inactivations on an accumulation of evidence task in the rat

Numerous brain regions have been shown to have neural correlates of gradually accumulating evidence for decision-making, but the causal roles of these regions in decisions driven by accumulation of evidence have yet to be determined. Here, in rats performing an auditory evidence accumulation task, we inactivated the frontal orienting fields (FOF) and posterior parietal cortex (PPC), two rat cortical regions that have neural correlates of accumulating evidence and that have been proposed as central to decision-making. We used a detailed model of the decision process to analyze the effect of inactivations. Inactivation of the FOF induced substantial performance impairments that were quantitatively best described as an impairment in the output pathway of an evidence accumulator with a long integration time constant (>240 ms). In contrast, we found a minimal role for PPC in decisions guided by accumulating auditory evidence, even while finding a strong role for PPC in internally-guided decisions.

Distinct relationships of parietal and prefrontal cortices to evidence accumulation

Gradual accumulation of evidence is thought to be fundamental for decision-making, and its neural correlates have been found in several brain regions. Here we develop a generalizable method to measure tuning curves that specify the relationship between neural responses and mentally accumulated evidence, and apply it to distinguish the encoding of decision variables in posterior parietal cortex and prefrontal cortex (frontal orienting fields, FOF). We recorded the firing rates of neurons in posterior parietal cortex and FOF from rats performing a perceptual decision-making task. Classical analyses uncovered correlates of accumulating evidence, similar to previous observations in primates and also similar across the two regions. However, tuning curve assays revealed that while the posterior parietal cortex encodes a graded value of the accumulating evidence, the FOF has a more categorical encoding that indicates, throughout the trial, the decision provisionally favoured by the evidence accumulated so far. Contrary to current views, this suggests that premotor activity in the frontal cortex does not have a role in the accumulation process, but instead has a more categorical function, such as transforming accumulated evidence into a discrete choice. To probe causally the role of FOF activity, we optogenetically silenced it during different time points of the trial. Consistent with a role in committing to a categorical choice at the end of the evidence accumulation process, but not consistent with a role during the accumulation itself, a behavioural effect was observed only when FOF silencing occurred at the end of the perceptual stimulus. Our results place important constraints on the circuit logic of brain regions involved in decision-making.

Cellular resolution functional imaging in behaving rats using voluntary head restraint

High-throughput operant conditioning systems for rodents provide efficient training on sophisticated behavioral tasks. Combining these systems with technologies for cellular resolution functional imaging would provide a powerful approach to study neural dynamics during behavior. Here we describe an integrated two-photon microscope and behavioral apparatus that allows cellular resolution functional imaging of cortical regions during epochs of voluntary head restraint. Rats were trained to initiate periods of restraint up to 8 s in duration, which provided the mechanical stability necessary for in vivo imaging while allowing free movement between behavioral trials. A mechanical registration system repositioned the head to within a few microns, allowing the same neuronal populations to be imaged on each trial. In proof-of-principle experiments, calcium-dependent fluorescence transients were recorded from GCaMP-labeled cortical neurons. In contrast to previous methods for head restraint, this system can be incorporated into high-throughput operant conditioning systems.

A low-frequency oscillatory neural signal in humans encodes a developing decision variable

We often make decisions based on sensory evidence that is accumulated over a period of time. How the evidence for such decisions is represented in the brain and how such a neural representation is used to guide a subsequent action are questions of considerable interest to decision sciences. The neural correlates of developing perceptual decisions have been thoroughly investigated in the oculomotor system of macaques who communicated their decisions using an eye movement. It has been found that the evidence informing a decision to make an eye movement is in part accumulated within the same oculomotor circuits that signal the upcoming eye movement. Recent evidence suggests that the somatomotor system may exhibit an analogous property for choices made using a hand movement. To investigate this possibility, we engaged humans in a decision task in which they integrated discrete quanta of sensory information over a period of time and signaled their decision using a hand movement or an eye movement. The discrete form of the sensory evidence allowed us to infer the decision variable on which subjects base their decision on each trial and to assess the neural processes related to each quantum of the incoming decision evidence. We found that a low-frequency electrophysiological signal recorded over centroparietal regions strongly encodes the decision variable inferred in this task, and that it does so specifically for hand movement choices. The signal ramps up with a rate that is proportional to the decision variable, remains graded by the decision variable throughout the delay period, reaches a common peak shortly before a hand movement, and falls off shortly after the hand movement. Furthermore, the signal encodes the polarity of each evidence quantum, with a short latency, and retains the response level over time. Thus, this neural signal shows properties of evidence accumulation. These findings suggest that the decision-related effects observed in the oculomotor system of the monkey during eye movement choices may share the same basic properties with the decision-related effects in the somatomotor system of humans during hand movement choices.

Rats and humans can optimally accumulate evidence for decision-making

The gradual and noisy accumulation of evidence is a fundamental component of decision-making, with noise playing a key role as the source of variability and errors. However, the origins of this noise have never been determined. We developed decision-making tasks in which sensory evidence is delivered in randomly timed pulses, and analyzed the resulting data with models that use the richly detailed information of each trial's pulse timing to distinguish between different decision-making mechanisms. This analysis allowed measurement of the magnitude of noise in the accumulator's memory, separately from noise associated with incoming sensory evidence. In our tasks, the accumulator's memory was noiseless, for both rats and humans. In contrast, the addition of new sensory evidence was the primary source of variability. We suggest our task and modeling approach as a powerful method for revealing internal properties of decision-making processes.

A cortical substrate for memory-guided orienting in the rat

Anatomical, stimulation, and lesion data have suggested a homology between the rat frontal orienting fields (FOF) (centered at +2 AP, ±1.3 ML mm from Bregma) and primate frontal cortices such as the frontal or supplementary eye fields. We investigated the functional role of the FOF using rats trained to perform a memory-guided orienting task, in which there was a delay period between the end of a sensory stimulus instructing orienting direction and the time of the allowed motor response. Unilateral inactivation of the FOF resulted in impaired contralateral responses. Extracellular recordings of single units revealed that 37% of FOF neurons had delay period firing rates that predicted the direction of the rats' later orienting motion. Our data provide the first electrophysiological and pharmacological evidence supporting the existence in the rat, as in the primate, of a frontal cortical area involved in the preparation and/or planning of orienting responses.

Minimal impairment in a rat model of duration discrimination following excitotoxic lesions of primary auditory and prefrontal cortices

We present a behavioral paradigm for the study of duration perception in the rat, and report the result of neurotoxic lesions that have the goal of identifying sites that mediate duration perception. Using a two-alternative forced-choice paradigm, rats were either trained to discriminate durations of pure tones (range = [200,500] ms; boundary = 316 ms; Weber fraction after training = 0.24 ± 0.04), or were trained to discriminate frequencies of pure tones (range = [8,16] kHz; boundary = 11.3 kHz; Weber = 0.16 ± 0.11); the latter task is a control for non-timing-specific aspects of the former. Both groups discriminate the same class of sensory stimuli, use the same motions to indicate decisions, have identical trial structures, and are trained to psychophysical threshold; the tasks are thus matched in a number of sensorimotor and cognitive demands. We made neurotoxic lesions of candidate timing-perception areas in the cerebral cortex of both groups. Following extensive bilateral lesions of the auditory cortex, the performance of the frequency discrimination group was significantly more impaired than that of the duration discrimination group. We also found that extensive bilateral lesions of the medial prefrontal cortex resulted in little to no impairment of both groups. The behavioral framework presented here provides an audition-based approach to study the neural mechanisms of time estimation and memory for durations.

Semi-automated atlas-based analysis of brain histological sections

Quantifying the location and/or number of features in a histological section of the brain currently requires one to first, manually register a corresponding section from a tissue atlas onto the experimental section and second, count the features. No automated method exists for the first process (registering), and most automated methods for the second process (feature counting) operate reliably only in a high signal-to-noise regime. To reduce experimenter bias and inconsistencies and increase the speed of these analyses, we developed Atlas Fitter, a semi-automated, open-source MatLab-based software package that assists in rapidly registering atlas panels onto histological sections. We also developed CellCounter, a novel fully automated cell counting algorithm that is designed to operate on images with non-uniform background intensities and low signal-to-noise ratios.

Human performance on the temporal bisection task

The perception and processing of temporal information are tasks the brain must continuously perform. These include measuring the duration of stimuli, storing duration information in memory, recalling such memories, and comparing two durations. How the brain accomplishes these tasks, however, is still open for debate. The temporal bisection task, which requires subjects to compare temporal stimuli to durations held in memory, is perfectly suited to address these questions. Here we perform a meta-analysis of human performance on the temporal bisection task collected from 148 experiments spread across 18 independent studies. With this expanded data set we are able to show that human performance on this task contains a number of significant peculiarities, which in total no single model yet proposed has been able to explain. Here we present a simple 2-step decision model that is capable of explaining all the idiosyncrasies seen in the data.

Design of continuous attractor networks with monotonic tuning using a symmetry principle

Neurons that sustain elevated firing in the absence of stimuli have been found in many neural systems. In graded persistent activity, neurons can sustain firing at many levels, suggesting a widely found type of network dynamics in which networks can relax to any one of a continuum of stationary states. The reproduction of these findings in model networks of nonlinear neurons has turned out to be nontrivial. A particularly insightful model has been the "bump attractor," in which a continuous attractor emerges through an underlying symmetry in the network connectivity matrix. This model, however, cannot account for data in which the persistent firing of neurons is a monotonic -- rather than a bell-shaped -- function of a stored variable. Here, we show that the symmetry used in the bump attractor network can be employed to create a whole family of continuous attractor networks, including those with monotonic tuning. Our design is based on tuning the external inputs to networks that have a connectivity matrix with Toeplitz symmetry. In particular, we provide a complete analytical solution of a line attractor network with monotonic tuning and show that for many other networks, the numerical tuning of synaptic weights reduces to the computation of a single parameter.

Neural codes for perceptual discrimination in primary somatosensory cortex

We sought to determine the neural code(s) for frequency discrimination of vibrotactile stimuli. We tested five possible candidate codes by analyzing the responses of single neurons recorded in primary somatosensory cortex of trained monkeys while they discriminated between two consecutive vibrotactile stimuli. Differences in the frequency of two stimuli could be discriminated using information from (i) time intervals between spikes, (ii) average spiking rate during each stimulus, (iii) absolute number of spikes elicited by each stimulus, (iv) average rate of bursts of spikes or (v) absolute number of spike bursts elicited by each stimulus. However, only a spike count code, in which spikes are integrated over a time window that has most of its mass in the first 250 ms of each stimulus period, covaried with behavior on a trial-by-trial basis, was consistent with psychophysical biases induced by manipulation of stimulus duration, and produced neurometric discrimination thresholds similar to behavioral psychophysical thresholds.

Flexible Control of Mutual Inhibition: A Neural Model of Two-Interval Discrimination

Networks adapt to environmental demands by switching between distinct dynamical behaviors. The activity of frontal-lobe neurons during two-interval discrimination tasks is an example of these adaptable dynamics. Subjects first perceive a stimulus, then hold it in working memory, and finally make a decision by comparing it with a second stimulus. We present a simple mutual-inhibition network model that captures all three task phases within a single framework. The model integrates both working memory and decision making because its dynamical properties are easily controlled without changing its connectivity. Mutual inhibition between nonlinear units is a useful design motif for networks that must display multiple behaviors.

Learning rules and network repair in spike-timing-based computation networks

Plasticity in connections between neurons allows learning and adaptation, but it also allows noise to degrade the function of a network. Ongoing network self-repair is thus necessary. We describe a method to derive spike-timing-dependent plasticity rules for self-repair, based on the firing patterns of a functioning network. These plasticity rules for self-repair also provide the basis for unsupervised learning of new tasks. The particular plasticity rule derived for a network depends on the network and task. Here, self-repair is illustrated for a model of the mammalian olfactory system in which the computational task is that of odor recognition. In this olfactory example, the derived rule has qualitative similarity with experimental results seen in spike-timing-dependent plasticity. Unsupervised learning of new tasks by using the derived self-repair rule is demonstrated by learning to recognize new odors.

A recurrent network model of somatosensory parametric working memory in the prefrontal cortex

A parametric working memory network stores the information of an analog stimulus in the form of persistent neural activity that is monotonically tuned to the stimulus. The family of persistent firing patterns with a continuous range of firing rates must all be realizable under exactly the same external conditions (during the delay when the transient stimulus is withdrawn). How this can be accomplished by neural mechanisms remains an unresolved question. Here we present a recurrent cortical network model of irregularly spiking neurons that was designed to simulate a somatosensory working memory experiment with behaving monkeys. Our model reproduces the observed positively and negatively monotonic persistent activity, and heterogeneous tuning curves of memory activity. We show that fine-tuning mathematically corresponds to a precise alignment of cusps in the bifurcation diagram of the network. Moreover, we show that the fine-tuned network can integrate stimulus inputs over several seconds. Assuming that such time integration occurs in neural populations downstream from a tonically persistent neural population, our model is able to account for the slow ramping-up and ramping-down behaviors of neurons observed in prefrontal cortex.