Comment on "Single-trial spike trains in parietal cortex reveal discrete steps during decision-making"

The centrality of population-level factors to network computation is demonstrated by a versatile approach for training spiking networks

Neural activity is often described in terms of population-level factors extracted from the responses of many neurons. Factors provide a lower-dimensional description with the aim of shedding light on network computations. Yet, mechanistically, computations are performed not by continuously valued factors but by interactions among neurons that spike discretely and variably. Models provide a means of bridging these levels of description. We developed a general method for training model networks of spiking neurons by leveraging factors extracted from either data or firing-rate-based networks. In addition to providing a useful model-building framework, this formalism illustrates how reliable and continuously valued factors can arise from seemingly stochastic spiking. Our framework establishes procedures for embedding this property in network models with different levels of realism. The relationship between spikes and factors in such networks provides a foundation for interpreting (and subtly redefining) commonly used quantities such as firing rates.

Allocation of Visuospatial Attention Indexes Evidence Accumulation for Reach Decisions

Visuospatial attention is a prerequisite for the performance of visually guided movements: perceptual discrimination is regularly enhanced at target locations before movement initiation. It is known that this attentional prioritization evolves over the time of movement preparation; however, it is not clear whether this build-up simply reflects a time requirement of attention formation or whether, instead, attention build-up reflects the emergence of the movement decision. To address this question, we combined behavioral experiments, psychophysics, and computational decision-making models to characterize the time course of attention build-up during motor preparation. Participants (n = 46, 29 female) executed center-out reaches to one of two potential target locations and reported the identity of a visual discrimination target (DT) that occurred concurrently at one of various time-points during movement preparation and execution. Visual discrimination increased simultaneously at the two potential target locations but was modulated by the experiment-wide probability that a given location would become the final goal. Attention increased further for the location that was then designated as the final goal location, with a time course closely related to movement initiation. A sequential sampling model of decision-making faithfully predicted key temporal characteristics of attentional allocation. Together, these findings provide evidence that visuospatial attentional prioritization during motor preparation does not simply reflect that a spatial location has been selected as movement goal, but rather indexes the time-extended, cumulative decision that leads to the selection, hence constituting a link between perceptual and motor aspects of sensorimotor decisions.

Slowly evolving dopaminergic activity modulates the moment-to-moment probability of reward-related self-timed movements

Clues from human movement disorders have long suggested that the neurotransmitter dopamine plays a role in motor control, but how the endogenous dopaminergic system influences movement is unknown. Here we examined the relationship between dopaminergic signaling and the timing of reward-related movements in mice. Animals were trained to initiate licking after a self-timed interval following a start-timing cue; reward was delivered in response to movements initiated after a criterion time. The movement time was variable from trial-to-trial, as expected from previous studies. Surprisingly, dopaminergic signals ramped-up over seconds between the start-timing cue and the self-timed movement, with variable dynamics that predicted the movement/reward time on single trials. Steeply rising signals preceded early lick-initiation, whereas slowly rising signals preceded later initiation. Higher baseline signals also predicted earlier self-timed movements. Optogenetic activation of dopamine neurons during self-timing did not trigger immediate movements, but rather caused systematic early-shifting of movement initiation, whereas inhibition caused late-shifting, as if modulating the probability of movement. Consistent with this view, the dynamics of the endogenous dopaminergic signals quantitatively predicted the moment-by-moment probability of movement initiation on single trials. We propose that ramping dopaminergic signals, likely encoding dynamic reward expectation, can modulate the decision of when to move.

Decision prioritization and causal reasoning in decision hierarchies

From cooking a meal to finding a route to a destination, many real life decisions can be decomposed into a hierarchy of sub-decisions. In a hierarchy, choosing which decision to think about requires planning over a potentially vast space of possible decision sequences. To gain insight into how people decide what to decide on, we studied a novel task that combines perceptual decision making, active sensing and hierarchical and counterfactual reasoning. Human participants had to find a target hidden at the lowest level of a decision tree. They could solicit information from the different nodes of the decision tree to gather noisy evidence about the target's location. Feedback was given only after errors at the leaf nodes and provided ambiguous evidence about the cause of the error. Despite the complexity of task (with 107 latent states) participants were able to plan efficiently in the task. A computational model of this process identified a small number of heuristics of low computational complexity that accounted for human behavior. These heuristics include making categorical decisions at the branching points of the decision tree rather than carrying forward entire probability distributions, discarding sensory evidence deemed unreliable to make a choice, and using choice confidence to infer the cause of the error after an initial plan failed. Plans based on probabilistic inference or myopic sampling norms could not capture participants' behavior. Our results show that it is possible to identify hallmarks of heuristic planning with sensing in human behavior and that the use of tasks of intermediate complexity helps identify the rules underlying human ability to reason over decision hierarchies.

The population doctrine in cognitive neuroscience

A major shift is happening within neurophysiology: a population doctrine is drawing level with the single-neuron doctrine that has long dominated the field. Population-level ideas have so far had their greatest impact in motor neuroscience, but they hold great promise for resolving open questions in cognition as well. Here, we codify the population doctrine and survey recent work that leverages this view to specifically probe cognition. Our discussion is organized around five core concepts that provide a foundation for population-level thinking: (1) state spaces, (2) manifolds, (3) coding dimensions, (4) subspaces, and (5) dynamics. The work we review illustrates the progress and promise that population-level thinking holds for cognitive neuroscience-for delivering new insight into attention, working memory, decision-making, executive function, learning, and reward processing.

Buildup and bistability in auditory streaming as an evidence accumulation process with saturation

A repeating triplet-sequence ABA- of non-overlapping brief tones, A and B, is a valued paradigm for studying auditory stream formation and the cocktail party problem. The stimulus is "heard" either as a galloping pattern (integration) or as two interleaved streams (segregation); the initial percept is typically integration then followed by spontaneous alternations between segregation and integration, each being dominant for a few seconds. The probability of segregation grows over seconds, from near-zero to a steady value, defining the buildup function, BUF. Its stationary level increases with the difference in tone frequencies, DF, and the BUF rises faster. Percept durations have DF-dependent means and are gamma-like distributed. Behavioral and computational studies usually characterize triplet streaming either during alternations or during buildup. Here, our experimental design and modeling encompass both. We propose a pseudo-neuromechanistic model that incorporates spiking activity in primary auditory cortex, A1, as input and resolves perception along two network-layers downstream of A1. Our model is straightforward and intuitive. It describes the noisy accumulation of evidence against the current percept which generates switches when reaching a threshold. Accumulation can saturate either above or below threshold; if below, the switching dynamics resemble noise-induced transitions from an attractor state. Our model accounts quantitatively for three key features of data: the BUFs, mean durations, and normalized dominance duration distributions, at various DF values. It describes perceptual alternations without competition per se, and underscores that treating triplets in the sequence independently and averaging across trials, as implemented in earlier widely cited studies, is inadequate.

Cerebellar Neurodynamics Predict Decision Timing and Outcome on the Single-Trial Level

Goal-directed behavior requires the interaction of multiple brain regions. How these regions and their interactions with brain-wide activity drive action selection is less understood. We have investigated this question by combining whole-brain volumetric calcium imaging using light-field microscopy and an operant-conditioning task in larval zebrafish. We find global, recurring dynamics of brain states to exhibit pre-motor bifurcations toward mutually exclusive decision outcomes. These dynamics arise from a distributed network displaying trial-by-trial functional connectivity changes, especially between cerebellum and habenula, which correlate with decision outcome. Within this network the cerebellum shows particularly strong and predictive pre-motor activity (>10 s before movement initiation), mainly within the granule cells. Turn directions are determined by the difference neuroactivity between the ipsilateral and contralateral hemispheres, while the rate of bi-hemispheric population ramping quantitatively predicts decision time on the trial-by-trial level. Our results highlight a cognitive role of the cerebellum and its importance in motor planning.

Accumulators, Neurons, and Response Time

The marriage of cognitive neurophysiology and mathematical psychology to understand decision-making has been exceptionally productive. This interdisciplinary area is based on the proposition that particular neurons or circuits instantiate the accumulation of evidence specified by mathematical models of sequential sampling and stochastic accumulation. This linking proposition has earned widespread endorsement. Here, a brief survey of the history of the proposition precedes a review of multiple conundrums and paradoxes concerning the accuracy, precision, and transparency of that linking proposition. Correctly establishing how abstract models of decision-making are instantiated by particular neural circuits would represent a remarkable accomplishment in mapping mind to brain. Failing would reveal challenging limits for cognitive neuroscience. This is such a vigorous area of research because so much is at stake.

Performance of normative and approximate evidence accumulation on the dynamic clicks task

The aim of a number of psychophysics tasks is to uncover how mammals make decisions in a world that is in flux. Here we examine the characteristics of ideal and near-ideal observers in a task of this type. We ask when and how performance depends on task parameters and design, and, in turn, what observer performance tells us about their decision-making process. In the dynamic clicks task subjects hear two streams (left and right) of Poisson clicks with different rates. Subjects are rewarded when they correctly identify the side with the higher rate, as this side switches unpredictably. We show that a reduced set of task parameters defines regions in parameter space in which optimal, but not near-optimal observers, maintain constant response accuracy. We also show that for a range of task parameters an approximate normative model must be finely tuned to reach near-optimal performance, illustrating a potential way to distinguish between normative models and their approximations. In addition, we show that using the negative log-likelihood and the 0/1-loss functions to fit these types of models is not equivalent: the 0/1-loss leads to a bias in parameter recovery that increases with sensory noise. These findings suggest ways to tease apart models that are hard to distinguish when tuned exactly, and point to general pitfalls in experimental design, model fitting, and interpretation of the resulting data.

Discrete Stepping and Nonlinear Ramping Dynamics Underlie Spiking Responses of LIP Neurons during Decision-Making

Neurons in LIP exhibit ramping trial-averaged responses during decision-making. Recent work sparked debate over whether single-trial LIP spike trains are better described by discrete "stepping" or continuous "ramping" dynamics. We extended latent dynamical spike train models and used Bayesian model comparison to address this controversy. First, we incorporated non-Poisson spiking into both models and found that more neurons were better described by stepping than ramping, even when conditioned on evidence or choice. Second, we extended the ramping model to include a non-zero baseline and compressive output nonlinearity. This model accounted for roughly as many neurons as the stepping model. However, latent dynamics inferred under this model exhibited high diffusion variance for many neurons, softening the distinction between continuous and discrete dynamics. Results generalized to additional datasets, demonstrating that substantial fractions of neurons are well described by either stepping or nonlinear ramping, which may be less categorically distinct than the original labels implied.

Behavioural and neural signatures of perceptual decision-making are modulated by pupil-linked arousal

The timing and accuracy of perceptual decision-making is exquisitely sensitive to fluctuations in arousal. Although extensive research has highlighted the role of various neural processing stages in forming decisions, our understanding of how arousal impacts these processes remains limited. Here we isolated electrophysiological signatures of decision-making alongside signals reflecting target selection, attentional engagement and motor output and examined their modulation as a function of tonic and phasic arousal, indexed by baseline and task-evoked pupil diameter, respectively. Reaction times were shorter on trials with lower tonic, and higher phasic arousal. Additionally, these two pupil measures were predictive of a unique set of EEG signatures that together represent multiple information processing steps of decision-making. Finally, behavioural variability associated with fluctuations in tonic and phasic arousal, indicative of neuromodulators acting on multiple timescales, was mediated by its effects on the EEG markers of attentional engagement, sensory processing and the variability in decision processing.

Driving along a busy street requires you to constantly monitor the behavior of other road users. You need to be able to spot and avoid the car that suddenly changes lane, or the pedestrian who steps out in front of you. How fast you can react to such events depends in part on your brain's level of alertness, or 'arousal'. This in turn depends on chemicals within the brain called neuromodulators.

Neuromodulators are a type of neurotransmitter. But whereas other neurotransmitters enable brain cells to signal to each other, neuromodulators turn the volume of these signals up or down. The activity of brain regions that produce neuromodulators varies over time, leading to changes in brain arousal. These changes take place over different time scales. Sudden unexpected events, such as those on the busy street above, trigger sub-second changes in arousal. But arousal levels also show spontaneous fluctuations over minutes to hours. We can follow these changes in real-time by looking into a participant’s eyes. This is because the brain regions that produce neuromodulators also control pupil size.

Van Kempen et al. have now combined measurements of pupil size with recordings of electrical brain activity. Healthy volunteers learned to press a button as soon as a target appeared on a screen. The larger a volunteer’s pupils were before the target appeared, the more slowly the volunteer responded on that trial. Large baseline pupil size is thought to indicate a high baseline level of brain arousal. By contrast, the larger the increase in pupil size in response to the target, the faster the volunteer responded on that trial. This increase in pupil size is thought to reflect an increase in brain arousal.

The recordings of brain activity provided clues to the underlying mechanisms. In trials with large baseline pupil size – and therefore high baseline arousal – the volunteers’ brains showed more variable responses to the target. But in trials with a large increase in pupil size – and a large increase in arousal – the volunteers’ brains showed less variable responses, as well as stronger signals related to attention.

Neuromodulators thus act on different timescales to influence different aspects of cognitive performance, including attention and target detection. Fluctuating levels of neuromodulator activity may help explain the variability in our behavior. Monitoring pupil size is one way to gain insights into the mechanisms that bring about these changes in neuromodulator activity.

Bridging Neural and Computational Viewpoints on Perceptual Decision-Making

Sequential sampling models have provided a dominant theoretical framework guiding computational and neurophysiological investigations of perceptual decision-making. While these models share the basic principle that decisions are formed by accumulating sensory evidence to a bound, they come in many forms that can make similar predictions of choice behaviour despite invoking fundamentally different mechanisms. The identification of neural signals that reflect some of the core computations underpinning decision formation offers new avenues for empirically testing and refining key model assumptions. Here, we highlight recent efforts to explore these avenues and, in so doing, consider the conceptual and methodological challenges that arise when seeking to infer decision computations from complex neural data.

On-line confidence monitoring during decision making

Humans can readily assess their degree of confidence in their decisions. Two models of confidence computation have been proposed: post hoc computation using post-decision variables and heuristics, versus online computation using continuous assessment of evidence throughout the decision-making process. Here, we arbitrate between these theories by continuously monitoring finger movements during a manual sequential decision-making task. Analysis of finger kinematics indicated that subjects kept separate online records of evidence and confidence: finger deviation continuously reflected the ongoing accumulation of evidence, whereas finger speed continuously reflected the momentary degree of confidence. Furthermore, end-of-trial finger speed predicted the post-decisional subjective confidence rating. These data indicate that confidence is computed on-line, throughout the decision process. Speed-confidence correlations were previously interpreted as a post-decision heuristics, whereby slow decisions decrease subjective confidence, but our results suggest an adaptive mechanism that involves the opposite causality: by slowing down when unconfident, participants gain time to improve their decisions.

What can neuronal populations tell us about cognition?

Nowadays, it is possible to record the activity of hundreds of cells at the same time in behaving animals. However, these data are often treated and analyzed as if they consisted of many independently recorded neurons. How can neuronal populations be uniquely used to learn about cognition? We describe recent work that shows that populations of simultaneously recorded neurons are fundamental to understand the basis of decision-making, including processes such as ongoing deliberations and decision confidence, which generally fall outside the reach of single-cell analysis. Thus, neuronal population data allow addressing novel questions, but they also come with so far unsolved challenges.

A Two-Stage Process Model of Sensory Discrimination: An Alternative to Drift-Diffusion

Discrimination of the direction of motion of a noisy stimulus is an example of sensory discrimination under uncertainty. For stimuli that are extended in time, reaction time is quicker for larger signal values (e.g., discrimination of opposite directions of motion compared with neighboring orientations) and larger signal strength (e.g., stimuli with higher contrast or motion coherence, that is, lower noise). The standard model of neural responses (e.g., in lateral intraparietal cortex) and reaction time for discrimination is drift-diffusion. This model makes two clear predictions. (1) The effects of signal strength and value on reaction time should interact multiplicatively because the diffusion process depends on the signal-to-noise ratio. (2) If the diffusion process is interrupted, as in a cued-response task, the time to decision after the cue should be independent of the strength of accumulated sensory evidence. In two experiments with human participants, we show that neither prediction holds. A simple alternative model is developed that is consistent with the results. In this estimate-then-decide model, evidence is accumulated until estimation precision reaches a threshold value. Then, a decision is made with duration that depends on the signal-to-noise ratio achieved by the first stage.

SIGNIFICANCE STATEMENT

Sensory decision-making under uncertainty is usually modeled as the slow accumulation of noisy sensory evidence until a threshold amount of evidence supporting one of the possible decision outcomes is reached. Furthermore, it has been suggested that this accumulation process is reflected in neural responses, e.g., in lateral intraparietal cortex. We derive two behavioral predictions of this model and show that neither prediction holds. We introduce a simple alternative model in which evidence is accumulated until a sufficiently precise estimate of the stimulus is achieved, and then that estimate is used to guide the discrimination decision. This model is consistent with the behavioral data.

Striatal prediction errors support dynamic control of declarative memory decisions

Adaptive memory requires context-dependent control over how information is retrieved, evaluated and used to guide action, yet the signals that drive adjustments to memory decisions remain unknown. Here we show that prediction errors (PEs) coded by the striatum support control over memory decisions. Human participants completed a recognition memory test that incorporated biased feedback to influence participants' recognition criterion. Using model-based fMRI, we find that PEs-the deviation between the outcome and expected value of a memory decision-correlate with striatal activity and predict individuals' final criterion. Importantly, the striatal PEs are scaled relative to memory strength rather than the expected trial outcome. Follow-up experiments show that the learned recognition criterion transfers to free recall, and targeting biased feedback to experimentally manipulate the magnitude of PEs influences criterion consistent with PEs scaled relative to memory strength. This provides convergent evidence that declarative memory decisions can be regulated via striatally mediated reinforcement learning signals.

Three challenges for connecting model to mechanism in decision-making

Recent years have seen a growing interest in understanding the neural mechanisms that support decision-making. The advent of new tools for measuring and manipulating neurons, alongside the inclusion of multiple new animal models and sensory systems has led to the generation of many novel datasets. The potential for these new approaches to constrain decision-making models is unprecedented. Here, we argue that to fully leverage these new approaches, three challenges must be met. First, experimenters must design well-controlled behavioral experiments that make it possible to distinguish competing behavioral strategies. Second, analyses of neural responses should think beyond single neurons, taking into account tradeoffs of single-trial versus trial-averaged approaches. Finally, quantitative model comparisons should be used, but must consider common obstacles.

Response to Comment on "Single-trial spike trains in parietal cortex reveal discrete steps during decision-making"

Shadlen et al's Comment focuses on extrapolations of our results that were not implied or asserted in our Report. They discuss alternate analyses of average firing rates in other tasks, the relationship between neural activity and behavior, and possible extensions of the standard models we examined. Although interesting to contemplate, these points are not germane to the findings of our Report: that stepping dynamics provided a better statistical description of lateral intraparietal area spike trains than diffusion-to-bound dynamics for a majority of neurons.