Neurally constrained modeling of perceptual decision making

The speed-accuracy tradeoff: history, physiology, methodology, and behavior

There are few behavioral effects as ubiquitous as the speed-accuracy tradeoff (SAT). From insects to rodents to primates, the tendency for decision speed to covary with decision accuracy seems an inescapable property of choice behavior. Recently, the SAT has received renewed interest, as neuroscience approaches begin to uncover its neural underpinnings and computational models are compelled to incorporate it as a necessary benchmark. The present work provides a comprehensive overview of SAT. First, I trace its history as a tractable behavioral phenomenon and the role it has played in shaping mathematical descriptions of the decision process. Second, I present a "users guide" of SAT methodology, including a critical review of common experimental manipulations and analysis techniques and a treatment of the typical behavioral patterns that emerge when SAT is manipulated directly. Finally, I review applications of this methodology in several domains.

Stochastic accumulation of feature information in perception and memory

It is now well established that the time course of perceptual processing influences the first second or so of performance in a wide variety of cognitive tasks. Over the last 20 years, there has been a shift from modeling the speed at which a display is processed, to modeling the speed at which different features of the display are perceived and formalizing how this perceptual information is used in decision making. The first of these models (Lamberts, 1995) was implemented to fit the time course of performance in a speeded perceptual categorization task and assumed a simple stochastic accumulation of feature information. Subsequently, similar approaches have been used to model performance in a range of cognitive tasks including identification, absolute identification, perceptual matching, recognition, visual search, and word processing, again assuming a simple stochastic accumulation of feature information from both the stimulus and representations held in memory. These models are typically fit to data from signal-to-respond experiments whereby the effects of stimulus exposure duration on performance are examined, but response times (RTs) and RT distributions have also been modeled. In this article, we review this approach and explore the insights it has provided about the interplay between perceptual processing, memory retrieval, and decision making in a variety of tasks. In so doing, we highlight how such approaches can continue to usefully contribute to our understanding of cognition.

Variability in visual working memory ability limits the efficiency of perceptual decision making

The ability to make rapid and accurate decisions based on limited sensory information is a critical component of visual cognition. Available evidence suggests that simple perceptual discriminations are based on the accumulation and integration of sensory evidence over time. However, the memory system(s) mediating this accumulation are unclear. One candidate system is working memory (WM), which enables the temporary maintenance of information in a readily accessible state. Here, we show that individual variability in WM capacity is strongly correlated with the speed of evidence accumulation in speeded two-alternative forced choice tasks. This relationship generalized across different decision-making tasks, and could not be easily explained by variability in general arousal or vigilance. Moreover, we show that performing a difficult discrimination task while maintaining a concurrent memory load has a deleterious effect on the latter, suggesting that WM storage and decision making are directly linked.

Lateralized readiness potentials reveal properties of a neural mechanism for implementing a decision threshold

Many perceptual decision making models posit that participants accumulate noisy evidence over time to improve the accuracy of their decisions, and that in free response tasks, participants respond when the accumulated evidence reaches a decision threshold. Research on the neural correlates of these models' components focuses primarily on evidence accumulation. Far less attention has been paid to the neural correlates of decision thresholds, reflecting the final commitment to a decision. Inspired by a model of bistable neural activity that implements a decision threshold, we reinterpret human lateralized readiness potentials (LRPs) as reflecting the crossing of a decision threshold. Interestingly, this threshold crossing preserves signatures of a drift-diffusion process of evidence accumulation that feeds in to the threshold mechanism. We show that, as our model predicts, LRP amplitudes and growth rates recorded while participants performed a motion discrimination task correlate with individual differences in behaviorally-estimated prior beliefs, decision thresholds and evidence accumulation rates. As such LRPs provide a useful measure to test dynamical models of both evidence accumulation and decision commitment processes non-invasively.

Humans optimize decision-making by delaying decision onset

Why do humans make errors on seemingly trivial perceptual decisions? It has been shown that such errors occur in part because the decision process (evidence accumulation) is initiated before selective attention has isolated the relevant sensory information from salient distractors. Nevertheless, it is typically assumed that subjects increase accuracy by prolonging the decision process rather than delaying decision onset. To date it has not been tested whether humans can strategically delay decision onset to increase response accuracy. To address this question we measured the time course of selective attention in a motion interference task using a novel variant of the response signal paradigm. Based on these measurements we estimated time-dependent drift rate and showed that subjects should in principle be able trade speed for accuracy very effectively by delaying decision onset. Using the time-dependent estimate of drift rate we show that subjects indeed delay decision onset in addition to raising response threshold when asked to stress accuracy over speed in a free reaction version of the same motion-interference task. These findings show that decision onset is a critical aspect of the decision process that can be adjusted to effectively improve decision accuracy.

Spatial patterns of fixation-switch behavior in strabismic monkeys

PURPOSE

Patients with strabismus perceptually suppress information from one eye to avoid double vision. Mechanisms of visual suppression likely lead to fixation-switch behavior wherein the subject acquires targets with a specific eye depending on target location in space. The purpose of this study was to investigate spatial patterns of fixation-switch behavior in strabismic monkeys.

METHODS

Eye movements were acquired in three exotropic and one esotropic monkey in a binocular viewing saccade task. Spatial patterns of fixation were analyzed by calculating incidence of using either eye to fixate targets presented at various gaze locations.

RESULTS

Broadly, spatial fixation patterns and fixation-switch behavior followed expectations if a portion of the temporal retina was suppressed in exotropia and a portion of the nasal retina was suppressed in esotropia. Fixation-switch occurred for horizontal target locations that were approximately greater than halfway between the lines of sight of the foveating and strabismic eyes. Surprisingly, the border between right eye and left eye fixation zones was not sharply defined and there was a significant extent (>10°) over which the monkeys could acquire a target with either eye.

CONCLUSIONS

We propose that spatial fixation patterns in strabismus can be accounted for in a decision framework wherein the oculomotor system has access to retinal error information from each eye and the brain chooses between them to prepare a saccade. For target locations approximately midway between the two foveae, strength of retinal error representations from each eye is almost equal, leading to trial-to-trial variability in choice of fixating eye.

Perceptual decision making: drift-diffusion model is equivalent to a Bayesian model

Behavioral data obtained with perceptual decision making experiments are typically analyzed with the drift-diffusion model. This parsimonious model accumulates noisy pieces of evidence toward a decision bound to explain the accuracy and reaction times of subjects. Recently, Bayesian models have been proposed to explain how the brain extracts information from noisy input as typically presented in perceptual decision making tasks. It has long been known that the drift-diffusion model is tightly linked with such functional Bayesian models but the precise relationship of the two mechanisms was never made explicit. Using a Bayesian model, we derived the equations which relate parameter values between these models. In practice we show that this equivalence is useful when fitting multi-subject data. We further show that the Bayesian model suggests different decision variables which all predict equal responses and discuss how these may be discriminated based on neural correlates of accumulated evidence. In addition, we discuss extensions to the Bayesian model which would be difficult to derive for the drift-diffusion model. We suggest that these and other extensions may be highly useful for deriving new experiments which test novel hypotheses.

A unified comparison of stimulus-driven, endogenous mandatory and 'free choice' saccades

It has been claimed that saccades arising from the three saccade triggering modes–stimulus-driven, endogenous mandatory and ‘free choice’–are driven by distinct mechanisms. We tested this claim by instructing observers to saccade from a white or black fixation disc to a same polarity (white or black) disc flashed for 100 or 200 ms presented either alone (Exo), or together with an opposite (Endo) or same (EndoFC) polarity disc (blocked and mixed sessions). Target(s) and distractor were presented at three inter-stimulus intervals (ISIs) relative to the fixation offset (ISI: −200, 0, +200 ms) and were displayed at random locations within a 4°-to-6° eccentricity range. The statistical analysis showed a global saccade triggering mode effect on saccade reaction times (SRTs) with Endo and EndoFC SRTs longer by about 27 ms than Exo-triggered ones but no effect for the Endo-EndoFC comparison. SRTs depended on both ISI (the “gap-effect”), and target duration. Bimodal best fits of the SRT-distributions were found in 65% of cases with their count not different across the three triggering modes. Percentages of saccades in the ‘fast’ and ‘slow’ ranges of bimodal fits did not depend on the triggering modes either. Bimodality tests failed to assert a significant difference between these modes. An analysis of the timing of a putative inhibition by the distractor (Endo) or by the duplicated target (EndoFC) yielded no significant difference between Endo and EndoFC saccades but showed a significant shortening with ISI similar to the SRT shortening suggesting that the distractor-target mutual inhibition is itself inhibited by ‘fixation’ neurons. While other experimental paradigms may well sustain claims of distinct mechanisms subtending the three saccade triggering modes, as here defined reflexive and voluntary saccades appear to differ primarily in the effectiveness with which inhibitory processes slow down the initial fast rise of the saccade triggering signal.

Response times from ensembles of accumulators

Decision-making is explained by psychologists through stochastic accumulator models and by neurophysiologists through the activity of neurons believed to instantiate these models. We investigated an overlooked scaling problem: How does a response time (RT) that can be explained by a single model accumulator arise from numerous, redundant accumulator neurons, each of which individually appears to explain the variability of RT? We explored this scaling problem by developing a unique ensemble model of RT, called e pluribus unum, which embodies the well-known dictum "out of many, one." We used the e pluribus unum model to analyze the RTs produced by ensembles of redundant, idiosyncratic stochastic accumulators under various termination mechanisms and accumulation rate correlations in computer simulations of ensembles of varying size. We found that predicted RT distributions are largely invariant to ensemble size if the accumulators share at least modestly correlated accumulation rates and RT is not governed by the most extreme accumulators. Under these regimes the termination times of individual accumulators was predictive of ensemble RT. We also found that the threshold measured on individual accumulators, corresponding to the firing rate of neurons measured at RT, can be invariant with RT but is equivalent to the specified model threshold only when the rate correlation is very high.

Decision making as a window on cognition

A decision is a commitment to a proposition or plan of action based on information and values associated with the possible outcomes. The process operates in a flexible timeframe that is free from the immediacy of evidence acquisition and the real time demands of action itself. Thus, it involves deliberation, planning, and strategizing. This Perspective focuses on perceptual decision making in nonhuman primates and the discovery of neural mechanisms that support accuracy, speed, and confidence in a decision. We suggest that these mechanisms expose principles of cognitive function in general, and we speculate about the challenges and directions before the field.

Integration of direction cues is invariant to the temporal gap between them

Many decisions involve integration of evidence conferred by discrete cues over time. However, the neural mechanism of this integration is poorly understood. Several decision-making models suggest that integration of evidence is implemented by a dynamic system whose state evolves toward a stable point representing the decision outcome. The internal dynamics of such point attractor models render them sensitive to the temporal gaps between cues because their internal forces push the state forward once it is dislodged from the initial stable point. We asked whether human subjects are as sensitive to such temporal gaps. Subjects reported the net direction of stochastic random dot motion, which was presented in one or two brief observation windows (pulses). Pulse strength and interpulse interval varied randomly from trial to trial. We found that subjects' performance was largely invariant to the interpulse intervals up to at least 1 s. The findings question the implementation of the integration process via mechanisms that rely on autonomous changes of network state. The mechanism should be capable of freezing the state of the network at a variety of firing rate levels during temporal gaps between the cues, compatible with a line of stable attractor states.

Mechanisms underlying the influence of saliency on value-based decisions

Objects in the environment differ in their low-level perceptual properties (e.g., how easily a fruit can be recognized) as well as in their subjective value (how tasty it is). We studied the influence of visual salience on value-based decisions using a two alternative forced choice task, in which human subjects rapidly chose items from a visual display. All targets were equally easy to detect. Nevertheless, both value and salience strongly affected choices made and reaction times. We analyzed the neuronal mechanisms underlying these behavioral effects using stochastic accumulator models, allowing us to characterize not only the averages of reaction times but their full distributions. Independent models without interaction between the possible choices failed to reproduce the observed choice behavior, while models with mutual inhibition between alternative choices produced much better results. Mutual inhibition thus is an important feature of the decision mechanism. Value influenced the amount of accumulation in all models. In contrast, increased salience could either lead to an earlier start (onset model) or to a higher rate (speed model) of accumulation. Both models explained the data from the choice trials equally well. However, salience also affected reaction times in no-choice trials in which only one item was present, as well as error trials. Only the onset model could explain the observed reaction time distributions of error trials and no-choice trials. In contrast, the speed model could not, irrespective of whether the rate increase resulted from more frequent accumulated quanta or from larger quanta. Visual salience thus likely provides an advantage in the onset, not in the processing speed, of value-based decision making.

Production, control, and visual guidance of saccadic eye movements

Primate vision is served by rapid shifts of gaze called saccades. This review will survey current knowledge and particular problems concerning the neural control and guidance of gaze shifts.

Spatial frequency information modulates response inhibition and decision-making processes

We interact with the world through the assessment of available, but sometimes imperfect, sensory information. However, little is known about how variance in the quality of sensory information affects the regulation of controlled actions. In a series of three experiments, comprising a total of seven behavioral studies, we examined how different types of spatial frequency information affect underlying processes of response inhibition and selection. Participants underwent a stop-signal task, a two choice speed/accuracy balance experiment, and a variant of both these tasks where prior information was given about the nature of stimuli. In all experiments, stimuli were either intact, or contained only high-, or low- spatial frequencies. Overall, drift diffusion model analysis showed a decreased rate of information processing when spatial frequencies were removed, whereas the criterion for information accumulation was lowered. When spatial frequency information was intact, the cost of response inhibition increased (longer SSRT), while a correct response was produced faster (shorter reaction times) and with more certainty (decreased errors). When we manipulated the motivation to respond with a deadline (i.e., be fast or accurate), removal of spatial frequency information slowed response times only when instructions emphasized accuracy. However, the slowing of response times did not improve error rates, when compared to fast instruction trials. These behavioral studies suggest that the removal of spatial frequency information differentially affects the speed of response initiation, inhibition, and the efficiency to balance fast or accurate responses. More generally, the present results indicate a task-independent influence of basic sensory information on strategic adjustments in action control.

Neural chronometry and coherency across speed-accuracy demands reveal lack of homomorphism between computational and neural mechanisms of evidence accumulation

The stochastic accumulation framework provides a mechanistic, quantitative account of perceptual decision-making and how task performance changes with experimental manipulations. Importantly, it provides an elegant account of the speed-accuracy trade-off (SAT), which has long been the litmus test for decision models, and also mimics the activity of single neurons in several key respects. Recently, we developed a paradigm whereby macaque monkeys trade speed for accuracy on cue during visual search task. Single-unit activity in frontal eye field (FEF) was not homomorphic with the architecture of models, demonstrating that stochastic accumulators are an incomplete description of neural activity under SAT. This paper summarizes and extends this work, further demonstrating that the SAT leads to extensive, widespread changes in brain activity never before predicted. We will begin by reviewing our recently published work that establishes how spiking activity in FEF accomplishes SAT. Next, we provide two important extensions of this work. First, we report a new chronometric analysis suggesting that increases in perceptual gain with speed stress are evident in FEF synaptic input, implicating afferent sensory-processing sources. Second, we report a new analysis demonstrating selective influence of SAT on frequency coupling between FEF neurons and local field potentials. None of these observations correspond to the mechanics of current accumulator models.

Decoding the brain's algorithm for categorization from its neural implementation

Acts of cognition can be described at different levels of analysis: what behavior should characterize the act, what algorithms and representations underlie the behavior, and how the algorithms are physically realized in neural activity [1]. Theories that bridge levels of analysis offer more complete explanations by leveraging the constraints present at each level [2-4]. Despite the great potential for theoretical advances, few studies of cognition bridge levels of analysis. For example, formal cognitive models of category decisions accurately predict human decision making [5, 6], but whether model algorithms and representations supporting category decisions are consistent with underlying neural implementation remains unknown. This uncertainty is largely due to the hurdle of forging links between theory and brain [7-9]. Here, we tackle this critical problem by using brain response to characterize the nature of mental computations that support category decisions to evaluate two dominant, and opposing, models of categorization. We found that brain states during category decisions were significantly more consistent with latent model representations from exemplar [5] rather than prototype theory [10, 11]. Representations of individual experiences, not the abstraction of experiences, are critical for category decision making. Holding models accountable for behavior and neural implementation provides a means for advancing more complete descriptions of the algorithms of cognition.

Saccade target selection relies on feedback competitive signal integration

It is often assumed that decision making involves neural competition, accumulation of evidence "scores" over time, and commitment to a particular alternative once its scores reach a critical decision threshold first. So far, however, neither the first-to-threshold rule nor the nature of competition (feedforward or feedback inhibition) has been revealed by experiments. Here, we presented two simultaneously flashed targets that reversed their intensity difference during each presentation and instructed human subjects to make a saccade toward the brightest target. All subjects preferentially chose the target that was brightest during the first stimulus phase. Unless this first phase lasted only 40 ms, this primacy effect persisted even if the second, reversed-intensity phase lasted longer. This effect did not result from premature commitment to the initially dominant target, because a strong target imbalance in the opposite direction later drove nearly all responses toward that location. Moreover, there was a nonmonotonic relation between target imbalance and primacy: increasing the target imbalance beyond 40 cd/m(2) caused an attenuation of primacy. These are the hallmarks of hysteresis, predicted by models in which target representations compete through strong feedback. Reaction times were independent of the choice probability. This dissociation suggests that target selection and movement initiation are distinct phenomena.

Separate evaluation of target facilitation and distractor suppression in the activity of macaque lateral intraparietal neurons during visual search

During visual search, neurons in the lateral intraparietal area (LIP) discriminate the target from distractors by exhibiting stronger activation when the target appears within the receptive field than when it appears outside the receptive field. It is generally thought that such target-discriminative activity is produced by the combination of target-related facilitation and distractor-related suppression. However, little is known about how the target-discriminative activity is constituted by these two types of neural modulation. To address this issue, we recorded activity from LIP of monkeys performing a visual search task that consisted of target-present and target-absent trials. Monkeys had to make a saccade to a target in the target-present trials, whereas they had to maintain fixation in the target-absent trials, in which only distractors were presented. By introducing the activity from the latter trials as neutral activity, we were able to separate the target-discriminative activity into target-related elevation and distractor-related reduction components. We found that the target-discriminative activity of most LIP neurons consisted of the combination of target-related elevation and distractor-related reduction or only target-related elevation. In contrast, target-discriminative activity composed of only distractor-related reduction was observed for very few neurons. We also found that, on average, target-related elevation was stronger and occurred earlier compared with distractor-related reduction. Finally, we consider possible underlying mechanisms, including lateral inhibitory interactions, responsible for target-discriminative activity in visual search. The present findings provide insight into how neuronal modulations shape target-discriminative activity during visual search.

Does linear separability really matter? Complex visual search is explained by simple search

Visual search in real life involves complex displays with a target among multiple types of distracters, but in the laboratory, it is often tested using simple displays with identical distracters. Can complex search be understood in terms of simple searches? This link may not be straightforward if complex search has emergent properties. One such property is linear separability, whereby search is hard when a target cannot be separated from its distracters using a single linear boundary. However, evidence in favor of linear separability is based on testing stimulus configurations in an external parametric space that need not be related to their true perceptual representation. We therefore set out to assess whether linear separability influences complex search at all. Our null hypothesis was that complex search performance depends only on classical factors such as target-distracter similarity and distracter homogeneity, which we measured using simple searches. Across three experiments involving a variety of artificial and natural objects, differences between linearly separable and nonseparable searches were explained using target-distracter similarity and distracter heterogeneity. Further, simple searches accurately predicted complex search regardless of linear separability (r = 0.91). Our results show that complex search is explained by simple search, refuting the widely held belief that linear separability influences visual search.

Brain rhythms in mental time travel

The memory theorist Endel Tulving referred to the ability to search through one's memories, and revisit events and episodes from one's past, as mental time travel. This process involves the reactivation of past mental states reflecting the perceptual and conceptual characteristics of the original experience. Widely distributed neural circuitry is engaged in the service of memory search, and the dynamics of these circuits are reflected in rhythmic oscillatory signals at widespread frequencies, recorded both in the local field around neurons and more globally at the scalp. Retrieved-context theory provides a theoretical bridge between the behavioral phenomena exhibited by participants in memory search tasks, and the neural signals reflecting the dynamics of the underlying circuitry. Computational models based on this theory make broad predictions regarding the representational structure of neural activity recorded during these tasks. In recent work, researchers have used multivariate analytic techniques on topographic patterns of oscillatory neural activity to confirm critical predictions of retrieved-context theory. We review the cognitive theory motivating this recent work, and the analytic techniques being developed to create integrated neural-behavioral models of human memory search.

Decision making and action implementation: evidence for an early visually triggered motor activation specific to potential actions

To make a decision may rely on accumulating evidence in favor of one alternative until a threshold is reached. Sequential-sampling models differ by the way of accumulating evidence and the link with action implementation. Here, we tested a model's prediction of an early action implementation specific to potential actions. We assessed the dynamics of action implementation in go/no-go and between-hand choice tasks by transcranial magnetic stimulation of the motor cortex (single- or paired-pulse TMS; 3-ms interstimulus interval). Prior to implementation of the selected action, the amplitude of the motor evoked potential first increased whatever the visual stimulus but only for the hand potentially involved in the to-be-produced action. These findings suggest that visual stimuli can trigger an early motor activation specific to potential actions, consistent with race-like models with continuous transmission between decision making and action implementation.

Accuracy and response-time distributions for decision-making: linear perfect integrators versus nonlinear attractor-based neural circuits

Animals choose actions based on imperfect, ambiguous data. “Noise” inherent in neural processing adds further variability to this already-noisy input signal. Mathematical analysis has suggested that the optimal apparatus (in terms of the speed/accuracy trade-off) for reaching decisions about such noisy inputs is perfect accumulation of the inputs by a temporal integrator. Thus, most highly cited models of neural circuitry underlying decision-making have been instantiations of a perfect integrator. Here, in accordance with a growing mathematical and empirical literature, we describe circumstances in which perfect integration is rendered suboptimal. In particular we highlight the impact of three biological constraints: (1) significant noise arising within the decision-making circuitry itself; (2) bounding of integration by maximal neural firing rates; and (3) time limitations on making a decision. Under conditions (1) and (2), an attractor system with stable attractor states can easily best an integrator when accuracy is more important than speed. Moreover, under conditions in which such stable attractor networks do not best the perfect integrator, a system with unstable initial states can do so if readout of the system’s final state is imperfect. Ubiquitously, an attractor system with a nonselective time-dependent input current is both more accurate and more robust to imprecise tuning of parameters than an integrator with such input. Given that neural responses that switch stochastically between discrete states can “masquerade” as integration in single-neuron and trial-averaged data, our results suggest that such networks should be considered as plausible alternatives to the integrator model.

Macrocircuits: decision networks

Decision-making requires stimulus categorization and localization to guide accurate responses that can be produced through multiple effectors. The success of actions is monitored so that performance can be adjusted to achieve goals. This review will survey recent empirical and theoretical developments very selectively with an emphasis on neurophysiological data from nonhuman primates that provide the clearest information about neural mechanisms.

Neural integrators for decision making: a favorable tradeoff between robustness and sensitivity

A key step in many perceptual decision tasks is the integration of sensory inputs over time, but a fundamental questions remain about how this is accomplished in neural circuits. One possibility is to balance decay modes of membranes and synapses with recurrent excitation. To allow integration over long timescales, however, this balance must be exceedingly precise. The need for fine tuning can be overcome via a “robust integrator” mechanism in which momentary inputs must be above a preset limit to be registered by the circuit. The degree of this limiting embodies a tradeoff between sensitivity to the input stream and robustness against parameter mistuning. Here, we analyze the consequences of this tradeoff for decision-making performance. For concreteness, we focus on the well-studied random dot motion discrimination task and constrain stimulus parameters by experimental data. We show that mistuning feedback in an integrator circuit decreases decision performance but that the robust integrator mechanism can limit this loss. Intriguingly, even for perfectly tuned circuits with no immediate need for a robustness mechanism, including one often does not impose a substantial penalty for decision-making performance. The implication is that robust integrators may be well suited to subserve the basic function of evidence integration in many cognitive tasks. We develop these ideas using simulations of coupled neural units and the mathematics of sequential analysis.

Neural mechanisms of speed-accuracy tradeoff

Intelligent agents balance speed of responding with accuracy of deciding. Stochastic accumulator models commonly explain this speed-accuracy tradeoff by strategic adjustment of response threshold. Several laboratories identify specific neurons in prefrontal and parietal cortex with this accumulation process, yet no neurophysiological correlates of speed-accuracy tradeoff have been described. We trained macaque monkeys to trade speed for accuracy on cue during visual search and recorded the activity of neurons in the frontal eye field. Unpredicted by any model, we discovered that speed-accuracy tradeoff is accomplished through several distinct adjustments. Visually responsive neurons modulated baseline firing rate, sensory gain, and the duration of perceptual processing. Movement neurons triggered responses with activity modulated in a direction opposite of model predictions. Thus, current stochastic accumulator models provide an incomplete description of the neural processes accomplishing speed-accuracy tradeoffs. The diversity of neural mechanisms was reconciled with the accumulator framework through an integrated accumulator model constrained by requirements of the motor system.

Changes in neural connectivity underlie decision threshold modulation for reward maximization

Using neuroimaging in combination with computational modeling, this study shows that decision threshold modulation for reward maximization is accompanied by a change in effective connectivity within corticostriatal and cerebellar-striatal brain systems. Research on perceptual decision making suggests that people make decisions by accumulating sensory evidence until a decision threshold is crossed. This threshold can be adjusted to changing circumstances, to maximize rewards. Decision making thus requires effectively managing the amount of accumulated evidence versus the amount of available time. Importantly, the neural substrate of this decision threshold modulation is unknown. Participants performed a perceptual decision-making task in blocks with identical duration but different reward schedules. Behavioral and modeling results indicate that human subjects modulated their decision threshold to maximize net reward. Neuroimaging results indicate that decision threshold modulation was achieved by adjusting effective connectivity within corticostriatal and cerebellar-striatal brain systems, the former being responsible for processing of accumulated sensory evidence and the latter being responsible for automatic, subsecond temporal processing. Participants who adjusted their threshold to a greater extent (and gained more net reward) also showed a greater modulation of effective connectivity. These results reveal a neural mechanism that underlies decision makers' abilities to adjust to changing circumstances to maximize reward.

Parameter variability and distributional assumptions in the diffusion model

If the diffusion model (Ratcliff & McKoon, 2008) is to account for the relative speeds of correct responses and errors, it is necessary that the components of processing identified by the model vary across the trials of a task. In standard applications, the rate at which information is accumulated by the diffusion process is assumed to be normally distributed across trials, the starting point for the process is assumed to be uniformly distributed across trials, and the time taken by processes outside the diffusion process is assumed to be uniformly distributed. With the studies in this article, I explore the consequences of alternative assumptions about the distributions, using a wide range of parameter values. The model with the standard assumptions was fit to predictions generated with the alternative assumptions, and the results showed that the recovered parameter values matched the values used to generate the predictions with only a few exceptions. These occurred when parameter combinations were extreme and when a skewed distribution (exponential) of nondecision times was used. The conclusion is that the standard model is robust to moderate changes in the across-trial distributions of parameter values.

Turning visual search time on its head

Our everyday visual experience frequently involves searching for objects in clutter. Why are some searches easy and others hard? It is generally believed that the time taken to find a target increases as it becomes similar to its surrounding distractors. Here, I show that while this is qualitatively true, the exact relationship is in fact not linear. In a simple search experiment, when subjects searched for a bar differing in orientation from its distractors, search time was inversely proportional to the angular difference in orientation. Thus, rather than taking search reaction time (RT) to be a measure of target-distractor similarity, we can literally turn search time on its head (i.e. take its reciprocal 1/RT) to obtain a measure of search dissimilarity that varies linearly over a large range of target-distractor differences. I show that this dissimilarity measure has the properties of a distance metric, and report two interesting insights come from this measure: First, for a large number of searches, search asymmetries are relatively rare and when they do occur, differ by a fixed distance. Second, search distances can be used to elucidate object representations that underlie search - for example, these representations are roughly invariant to three-dimensional view. Finally, search distance has a straightforward interpretation in the context of accumulator models of search, where it is proportional to the discriminative signal that is integrated to produce a response. This is consistent with recent studies that have linked this distance to neuronal discriminability in visual cortex. Thus, while search time remains the more direct measure of visual search, its reciprocal also has the potential for interesting and novel insights.

Temporal production signals in parietal cortex

We often perform movements and actions on the basis of internal motivations and without any explicit instructions or cues. One common example of such behaviors is our ability to initiate movements solely on the basis of an internally generated sense of the passage of time. In order to isolate the neuronal signals responsible for such timed behaviors, we devised a task that requires nonhuman primates to move their eyes consistently at regular time intervals in the absence of any external stimulus events and without an immediate expectation of reward. Despite the lack of sensory information, we found that animals were remarkably precise and consistent in timed behaviors, with standard deviations on the order of 100 ms. To examine the potential neural basis of this precision, we recorded from single neurons in the lateral intraparietal area (LIP), which has been implicated in the planning and execution of eye movements. In contrast to previous studies that observed a build-up of activity associated with the passage of time, we found that LIP activity decreased at a constant rate between timed movements. Moreover, the magnitude of activity was predictive of the timing of the impending movement. Interestingly, this relationship depended on eye movement direction: activity was negatively correlated with timing when the upcoming saccade was toward the neuron's response field and positively correlated when the upcoming saccade was directed away from the response field. This suggests that LIP activity encodes timed movements in a push-pull manner by signaling for both saccade initiation towards one target and prolonged fixation for the other target. Thus timed movements in this task appear to reflect the competition between local populations of task relevant neurons rather than a global timing signal.

Neural correlates and neural computations in posterior parietal cortex during perceptual decision-making

A recent line of work has found remarkable success in relating perceptual decision-making and the spiking activity in the macaque lateral intraparietal area (LIP). In this review, we focus on questions about the neural computations in LIP that are not answered by demonstrations of neural correlates of psychological processes. We highlight three areas of limitations in our current understanding of the precise neural computations that might underlie neural correlates of decisions: (1) empirical questions not yet answered by existing data; (2) implementation issues related to how neural circuits could actually implement the mechanisms suggested by both extracellular neurophysiology and psychophysics; and (3) ecological constraints related to the use of well-controlled laboratory tasks and whether they provide an accurate window on sensorimotor computation. These issues motivate the adoption of a more general "encoding-decoding framework" that will be fruitful for more detailed contemplation of how neural computations in LIP relate to the formation of perceptual decisions.

Consistent implementation of decisions in the brain

Despite the complexity and variability of decision processes, motor responses are generally stereotypical and independent of decision difficulty. How is this consistency achieved? Through an engineering analogy we consider how and why a system should be designed to realise not only flexible decision-making, but also consistent decision implementation. We specifically consider neurobiologically-plausible accumulator models of decision-making, in which decisions are made when a decision threshold is reached. To trade-off between the speed and accuracy of the decision in these models, one can either adjust the thresholds themselves or, equivalently, fix the thresholds and adjust baseline activation. Here we review how this equivalence can be implemented in such models. We then argue that manipulating baseline activation is preferable as it realises consistent decision implementation by ensuring consistency of motor inputs, summarise empirical evidence in support of this hypothesis, and suggest that it could be a general principle of decision making and implementation. Our goal is therefore to review how neurobiologically-plausible models of decision-making can manipulate speed-accuracy trade-offs using different mechanisms, to consider which of these mechanisms has more desirable decision-implementation properties, and then review the relevant neuroscientific data on which mechanism brains actually use.

Response variability of frontal eye field neurons modulates with sensory input and saccade preparation but not visual search salience

Discharge rate modulation of frontal eye field (FEF) neurons has been identified with a representation of visual search salience (physical conspicuity and behavioral relevance) and saccade preparation. We tested whether salience or saccade preparation are evident in the trial-to-trial variability of discharge rate. We quantified response variability via the Fano factor in FEF neurons recorded in monkeys performing efficient and inefficient visual search tasks. Response variability declined following stimulus presentation in most neurons, but despite clear discharge rate modulation, variability did not change with target salience. Instead, we found that response variability was modulated by stimulus luminance and the number of items in the visual field independently of attentional demands. Response variability declined to a minimum before saccade initiation, and presaccadic response variability was directionally tuned. In addition, response variability was correlated with the response time of memory-guided saccades. These results indicate that the trial-by-trial response variability of FEF neurons reflects saccade preparation and the strength of sensory input, but not visual search salience or attentional allocation.

The optimality of sensory processing during the speed-accuracy tradeoff

When people make decisions quickly, accuracy suffers. Traditionally, speed-accuracy tradeoffs (SATs) have been almost exclusively ascribed to changes in the amount of sensory evidence required to support a response ("response caution") and the neural correlates associated with the later stages of decision making (e.g., motor response generation and execution). Here, we investigated whether performance decrements under speed pressure also reflect suboptimal information processing in early sensory areas such as primary visual cortex (V1). Human subjects performed an orientation discrimination task while emphasizing either response speed or accuracy. A model of choice behavior revealed that the rate of sensory evidence accumulation was selectively modulated when subjects emphasized accuracy, but not speed, suggesting that changes in sensory processing also influence the SAT. We then used fMRI and a forward encoding model to derive orientation-selective tuning functions based on activation patterns in V1. When accuracy was emphasized, the extent to which orientation-selective tuning profiles exhibited a theoretically optimal gain pattern predicted both response accuracy and the rate of sensory evidence accumulation. However, these relationships were not observed when subjects emphasized speed. Collectively, our findings suggest that, in addition to lowered response thresholds, the performance decrements observed during speeded decision making may result from a failure to optimally process sensory signals.

Category-specific neural oscillations predict recall organization during memory search

Retrieved-context models of human memory propose that as material is studied, retrieval cues are constructed that allow one to target particular aspects of past experience. We examined the neural predictions of these models by using electrocorticographic/depth recordings and scalp electroencephalography (EEG) to characterize category-specific oscillatory activity, while participants studied and recalled items from distinct, neurally discriminable categories. During study, these category-specific patterns predict whether a studied item will be recalled. In the scalp EEG experiment, category-specific activity during study also predicts whether a given item will be recalled adjacent to other same-category items, consistent with the proposal that a category-specific retrieval cue is used to guide memory search. Retrieved-context models suggest that integrative neural circuitry is involved in the construction and maintenance of the retrieval cue. Consistent with this hypothesis, we observe category-specific patterns that rise in strength as multiple same-category items are studied sequentially, and find that individual differences in this category-specific neural integration during study predict the degree to which a participant will use category information to organize memory search. Finally, we track the deployment of this retrieval cue during memory search: Category-specific patterns are stronger when participants organize their responses according to the category of the studied material.

The effects of evidence bounds on decision-making: theoretical and empirical developments

Converging findings from behavioral, neurophysiological, and neuroimaging studies suggest an integration-to-boundary mechanism governing decision formation and choice selection. This mechanism is supported by sequential sampling models of choice decisions, which can implement statistically optimal decision strategies for selecting between multiple alternative options on the basis of sensory evidence. This review focuses on recent developments in understanding the evidence boundary, an important component of decision-making raised by experimental findings and models. The article starts by reviewing the neurobiology of perceptual decisions and several influential sequential sampling models, in particular the drift-diffusion model, the Ornstein–Uhlenbeck model and the leaky-competing-accumulator model. In the second part, the article examines how the boundary may affect a model’s dynamics and performance and to what extent it may improve a model’s fits to experimental data. In the third part, the article examines recent findings that support the presence and site of boundaries in the brain. The article considers two questions: (1) whether the boundary is a spontaneous property of neural integrators, or is controlled by dedicated neural circuits; (2) if the boundary is variable, what could be the driving factors behind boundary changes? The review brings together studies using different experimental methods in seeking answers to these questions, highlights psychological and physiological factors that may be associated with the boundary and its changes, and further considers the evidence boundary as a generic mechanism to guide complex behavior.

Evidence Accumulator or Decision Threshold - Which Cortical Mechanism are We Observing?

Most psychological models of perceptual decision making are of the accumulation-to-threshold variety. The neural basis of accumulation in parietal and prefrontal cortex is therefore a topic of great interest in neuroscience. In contrast, threshold mechanisms have received less attention, and their neural basis has usually been sought in subcortical structures. Here I analyze a model of a decision threshold that can be implemented in the same cortical areas as evidence accumulators, and whose behavior bears on two open questions in decision neuroscience: (1) When ramping activity is observed in a brain region during decision making, does it reflect evidence accumulation? (2) Are changes in speed-accuracy tradeoffs and response biases more likely to be achieved by changes in thresholds, or in accumulation rates and starting points? The analysis suggests that task-modulated ramping activity, by itself, is weak evidence that a brain area mediates evidence accumulation as opposed to threshold readout; and that signs of modulated accumulation are as likely to indicate threshold adaptation as adaptation of starting points and accumulation rates. These conclusions imply that how thresholds are modeled can dramatically impact accumulator-based interpretations of this data.

Computational models of decision making: integration, stability, and noise

Decision making demands the accumulation of sensory evidence over time. Questions remain about how this occurs, but recent years have seen progress on several fronts. The first concerns when optimal accumulation of evidence coincides with the simplest method of accumulating neural activity: summation over time. The second involves what computations the brain might perform when summation is difficult due to imprecision in neural circuits or is suboptimal due to uncertainty or variability in how evidence arrives. Finally, the third concerns sources of noise in decision circuits. Empirical studies have better constrained the extent of this noise, and modeling work is helping to clarify its possible origins.

From salience to saccades: multiple-alternative gated stochastic accumulator model of visual search

We describe a stochastic accumulator model demonstrating that visual search performance can be understood as a gated feedforward cascade from a salience map to multiple competing accumulators. The model quantitatively accounts for behavior and predicts neural dynamics of macaque monkeys performing visual search for a target stimulus among different numbers of distractors. The salience accumulated in the model is equated with the spike trains recorded from visually responsive neurons in the frontal eye field. Accumulated variability in the firing rates of these neurons explains choice probabilities and the distributions of correct and error response times with search arrays of different set sizes if the accumulators are mutually inhibitory. The dynamics of the stochastic accumulators quantitatively predict the activity of presaccadic movement neurons that initiate eye movements if gating inhibition prevents accumulation before the representation of stimulus salience emerges. Adjustments in the level of gating inhibition can control trade-offs in speed and accuracy that optimize visual search performance.

Deliberation in the motor system: reflex gains track evolving evidence leading to a decision

Both decision making and sensorimotor control require real-time processing of noisy information streams. Historically these processes were thought to operate sequentially: cognitive processing leads to a decision, and the outcome is passed to the motor system to be converted into action. Recently, it has been suggested that the decision process may provide a continuous flow of information to the motor system, allowing it to prepare in a graded fashion for the probable outcome. Such continuous flow is supported by electrophysiology in nonhuman primates. Here we provide direct evidence for the continuous flow of an evolving decision variable to the motor system in humans. Subjects viewed a dynamic random dot display and were asked to indicate their decision about direction by moving a handle to one of two targets. We probed the state of the motor system by perturbing the arm at random times during decision formation. Reflex gains were modulated by the strength and duration of motion, reflecting the accumulated evidence in support of the evolving decision. The magnitude and variance of these gains tracked a decision variable that explained the subject's decision accuracy. The findings support a continuous process linking the evolving computations associated with decision making and sensorimotor control.

The spatial impact of visual distractors on saccade latency

Remote transient changes in the environment, such as the onset of visual distractors, impact on the execution of target directed saccadic eye movements. Studies that have examined the latency of the saccade response have shown conflicting results. When there was an element of target selection, saccade latency increased as the distance between distractor and target increased. In contrast, when target selection is minimized by restricting the target to appear on one axis position, latency has been found to be slowest when the distractor is shown at fixation and reduces as it moves away from this position, rather than from the target. Here we report four experiments examining saccade latency as target and distractor positions are varied. We find support for both a dependence of saccade latency on distractor distance from target and from fixation: saccade latency was longer when distractor is shown close to fixation and even longer still when shown in an opposite location (180°) to the target. We suggest that this is due to inhibitory interactions between the distractor, fixation and the target interfering with fixation disengagement and target selection.

Paradoxical evidence integration in rapid decision processes

Decisions about noisy stimuli require evidence integration over time. Traditionally, evidence integration and decision making are described as a one-stage process: a decision is made when evidence for the presence of a stimulus crosses a threshold. Here, we show that one-stage models cannot explain psychophysical experiments on feature fusion, where two visual stimuli are presented in rapid succession. Paradoxically, the second stimulus biases decisions more strongly than the first one, contrary to predictions of one-stage models and intuition. We present a two-stage model where sensory information is integrated and buffered before it is fed into a drift diffusion process. The model is tested in a series of psychophysical experiments and explains both accuracy and reaction time distributions.

In models of decision making, evidence is accumulated until it crosses a threshold. The amount of evidence is directly related to the strength of the sensory input for the decision alternatives. Such one-stage models predict that if two stimulus alternatives are presented in succession, the stimulus alternative presented first dominates the decision, as the accumulated evidence will reach the threshold for this alternative first. Here, we show that for short stimulus durations decision making is not dominated by the first, but by the second stimulus. This result cannot be explained by classical one-stage decision models. We present a two-stage model where sensory input is first integrated before its outcome is fed into a classical decision process.

Neural basis of feature-based contextual effects on visual search behavior

Searching for a visual object is known to be adaptable to context, and it is thought to result from the selection of neural representations distributed on a visual salience map, wherein stimulus-driven and goal-directed signals are combined. Here we investigated the neural basis of this adaptability by recording superior colliculus (SC) neurons while three female rhesus monkeys (Macaca mulatta) searched with saccadic eye movements for a target presented in an array of visual stimuli whose feature composition varied from trial to trial. We found that sensory-motor activity associated with distracters was enhanced or suppressed depending on the search array composition and that it corresponded to the monkey's search strategy, as assessed by the distribution of the occasional errant saccades. This feature-related modulation occurred independently from the saccade goal and facilitated the process of saccade target selection. We also observed feature-related enhancement in the activity associated with distracters that had been the search target during the previous session. Consistent with recurrent processing, both feature-related neuronal modulations occurred more than 60 ms after the onset of the visually evoked responses, and their near coincidence with the time of saccade target selection suggests that they are integral to this process. These results suggest that SC neuronal activity is shaped by the visual context as dictated by both stimulus-driven and goal-directed signals. Given the close proximity of the SC to the motor circuit, our findings suggest a direct link between perception and action and no need for distinct salience and motor maps.

Activation in the neural network responsible for categorization and recognition reflects parameter changes

According to various influential formal models of cognition, perceptual categorization and old-new recognition recruit the same memory system. By contrast, the prevailing view in the cognitive neuroscience literature is that separate neural systems mediate perceptual categorization and recognition. A direct form of evidence is that separate brain regions are activated when observers engage in categorization and recognition tasks involving the same types of stimuli. However, even if the same memory-based comparison processes underlie categorization and recognition, one would not expect to see identical patterns of brain activity across the tasks; the reason is that observers would adjust parameter settings (e.g., vary criterion settings) across the tasks to satisfy the different task goals. In this fMRI study, we conducted categorization and recognition tasks in which stimulus conditions were held constant, and in which observers were induced to vary hypothesized parameter settings across conditions. A formal exemplar model was fitted to the data to track the changes in parameters to help interpret the fMRI results. We observed systematic effects of changes in parameters on patterns of brain activity, which were interpretable in terms of differing forms of evidence accumulation that resulted from the changed parameter settings. After controlling for stimulus and parameter-related differences, we found little evidence that categorization and recognition recruit separate memory systems.

Neural correlates of evidence accumulation in a perceptual decision task

Sequential sampling models provide a useful framework for understanding human decision making. A key component of these models is an evidence accumulation process in which information is accrued over time to a threshold, at which point a choice is made. Previous neurophysiological studies on perceptual decision making have suggested accumulation occurs only in sensorimotor areas involved in making the action for the choice. Here we investigated the neural correlates of evidence accumulation in the human brain using functional magnetic resonance imaging (fMRI) while manipulating the quality of sensory evidence, the response modality, and the foreknowledge of the response modality. We trained subjects to perform a random dot motion direction discrimination task by either moving their eyes or pressing buttons to make their responses. In addition, they were cued about the response modality either in advance of the stimulus or after a delay. We isolated fMRI responses for perceptual decisions in both independently defined sensorimotor areas and task-defined nonsensorimotor areas. We found neural signatures of evidence accumulation, a higher fMRI response on low coherence trials than high coherence trials, primarily in saccade-related sensorimotor areas (frontal eye field and intraparietal sulcus) and nonsensorimotor areas in anterior insula and inferior frontal sulcus. Critically, such neural signatures did not depend on response modality or foreknowledge. These results help establish human brain areas involved in evidence accumulation and suggest that the neural mechanism for evidence accumulation is not specific to effectors. Instead, the neural system might accumulate evidence for particular stimulus features relevant to a perceptual task.

Neural basis of adaptive response time adjustment during saccade countermanding

Humans and macaque monkeys adjust their response time adaptively in stop-signal (countermanding) tasks, responding slower after stop-signal trials than after control trials with no stop signal. We investigated the neural mechanism underlying this adaptive response time adjustment in macaque monkeys performing a saccade countermanding task. Earlier research showed that movements are initiated when the random accumulation of presaccadic movement-related activity reaches a fixed threshold. We found that a systematic delay in response time after stop-signal trials was accomplished not through a change of threshold, baseline, or accumulation rate, but instead through a change in the time when activity first began to accumulate. The neurons underlying movement initiation have been identified with stochastic accumulator models of response time performance. Therefore, this new result provides surprising new insights into the neural instantiation of stochastic accumulator models and the mechanisms through which executive control can be exerted.