From shunting inhibition to dynamic normalization: Attentional selection and decision-making in brief visual displays

A revised diffusion model for conflict tasks

The recently developed diffusion model for conflict tasks (DMC) Ulrich et al. (Cognitive Psychology, 78, 148-174, 2015) provides a good account of data from all standard conflict tasks (e.g., Stroop, Simon, and flanker tasks) within a common evidence accumulation framework. A central feature of DMC's processing dynamics is that there is an initial phase of rapid accumulation of distractor evidence that is then selectively withdrawn from the decision mechanism as processing continues. We argue that this assumption is potentially troubling because it could be viewed as implying qualitative changes in the representation of distractor information over the time course of processing. These changes suggest more than simple inhibition or suppression of distractor information, as they involve evidence produced by distractor processing "changing sign" over time. In this article, we (a) develop a revised DMC (RDMC) whose dynamics operate strictly within the limits of inhibition/suppression (i.e., evidence strength can change monotonically, but cannot change sign); (b) demonstrate that RDMC can predict the full range of delta plots observed in the literature (i.e., both positive-going and negative-going); and (c) show that the model provides excellent fits to Simon and flanker data used to benchmark the original DMC at both the individual and group level. Our model provides a novel account of processing differences across Simon and flanker tasks. Specifically, that they differ in how distractor information is processed on congruent trials, rather than incongruent trials: congruent trials in the Simon task show relatively slow attention shifting away from distractor information (i.e., location) while complete and rapid attention shifting occurs in the flanker task. Our new model highlights the importance of considering dynamic interactions between top-down goals and bottom-up stimulus effects in conflict processing.

"Reliable organisms from unreliable components" revisited: the linear drift, linear infinitesimal variance model of decision making

Diffusion models of decision making, in which successive samples of noisy evidence are accumulated to decision criteria, provide a theoretical solution to von Neumann's (1956) problem of how to increase the reliability of neural computation in the presence of noise. I introduce and evaluate a new neurally-inspired dual diffusion model, the linear drift, linear infinitesimal variance (LDLIV) model, which embodies three features often thought to characterize neural mechanisms of decision making. The accumulating evidence is intrinsically positively-valued, saturates at high intensities, and is accumulated for each alternative separately. I present explicit integral-equation predictions for the response time distribution and choice probabilities for the LDLIV model and compare its performance on two benchmark sets of data to three other models: the standard diffusion model and two dual diffusion model composed of racing Wiener processes, one between absorbing and reflecting boundaries and one with absorbing boundaries only. The LDLIV model and the standard diffusion model performed similarly to one another, although the standard diffusion model is more parsimonious, and both performed appreciably better than the other two dual diffusion models. I argue that accumulation of noisy evidence by a diffusion process and drift rate variability are both expressions of how the cognitive system solves von Neumann's problem, by aggregating noisy representations over time and over elements of a neural population. I also argue that models that do not solve von Neumann's problem do not address the main theoretical question that historically motivated research in this area.

A dynamic normalization model of temporal attention

Vision is dynamic, handling a continuously changing stream of input, yet most models of visual attention are static. Here, we develop a dynamic normalization model of visual temporal attention and constrain it with new psychophysical human data. We manipulated temporal attention-the prioritization of visual information at specific points in time-to a sequence of two stimuli separated by a variable time interval. Voluntary temporal attention improved perceptual sensitivity only over a specific interval range. To explain these data, we modelled voluntary and involuntary attentional gain dynamics. Voluntary gain enhancement took the form of a limited resource over short time intervals, which recovered over time. Taken together, our theoretical and experimental results formalize and generalize the idea of limited attentional resources across space at a single moment to limited resources across time at a single location.

Adaptive mechanisms of visual motion discrimination, integration, and segregation

Under ecological conditions, the luminance impinging on the retina varies within a dynamic range of 220 dB. Stimulus contrast can also vary drastically within a scene and eye movements leave little time for sampling luminance. Given these fundamental problems, the human brain allocates a significant amount of resources and deploys both structural and functional solutions that work in tandem to compress this range. Here we propose a new dynamic neural model built upon well-established canonical neural mechanisms. The model consists of two feed-forward stages. The first stage encodes the stimulus spatially and normalizes its activity by extracting contrast and discounting the background luminance. These normalized activities allow a second stage to implement a contrast-dependent spatial-integration strategy. We show how the properties of this model can account for adaptive properties of motion discrimination, integration, and segregation.

A single, simple, statistical mechanism explains resource distribution and temporal updating in visual short-term memory

Investigations into the way that information is held and integrated within the visual system provides some basis for understanding how visual information is represented and processed. Just over sixty years ago, Swets, Shipley, McKey, and Green (1959) demonstrated that performance within an auditory detection task increases as a function of the square root of the number of stimulus observation intervals, following the predictions of basic sampling theory, indicating the efficient perceptual integration of stimulus information. This principle of observer performance contingent on a constant rate of stimulus sampling also forms the basis of the sample-size model (Palmer, 1990; Sewell, Lilburn, & Smith, 2014) which seeks to provide an account of how memory resources might be divided among item representations in visual short-term memory (VSTM). In this article, we combine the multiple observations paradigm of Swets and colleagues with the VSTM paradigm of Sewell and colleagues and show that the sample-size relationship accounts for both the increase in performance with the number of presentation intervals and the way that performance changes as a function of the number of items in memory. The model provides an account of both the overall information limit of VSTM and an account of the dynamics of that limit, demonstrating not only that observers can selectively update specific representations in memory but that performance in this task is accounted for by a simple statistical constraint. We discuss the implications for models of VSTM capacity and architecture generally, focusing on the implications for objecthood and the characteristics of encoding to and retrieval from memory.

A diffusion model analysis of target detection in near-threshold visual search

When searching for a target briefly presented among distractors how do people combine information across display locations to make a decision and how does the quality of the evidence entering the decision process vary with the type of items in the display? Research on accuracy in near-threshold visual search has had difficulty in distinguishing between models that make similar predictions about accuracy but make different assumptions about the underlying psychological processes. We used the diffusion model to analyse response times and accuracy data from four near-threshold search tasks which showed striking asymmetries between response-time distributions on target-present and target-absent trials. We found that performance was better explained by a model in which evidence was accumulated in parallel about each stimulus separately than one in which the evidence was pooled into a single decision process. We found that as contrast increased, the quality of the evidence entering the decision process about targets was markedly stronger than the evidence about nontargets. The overall pattern of evidence strength for stimuli on target-present and target-absent trials was consistent with a fixed-capacity memory system in which early visual processes assigned resources preferentially to targets over nontargets. The asymmetry was somewhat reduced in a letter-digit discrimination task that used heterogeneous targets and distractors, likely because heterogeneity reduces the efficiency of the preattentive filtering processes.

Predicting neuronal dynamics with a delayed gain control model

Visual neurons respond to static images with specific dynamics: neuronal responses sum sub-additively over time, reduce in amplitude with repeated or sustained stimuli (neuronal adaptation), and are slower at low stimulus contrast. Here, we propose a simple model that predicts these seemingly disparate response patterns observed in a diverse set of measurements-intracranial electrodes in patients, fMRI, and macaque single unit spiking. The model takes a time-varying contrast time course of a stimulus as input, and produces predicted neuronal dynamics as output. Model computation consists of linear filtering, expansive exponentiation, and a divisive gain control. The gain control signal relates to but is slower than the linear signal, and this delay is critical in giving rise to predictions matched to the observed dynamics. Our model is simpler than previously proposed related models, and fitting the model to intracranial EEG data uncovers two regularities across human visual field maps: estimated linear filters (temporal receptive fields) systematically differ across and within visual field maps, and later areas exhibit more rapid and substantial gain control. The model is further generalizable to account for dynamics of contrast-dependent spike rates in macaque V1, and amplitudes of fMRI BOLD in human V1.

The separable effects of feature precision and item load in visual short-term memory

Visual short-term memory (VSTM) has been described as being limited by the number of discrete visual objects, the aggregate quantity of information across multiple visual objects, or some combination of the two. Many recent studies examining these capacity limitations have shown that increasing the number of items in VSTM increases the frequency and magnitude of errors in a participant's recall of the stimulus. This increase in response dispersion has been interpreted as a loss of precision in an item's representation as the number of items in memory increases, possibly due to a change in the tuning of the underlying representation. However, increased response dispersion can also be caused by a reduction in the total memory strength available for decision making as a consequence of a reduction in the total amount of a fixed resource representing a stimulus. We investigated the effects of load on the precision of memory representations in a fine orientation discrimination task. Accuracy was well captured by extending a simple sample-size model of VSTM, using a tuning function to account for the effect of orientation precision on performance. The best model of the data was one in which the item strength decreased progressively with memory load at all stimulus exposure durations but in which tuning bandwidth was invariant. Our results imply that memory strength and feature precision are experimentally dissociable attributes of VSTM.

Temporally evolving gain mechanisms of attention in macaque area V4

Cognitive attention and perceptual saliency jointly govern our interaction with the environment. Yet, we still lack a universally accepted account of the interplay between attention and luminance contrast, a fundamental dimension of saliency. We measured the attentional modulation of V4 neurons' contrast response functions (CRFs) in awake, behaving macaque monkeys and applied a new approach that emphasizes the temporal dynamics of cell responses. We found that attention modulates CRFs via different gain mechanisms during subsequent epochs of visually driven activity: an early contrast-gain, strongly dependent on prestimulus activity changes (baseline shift); a time-limited stimulus-dependent multiplicative modulation, reaching its maximal expression around 150 ms after stimulus onset; and a late resurgence of contrast-gain modulation. Attention produced comparable time-dependent attentional gain changes on cells heterogeneously coding contrast, supporting the notion that the same circuits mediate attention mechanisms in V4 regardless of the form of contrast selectivity expressed by the given neuron. Surprisingly, attention was also sometimes capable of inducing radical transformations in the shape of CRFs. These findings offer important insights into the mechanisms that underlie contrast coding and attention in primate visual cortex and a new perspective on their interplay, one in which time becomes a fundamental factor.NEW & NOTEWORTHY We offer an innovative perspective on the interplay between attention and luminance contrast in macaque area V4, one in which time becomes a fundamental factor. We place emphasis on the temporal dynamics of attentional effects, pioneering the notion that attention modulates contrast response functions of V4 neurons via the sequential engagement of distinct gain mechanisms. These findings advance understanding of attentional influences on visual processing and help reconcile divergent results in the literature.

The attention-weighted sample-size model of visual short-term memory: Attention capture predicts resource allocation and memory load

We investigated the capacity of visual short-term memory (VSTM) in a phase discrimination task that required judgments about the configural relations between pairs of black and white features. Sewell et al. (2014) previously showed that VSTM capacity in an orientation discrimination task was well described by a sample-size model, which views VSTM as a resource comprised of a finite number of noisy stimulus samples. The model predicts the invariance of [Formula: see text] , the sum of squared sensitivities across items, for displays of different sizes. For phase discrimination, the set-size effect significantly exceeded that predicted by the sample-size model for both simultaneously and sequentially presented stimuli. Instead, the set-size effect and the serial position curves with sequential presentation were predicted by an attention-weighted version of the sample-size model, which assumes that one of the items in the display captures attention and receives a disproportionate share of resources. The choice probabilities and response time distributions from the task were well described by a diffusion decision model in which the drift rates embodied the assumptions of the attention-weighted sample-size model.