Attention directed by expectations enhances receptive fields in cortical area MT

Changes in visual speed perception induced by anticipatory smooth eye movements

Expectations about forthcoming visual motion shaped by observers' experiences are known to induce anticipatory smooth eye movements (ASEM) and changes in visual perception. Previous studies have demonstrated discrete effects of expectations on the control of ASEM and perception. However, the tasks designed in these studies were not able to segregate the effects of expectations and execution of ASEM itself on perception. In the current study, we attempted to directly examine the effect of ASEM itself on visual speed perception using a two-alternative forced-choice task (2AFC task), in which observers were asked to track a pair of sequentially presented visual motion stimuli with their eyes and to judge whether the second stimulus (test stimulus) was faster or slower than the first (reference stimulus). Our results showed that observers' visual speed perception, quantified by a psychometric function, shifted according to ASEM velocity. This was the case, even though there was no difference in the steady-state eye velocity. Further analyses revealed that the observers' perceptual decisions could be explained by a difference in the magnitude of retinal slip velocity in the initial phase of ocular tracking when the reference and test stimuli were presented, rather than in the steady-state phase. Our results provide psychophysical evidence of the importance of initial ocular tracking in visual speed perception and the strong impact of ASEM.

Micropools of reliable area MT neurons explain rapid motion detection

Many models of perceptually based decisions postulate that actions are initiated when accumulated sensory signals reach a threshold level of activity. These models have received considerable neurophysiological support from recordings of individual neurons while animals are engaged in motion discrimination tasks. These experiments have found that the activity of neurons in a particular visual area strongly associated with motion processing (MT), when pooled over hundreds of milliseconds, is sufficient to explain behavioral timing and performance. However, this level of pooling may be problematic for urgent perceptual decisions in which rapid detection dictates temporally precise integration. In this paper, we explore the physiological basis of one such task in which macaques detected brief (~70 ms) transients of coherent motion within ~240 ms. We find that a simple linear summation model based on realistic stimulus responses of as few as 40 correlated neurons can predict the reliability and timing of rapid motion detection. The model naturally reproduces a distinctive physiological relationship observed in rapid detection tasks in which the individual neurons with the most reliable stimulus responses are also the most predictive of impending behavioral choices. Remarkably, we observed this relationship across our simulated neuronal populations even when all neurons within the pool were weighted equally with respect to readout. These results demonstrate that small numbers of reliable sensory neurons can dominate perceptual judgments without any explicit reliability based weighting and are sufficient to explain the accuracy, latency, and temporal precision of rapid detection. NEW & NOTEWORTHY Computational and psychophysical models suggest that performance in many perceptual tasks may be based on the preferential sampling of reliable neurons. Recent studies of MT neurons during rapid motion detection, in which only those neurons with the most reliable sensory responses were strongly predictive of the animals' decisions, seemingly support this notion. Here we show that a simple threshold model without explicit reliability biases can explain both the behavioral accuracy and precision of these detections and the distribution of sensory- and choice-related signals across neurons.

The Role of the Lateral Intraparietal Area in (the Study of) Decision Making

Over the past two decades, neurophysiological responses in the lateral intraparietal area (LIP) have received extensive study for insight into decision making. In a parallel manner, inferred cognitive processes have enriched interpretations of LIP activity. Because of this bidirectional interplay between physiology and cognition, LIP has served as fertile ground for developing quantitative models that link neural activity with decision making. These models stand as some of the most important frameworks for linking brain and mind, and they are now mature enough to be evaluated in finer detail and integrated with other lines of investigation of LIP function. Here, we focus on the relationship between LIP responses and known sensory and motor events in perceptual decision-making tasks, as assessed by correlative and causal methods. The resulting sensorimotor-focused approach offers an account of LIP activity as a multiplexed amalgam of sensory, cognitive, and motor-related activity, with a complex and often indirect relationship to decision processes. Our data-driven focus on multiplexing (and de-multiplexing) of various response components can complement decision-focused models and provides more detailed insight into how neural signals might relate to cognitive processes such as decision making.

Neuronal Mechanisms of Visual Attention

Advances on several fronts have refined our understanding of the neuronal mechanisms of attention. This review focuses on recent progress in understanding visual attention through single-neuron recordings made in behaving subjects. Simultaneous recordings from populations of individual cells have shown that attention is associated with changes in the correlated firing of neurons that can enhance the quality of sensory representations. Other work has shown that sensory normalization mechanisms are important for explaining many aspects of how visual representations change with attention, and these mechanisms must be taken into account when evaluating attention-related neuronal modulations. Studies comparing different brain structures suggest that attention is composed of several cognitive processes, which might be controlled by different brain regions. Collectively, these and other recent findings provide a clearer picture of how representations in the visual system change when attention shifts from one target to another.

SMA Selectively Codes the Active Accumulation of Temporal, Not Spatial, Magnitude

Estimating duration depends on the sequential integration (accumulation) of temporal information in working memory. Using fMRI, we directly compared the accumulation of information in temporal versus spatial domains. Participants estimated either the duration or distance of the dynamic trajectory of a moving dot or, in a control condition, a static line stimulus. Comparing the duration versus distance of static lines activated an extensive cortico-striatal network. By contrast, comparing the duration versus distance of dynamic trajectories, both of which required sequential integration of information, activated SMA alone. Indeed, activity in SMA, as well as right inferior occipital cortex, increased parametrically as a function of stimulus duration and also correlated with individual differences in the propensity to overestimate stimulus duration. By contrast, activity in primary visual cortex increased parametrically as a function of stimulus distance. Crucially, a direct comparison of the parametric responses to duration versus distance revealed that activity in SMA increased incrementally as a function of stimulus duration but not as a function of stimulus distance. Collectively, our results indicate that SMA responds to the active accumulation of information selectively in the temporal domain.

Nonlinear population receptive field changes in human area V5/MT+ of healthy subjects with simulated visual field scotomas

There is extensive controversy over whether the adult visual cortex is able to reorganize following visual field loss (scotoma) as a result of retinal or cortical lesions. Functional magnetic resonance imaging (fMRI) methods provide a useful tool to study the aggregate receptive field properties and assess the capacity of the human visual cortex to reorganize following injury. However, these methods are prone to biases near the boundaries of the scotoma. Retinotopic changes resembling reorganization have been observed in the early visual cortex of normal subjects when the visual stimulus is masked to simulate retinal or cortical scotomas. It is not known how the receptive fields of higher visual areas, like hV5/MT+, are affected by partial stimulus deprivation. We measured population receptive field (pRF) responses in human area V5/MT+ of 5 healthy participants under full stimulation and compared them with responses obtained from the same area while masking the left superior quadrant of the visual field (“artificial scotoma” or AS). We found that pRF estimations in area hV5/MT+ are nonlinearly affected by the AS. Specifically, pRF centers shift towards the AS, while the pRF amplitude increases and the pRF size decreases near the AS border. The observed pRF changes do not reflect reorganization but reveal important properties of normal visual processing under different test-stimulus conditions.

Dynamic Control of Response Criterion in Premotor Cortex during Perceptual Detection under Temporal Uncertainty

Under uncertainty, the brain uses previous knowledge to transform sensory inputs into the percepts on which decisions are based. When the uncertainty lies in the timing of sensory evidence, however, the mechanism underlying the use of previously acquired temporal information remains unknown. We study this issue in monkeys performing a detection task with variable stimulation times. We use the neural correlates of false alarms to infer the subject's response criterion and find that it modulates over the course of a trial. Analysis of premotor cortex activity shows that this modulation is represented by the dynamics of population responses. A trained recurrent network model reproduces the experimental findings and demonstrates a neural mechanism to benefit from temporal expectations in perceptual detection. Previous knowledge about the probability of stimulation over time can be intrinsically encoded in the neural population dynamics, allowing a flexible control of the response criterion over time.

Combining spatial and temporal expectations to improve visual perception

The importance of temporal expectations in modulating perceptual functions is increasingly recognized. However, the means through which temporal expectations can bias perceptual information processing remains ill understood. Recent theories propose that modulatory effects of temporal expectations rely on the co-existence of other biases based on receptive-field properties, such as spatial location. We tested whether perceptual benefits of temporal expectations in a perceptually demanding psychophysical task depended on the presence of spatial expectations. Foveally presented symbolic arrow cues indicated simultaneously where (location) and when (time) target events were more likely to occur. The direction of the arrow indicated target location (80% validity), while its color (pink or blue) indicated the interval (80% validity) for target appearance. Our results confirmed a strong synergistic interaction between temporal and spatial expectations in enhancing visual discrimination. Temporal expectation significantly boosted the effectiveness of spatial expectation in sharpening perception. However, benefits for temporal expectation disappeared when targets occurred at unattended locations. Our findings suggest that anticipated receptive-field properties of targets provide a natural template upon which temporal expectations can operate in order to help prioritize goal-relevant events from early perceptual stages.

Diverse suppressive influences in area MT and selectivity to complex motion features

Neuronal selectivity results from both excitatory and suppressive inputs to a given neuron. Suppressive influences can often significantly modulate neuronal responses and impart novel selectivity in the context of behaviorally relevant stimuli. In this work, we use a naturalistic optic flow stimulus to explore the responses of neurons in the middle temporal area (MT) of the alert macaque monkey; these responses are interpreted using a hierarchical model that incorporates relevant nonlinear properties of upstream processing in the primary visual cortex (V1). In this stimulus context, MT neuron responses can be predicted from distinct excitatory and suppressive components. Excitation is spatially localized and matches the measured preferred direction of each neuron. Suppression is typically composed of two distinct components: (1) a directionally untuned component, which appears to play the role of surround suppression and normalization; and (2) a direction-selective component, with comparable tuning width as excitation and a distinct spatial footprint that is usually partially overlapping with excitation. The direction preference of this direction-tuned suppression varies widely across MT neurons: approximately one-third have overlapping suppression in the opposite direction as excitation, and many other neurons have suppression with similar direction preferences to excitation. There is also a population of MT neurons with orthogonally oriented suppression. We demonstrate that direction-selective suppression can impart selectivity of MT neurons to more complex velocity fields and that it can be used for improved estimation of the three-dimensional velocity of moving objects. Thus, considering MT neurons in a complex stimulus context reveals a diverse set of computations likely relevant for visual processing in natural visual contexts.

Temporal expectancy in the context of a theory of visual attention

Temporal expectation is expectation with respect to the timing of an event such as the appearance of a certain stimulus. In this paper, temporal expectancy is investigated in the context of the theory of visual attention (TVA), and we begin by summarizing the foundations of this theoretical framework. Next, we present a parametric experiment exploring the effects of temporal expectation on perceptual processing speed in cued single-stimulus letter recognition with unspeeded motor responses. The length of the cue-stimulus foreperiod was exponentially distributed with one of six hazard rates varying between blocks. We hypothesized that this manipulation would result in a distinct temporal expectation in each hazard rate condition. Stimulus exposures were varied such that both the temporal threshold of conscious perception (t0 ms) and the perceptual processing speed (v letters s(-1)) could be estimated using TVA. We found that the temporal threshold t0 was unaffected by temporal expectation, but the perceptual processing speed v was a strikingly linear function of the logarithm of the hazard rate of the stimulus presentation. We argue that the effects on the v values were generated by changes in perceptual biases, suggesting that our perceptual biases are directly related to our temporal expectations.

Inattention blindness to motion in middle temporal area

Subjects naturally form and use expectations to solve familiar tasks, but the accuracy of these expectations and the neuronal mechanisms by which these expectations enhance behavior are unclear. We trained animals (Macaca mulatta) in a challenging perceptual task in which the likelihood of a very brief pulse of motion was consistently modulated over time and space. Pulse likelihood had dramatic effects on behavior: unexpected pulses were nearly invisible to the animals. To examine the neuronal basis of such inattention blindness, we recorded from single neurons in the middle temporal (MT) area, an area related to motion perception. Fluctuations in how reliably MT neurons both signaled stimulus events and predicted behavioral choices were highly correlated with changes in performance over the course of individual trials. A simple neuronal pooling model reveals that the dramatic behavioral effects of attention in this task can be completely explained by changes in the reliability of a small number of MT neurons.

The ups and downs of temporal orienting: a review of auditory temporal orienting studies and a model associating the heterogeneous findings on the auditory N1 with opposite effects of attention and prediction

The temporal orienting of attention refers to the process of focusing (neural) resources on a particular time point in order to boost the processing of and the responding to sensory events. Temporal attention is manipulated by varying the task-relevance of events at different time points or by inducing expectations that an event occurs at a particular time point. Notably, the electrophysiological correlates of these manipulations at early processing stages are not identical: Auditory studies operationalizing temporal attention through task-relevance consistently found enhancements of early, sensory processing, as shown in the N1 component of the auditory event-related potential (ERP). By contrast, previous work on temporal orienting based on expectations showed mixed results: early, sensory processing was either enhanced or attenuated or not affected at all. In the present work, I will review existing findings on temporal orienting with a special focus on the auditory modality and present a working model to reconcile the previously heterogeneous results. Specifically, I will suggest that when expectations are used to manipulate attention, this will lead both to an orienting of attention and to the generation of precise predictions about the upcoming event. Attention and prediction are assumed to have opposite effects on early auditory processing, with temporal attention increasing and temporal predictions decreasing the associated ERP correlate, the auditory N1. The heterogeneous findings of studies manipulating temporal orienting by inducing expectations may thus be the consequence of differences in the relative contribution of attention and prediction processes. The model's predictions will be discussed in the context of a functional interpretation of the auditory N1 as an attention call signal, as presented in a recent model on auditory processing.

Temporal expectation enhances contrast sensitivity by phase entrainment of low-frequency oscillations in visual cortex

Although it is increasingly accepted that temporal expectation can modulate early perceptual processing, the underlying neural computations remain unknown. In the present study, we combined a psychophysical paradigm with electrophysiological recordings to investigate the putative contribution of low-frequency oscillatory activity in mediating the modulation of visual perception by temporal expectation. Human participants judged the orientation of brief targets (visual Gabor patterns tilted clockwise or counterclockwise) embedded within temporally regular or irregular streams of noise-patches used as temporal cues. Psychophysical results indicated that temporal expectation enhanced the contrast sensitivity of visual targets. A diffusion model indicated that rhythmic temporal expectation modulated the signal-to-noise gain of visual processing. The concurrent electrophysiological data revealed that the phase of delta oscillations overlying human visual cortex (1-4 Hz) was predictive of the quality of target processing only in regular streams of events. Moreover, in the regular condition, the optimum phase of these perception-predictive oscillations occurred in anticipation of the expected events. Together, these results show a strong correspondence between psychophysical and neurophysiological data, suggesting that the phase entrainment of low-frequency oscillations to external sensory cues can serve as an important and flexible mechanism for enhancing sensory processing.

Temporal expectation improves the quality of sensory information

It is increasingly clear that we extract patterns of temporal regularity between events to optimize information processing. Whereas some of the mechanisms for facilitating action preparation and execution have been well documented, much less is understood about whether and how temporal expectations influence visual perception. We used a psychophysical paradigm and computational modeling to investigate the mechanisms by which temporal expectation can modulate visual perception. Visual targets appeared in a stream of noise-patches separated by a fixed (400 ms regular condition) or jittered (200/300/400/500/600 ms irregular condition) intervals. Targets were visual gratings tilted 45° clockwise or counter-clockwise, presented at one of seven contrast levels. Human observers were required to perform an orientation discrimination (i.e., left or right). Psychometric functions for contrast sensitivity fitted for the regular and irregular conditions indicated that temporal expectation modulates perceptual processing by enhancing the contrast sensitivity of visual targets. This increase in the signal strength was accompanied by a reduction in reaction times. A diffusion model indicated that rhythmic temporal expectation enhanced the signal-to-noise gain of the sensory evidence upon which decisions were made. These effects support the idea that temporal structure of external events can entrain the attentional focus and psychophysical data, optimizing the processing of relevant sensory information.

Extinction reveals that primary sensory cortex predicts reinforcement outcome

Primary sensory cortices are traditionally regarded as stimulus analysers. However, studies of associative learning-induced plasticity in the primary auditory cortex (A1) indicate involvement in learning, memory and other cognitive processes. For example, the area of representation of a tone becomes larger for stronger auditory memories and the magnitude of area gain is proportional to the degree that a tone becomes behaviorally important. Here, we used extinction to investigate whether 'behavioral importance' specifically reflects a sound's ability to predict reinforcement (reward or punishment) vs. to predict any significant change in the meaning of a sound. If the former, then extinction should reverse area gains as the signal no longer predicts reinforcement. Rats (n = 11) were trained to bar-press to a signal tone (5.0 kHz) for water-rewards, to induce signal-specific area gains in A1. After subsequent withdrawal of reward, A1 was mapped to determine representational areas. Signal-specific area gains, estimated from a previously established brain-behavior quantitative function, were reversed, supporting the 'reinforcement prediction' hypothesis. Area loss was specific to the signal tone vs. test tones, further indicating that withdrawal of reinforcement, rather than unreinforced tone presentation per se, was responsible for area loss. Importantly, the amount of area loss was correlated with the amount of extinction (r = 0.82, P < 0.01). These findings show that primary sensory cortical representation can encode behavioral importance as a signal's value to predict reinforcement, and that the number of cells tuned to a stimulus can dictate its ability to command behavior.

α oscillations related to anticipatory attention follow temporal expectations

Temporal expectations have been shown to enhance visual analysis of task-relevant events, especially when these are coupled with spatial expectations. Oscillatory brain activity, particularly in the alpha band, has been implicated in regulating excitability in visual areas as a function of anticipatory spatial attention. Here we asked whether temporal expectations derived from regular, rhythmic events can modulate ongoing oscillatory alpha-band activity, so that the changes in cortical excitability are focused over the time intervals at which target events are expected. The task we used involved making a perceptual discrimination about a small target stimulus that reappeared from "behind" a peripheral occluding band. Temporal expectations were manipulated by the regular, rhythmic versus irregular, arrhythmic approach of the stimulus toward the occluding band. Alpha-band activity was measured during the occlusion period, in which no stimulus was presented, but target reappearance was anticipated in conditions of high versus low temporal expectation. Time-frequency analysis showed that the amplitude of alpha-desynchronization followed the time course of temporal expectations. Alpha desynchronization increased rhythmically, peaking just before the expected reappearance of target times. Analysis of the event-related potentials evoked by the subsequent target stimuli showed enhancement of processing at both visual and motor stages. Our findings support a role for oscillations in regulating cortical excitability and suggest a plausible mechanism for biasing perception and action by temporal expectations.

Adaptive shape processing in primary visual cortex

The ability to derive meaning from complex sensory input requires the integration of information over space and time, as well as cognitive mechanisms to shape that integration. We studied these processes in the primary visual cortex (V1), where neurons are thought to integrate visual inputs along contours defined by an association field (AF). We recorded extracellularly from single cells in macaque V1 to map the AF, by using an optimization algorithm to find the contours that maximally activated individual cells. We combined the algorithm with a delayed-match-to-sample task, to test how the optimal contours might be molded by the monkey's expectation for particular cue shapes. We found that V1 neurons were selective for complex shapes, a property previously ascribed to higher cortical areas. Furthermore, the shape selectivity was reprogrammed by perceptual task: Over the whole network, the optimal modes of geometric selectivity shifted between distinct subsets of the AF, alternately representing different stimulus features known to predominate in natural scenes. Our results suggest a general model of cortical function, whereby horizontal connections provide a broad domain of potential associations, and top-down inputs dynamically gate these linkages to task switch the function of a network.