Receptive field shift and shrinkage in macaque middle temporal area through attentional gain modulation

Spatial attention alters visual cortical representation during target anticipation

Attention enables us to efficiently and flexibly interact with the environment by prioritizing some image features in preparation for responding to a stimulus. Using a concurrent psychophysics- fMRI experiment, we investigated how covert spatial attention affects responses in human visual cortex prior to target onset, and how it affects subsequent behavioral performance. Performance improved at cued locations and worsened at uncued locations, relative to distributed attention, demonstrating a selective tradeoff in processing. Pre-target BOLD responses in cortical visual field maps changed in two ways: First, there was a stimulus-independent baseline shift, positive in map locations near the cued location and negative elsewhere, paralleling the behavioral results. Second, population receptive field centers shifted toward the attended location. Both effects increased in higher visual areas. Together, the results show that spatial attention has large effects on visual cortex prior to target appearance, altering neural response properties throughout and across multiple visual field maps.

An attentional approach to geometrical illusions

It is known for a long time that some drawings composed of points, lines, and areas are systematically misperceived. The origin of these geometrical illusions is still unknown. Here we outline how a recent progress in attentional research contributes to a better understanding of such perceptual distortions. The basic idea behind this approach is that crucial elements of a drawing are differently attended. These changes in the allocation of spatial attention go along with systematic changes in low-level spatial coding. As a result, changes in the perception of spatial extent, angles, positions, and shapes can arise. How this approach can be applied to individual illusions is discussed.

Are neuronal mechanisms of attention universal across human sensory and motor brain maps?

One's experience of shifting attention from the color to the smell to the act of picking a flower seems like a unitary process applied, at will, to one modality after another. Yet, the unique and separable experiences of sight versus smell versus movement might suggest that the neural mechanisms of attention have been separately optimized to employ each modality to its greatest advantage. Moreover, addressing the issue of universality can be particularly difficult due to a paucity of existing cross-modal comparisons and a dearth of neurophysiological methods that can be applied equally well across disparate modalities. Here we outline some of the conceptual and methodological issues related to this problem and present an instructive example of an experimental approach that can be applied widely throughout the human brain to permit detailed, quantitative comparison of attentional mechanisms across modalities. The ultimate goal is to spur efforts across disciplines to provide a large and varied database of empirical observations that will either support the notion of a universal neural substrate for attention or more clearly identify the degree to which attentional mechanisms are specialized for each modality.

Alpha and SSVEP power outperform gamma power in capturing attentional modulation in human EEG

Attention typically reduces power in the alpha (8-12 Hz) band and increases power in gamma (>30 Hz) band in brain signals, as reported in macaque local field potential (LFP) and human electro/magneto-encephalogram (EEG/MEG) studies. In addition, EEG studies often use flickering stimuli that produce a specific measure called steady-state-visually-evoked-potential (SSVEP), whose power also increases with attention. However, effectiveness of these neural measures in capturing attentional modulation is unknown since stimuli and task paradigms vary widely across studies. In a recent macaque study, attentional modulation was more salient in the gamma band of the LFP, compared to alpha or SSVEP. To compare this with human EEG, we designed an orientation change detection task where we presented both static and counterphasing stimuli of matched difficulty levels to 26 subjects and compared attentional modulation of various measures under similar conditions. We report two main results. First, attentional modulation was comparable for SSVEP and alpha. Second, non-foveal stimuli produced weak gamma despite various stimulus optimizations and showed negligible attentional modulation although full-screen gratings showed robust gamma activity. Our results are useful for brain-machine-interfacing studies where suitable features are used for decoding attention, and also provide clues about spatial scales of neural mechanisms underlying attention.

A selection and targeting framework of cortical locations for line-scanning fMRI

Depth-resolved functional magnetic resonance imaging (fMRI) is an emerging field growing in popularity given the potential of separating signals from different computational processes in cerebral cortex. Conventional acquisition schemes suffer from low spatial and temporal resolutions. Line-scanning methods allow depth-resolved fMRI by sacrificing spatial coverage to sample blood oxygenated level-dependent (BOLD) responses at ultra-high temporal and spatial resolution. For neuroscience applications, it is critical to be able to place the line accurately to (1) sample the right neural population and (2) target that neural population with tailored stimuli or tasks. To this end, we devised a multi-session framework where a target cortical location is selected based on anatomical and functional properties. The line is then positioned according to this information in a separate second session, and we tailor the experiment to focus on the target location. Anatomically, the precision of the line placement was confirmed by projecting a nominal representation of the acquired line back onto the surface. Functional estimates of neural selectivities in the line, as quantified by a visual population-receptive field model, resembled the target selectivities well for most subjects. This functional precision was quantified in detail by estimating the distance between the visual field location of the targeted vertex and the location in visual cortex (V1) that most closely resembled the line-scanning estimates; this distance was on average ~5.5 mm. Given the dimensions of the line, differences in acquisition, session, and stimulus design, this validates that line-scanning can be used to probe local neural sensitivities across sessions. In summary, we present an accurate framework for line-scanning MRI; we believe such a framework is required to harness the full potential of line-scanning and maximize its utility. Furthermore, this approach bridges canonical fMRI experiments with electrophysiological experiments, which in turn allows novel avenues for studying human physiology non-invasively.

Multisensory integration in the mammalian brain: diversity and flexibility in health and disease

Multisensory integration (MSI) occurs in a variety of brain areas, spanning cortical and subcortical regions. In traditional studies on sensory processing, the sensory cortices have been considered for processing sensory information in a modality-specific manner. The sensory cortices, however, send the information to other cortical and subcortical areas, including the higher association cortices and the other sensory cortices, where the multiple modality inputs converge and integrate to generate a meaningful percept. This integration process is neither simple nor fixed because these brain areas interact with each other via complicated circuits, which can be modulated by numerous internal and external conditions. As a result, dynamic MSI makes multisensory decisions flexible and adaptive in behaving animals. Impairments in MSI occur in many psychiatric disorders, which may result in an altered perception of the multisensory stimuli and an abnormal reaction to them. This review discusses the diversity and flexibility of MSI in mammals, including humans, primates and rodents, as well as the brain areas involved. It further explains how such flexibility influences perceptual experiences in behaving animals in both health and disease. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

On the origin of the Helmholtz's square illusion: An attentional account

A square filled with parallel horizontal or vertical lines appears perceptually extended in the direction orthogonal to the lines. Here, we suggest that this Helmholtz illusion arises due to changes in spatial attention that entail changes at very early stages of perceptual processing. Three experiments are reported which tested this assumption. In Experiment1 and Experiment2, transient attentional cues were flashed in such a way that they either promoted (congruent condition) or hindered (incongruent condition) the attentional state presumably induced by the target objects. We predicted a decline of the illusion in the incongruent condition compared with the congruent condition. This prediction was confirmed in both experiments. However, the influence of (in)congruent attention cuing on the Helmholtz illusion depended on more sustained distributions of attention as well. An influence of sustained attention on the illusion was confirmed in Experiment 3, in which changes of attentional focus were induced by a secondary task. Overall, the results were consistent with our claim that the origin of the Helmholtz illusion is closely linked to the distribution of spatial attention.

Circadian regulation of developmental synaptogenesis via the hypocretinergic system

The circadian clock orchestrates a wide variety of physiological and behavioral processes, enabling animals to adapt to daily environmental changes, particularly the day-night cycle. However, the circadian clock's role in the developmental processes remains unclear. Here, we employ the in vivo long-term time-lapse imaging of retinotectal synapses in the optic tectum of larval zebrafish and reveal that synaptogenesis, a fundamental developmental process for neural circuit formation, exhibits circadian rhythm. This rhythmicity arises primarily from the synapse formation rather than elimination and requires the hypocretinergic neural system. Disruption of this synaptogenic rhythm, by impairing either the circadian clock or the hypocretinergic system, affects the arrangement of the retinotectal synapses on axon arbors and the refinement of the postsynaptic tectal neuron's receptive field. Thus, our findings demonstrate that the developmental synaptogenesis is under hypocretin-dependent circadian regulation, suggesting an important role of the circadian clock in neural development.

Decoding of attentional state using local field potentials

We review recent efforts to decode visual spatial attention from different types of brain signals, such as spikes and local field potentials (LFPs). Combining signals from more electrodes improves decoding, but the pattern of improvement varies considerably depending on the signal as well as the task (for example, decoding of sensory stimulus/motor intention versus location of attention). We argue that this pattern of results conveys important information not only about the usefulness of a particular brain signal for decoding attention, but also about the spatial scale over which attention operates in the brain. The spatial scale, in turn, likely depends on the extent of underlying mechanisms such as normalization, gain control via excitation-inhibition interactions, and neuromodulatory regulation of attention.

Visual Attention in the Prefrontal Cortex

Voluntary attention selects behaviorally relevant signals for further processing while filtering out distracter signals. Neural correlates of voluntary visual attention have been reported across multiple areas of the primate visual processing streams, with the earliest and strongest effects isolated in the prefrontal cortex. In this article, I review evidence supporting the hypothesis that signals guiding the allocation of voluntary attention emerge in areas of the prefrontal cortex and reach upstream areas to modulate the processing of incoming visual information according to its behavioral relevance. Areas located anterior and dorsal to the arcuate sulcus and the frontal eye fields produce signals that guide the allocation of spatial attention. Areas located anterior and ventral to the arcuate sulcus produce signals for feature-based attention. Prefrontal microcircuits are particularly suited to supporting voluntary attention because of their ability to generate attentional template signals and implement signal gating and their extensive connectivity with the rest of the brain. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

A neural mechanism for detecting object motion during self-motion

Detection of objects that move in a scene is a fundamental computation performed by the visual system. This computation is greatly complicated by observer motion, which causes most objects to move across the retinal image. How the visual system detects scene-relative object motion during self-motion is poorly understood. Human behavioral studies suggest that the visual system may identify local conflicts between motion parallax and binocular disparity cues to depth and may use these signals to detect moving objects. We describe a novel mechanism for performing this computation based on neurons in macaque middle temporal (MT) area with incongruent depth tuning for binocular disparity and motion parallax cues. Neurons with incongruent tuning respond selectively to scene-relative object motion, and their responses are predictive of perceptual decisions when animals are trained to detect a moving object during self-motion. This finding establishes a novel functional role for neurons with incongruent tuning for multiple depth cues.

The dynamics of microsaccade amplitude reflect shifting of covert attention

Attention flexibly shifts between spatial locations to accommodate task demands. The present study examined if the dynamics of attentional shifting are seen in microsaccades whose direction has been shown to accompany the shifts of covert attention. In a spatial cueing task, the cue predicted the target location on 100%, 75%, or 50% of the trials. The results revealed that microsaccade rate and amplitude were both reduced following cue onset and then rebounded. Both microsaccade rate and amplitude were biased towards the opposite direction of the cue and then returned to the cued direction. Importantly, the cue validity modulated the temporal profile of microsaccade amplitude but had little impact on the temporal profile of microsaccade rate. In line with this, the cueing effect measured with target response accuracy was correlated with the microsaccade amplitude only. These results indicate that the temporal dynamics of microsaccade amplitude reflect shifting of covert attention.

Larger images are better remembered during naturalistic encoding

We are constantly exposed to multiple visual scenes, and while freely viewing them without an intentional effort to memorize or encode them, only some are remembered. It has been suggested that image memory is influenced by multiple factors, such as depth of processing, familiarity, and visual category. However, this is typically investigated when people are instructed to perform a task (e.g., remember or make some judgment about the images), which may modulate processing at multiple levels and thus, may not generalize to naturalistic visual behavior. Visual memory is assumed to rely on high-level visual perception that shows a level of size invariance and therefore is not assumed to be highly dependent on image size. Here, we reasoned that during naturalistic vision, free of task-related modulations, bigger images stimulate more visual system processing resources (from retina to cortex) and would, therefore, be better remembered. In an extensive set of seven experiments, naïve participants (n = 182) were asked to freely view presented images (sized 3° to 24°) without any instructed encoding task. Afterward, they were given a surprise recognition test (midsized images, 50% already seen). Larger images were remembered better than smaller ones across all experiments (∼20% higher accuracy or ∼1.5 times better). Memory was proportional to image size, faces were better remembered, and outdoors the least. Results were robust even when controlling for image set, presentation order, screen resolution, image scaling at test, or the amount of information. While multiple factors affect image memory, our results suggest that low- to high-level processes may all contribute to image memory.

Predictive masking of an artificial scotoma is associated with a system-wide reconfiguration of neural populations in the human visual cortex

The visual brain has the remarkable capacity to complete our percept of the world even when the information extracted from the visual scene is incomplete. This ability to predict missing information based on information from spatially adjacent regions is an intriguing attribute of healthy vision. Yet, it gains particular significance when it masks the perceptual consequences of a retinal lesion, leaving patients unaware of their partial loss of vision and ultimately delaying diagnosis and treatment. At present, our understanding of the neural basis of this masking process is limited which hinders both quantitative modeling as well as translational application. To overcome this, we asked the participants to view visual stimuli with and without superimposed artificial scotoma (AS). We used fMRI to record the associated cortical activity and applied model-based analyzes to track changes in cortical population receptive fields and connectivity in response to the introduction of the AS. We found that throughout the visual field and cortical hierarchy, pRFs shifted their preferred position towards the AS border. Moreover, extrastriate areas biased their sampling of V1 towards sections outside the AS projection zone, thereby effectively masking the AS with signals from spared portions of the visual field. We speculate that the signals that drive these system-wide population modifications originate in extrastriate visual areas and, through feedback, also reconfigure the neural populations in the earlier visual areas.

An image-computable model of how endogenous and exogenous attention differentially alter visual perception

Attention alters perception across the visual field. Typically, endogenous (voluntary) and exogenous (involuntary) attention similarly improve performance in many visual tasks, but they have differential effects in some tasks. Extant models of visual attention assume that the effects of these two types of attention are identical and consequently do not explain differences between them. Here, we develop a model of spatial resolution and attention that distinguishes between endogenous and exogenous attention. We focus on texture-based segmentation as a model system because it has revealed a clear dissociation between both attention types. For a texture for which performance peaks at parafoveal locations, endogenous attention improves performance across eccentricity, whereas exogenous attention improves performance where the resolution is low (peripheral locations) but impairs it where the resolution is high (foveal locations) for the scale of the texture. Our model emulates sensory encoding to segment figures from their background and predict behavioral performance. To explain attentional effects, endogenous and exogenous attention require separate operating regimes across visual detail (spatial frequency). Our model reproduces behavioral performance across several experiments and simultaneously resolves three unexplained phenomena: 1) the parafoveal advantage in segmentation, 2) the uniform improvements across eccentricity by endogenous attention, and 3) the peripheral improvements and foveal impairments by exogenous attention. Overall, we unveil a computational dissociation between each attention type and provide a generalizable framework for predicting their effects on perception across the visual field.

On the origin of the Ebbinghaus illusion: The role of figural extent and spatial frequency of stimuli

An object is perceived as larger when it is surrounded by smaller context objects than when it is surrounded by larger context objects. The origin of this well-known phenomenon, called as Ebbinghaus or Titchener circles illusion, is still puzzling. Here we introduce a basic explanation of how this illusion could emerge and provide some preliminary empirical support for this idea. In essence, we suggest that changes in the figural extent and in the spatial frequency of the stimulus pattern entail adjustments of the size and resolution of the attentional field, which are accompanied by changes in spatial coding. This approach is consistent with previous observations and can enable a deeper understanding of geometric illusions.

Feature-based attention processes in primate prefrontal cortex do not rely on feature similarity

Feature-based attention enables privileged processing of specific visual properties. During feature-based attention, neurons in visual cortices show "gain modulation" by enhancing neuronal responses to the features of attended stimuli due to top-down signals originating from prefrontal cortex (PFC). Attentional modulation in visual cortices requires "feature similarity:" neurons only increase their responses when the attended feature variable and the neurons' preferred feature coincide. However, whether gain modulation based on feature similarity is a general attentional mechanism is currently unknown. To address this issue, we record single-unit activity from PFC of macaques trained to switch attention between two conjunctive feature parameters. We find that PFC neurons experience gain modulation in response to attentional demands. However, this attentional gain modulation in PFC is independent of the feature-tuning preferences of neurons. These findings suggest that feature similarity is not a general mechanism in feature-based attention throughout the cortical processing hierarchy.

To look or not to look: dissociating presaccadic and covert spatial attention

Attention is a central neural process that enables selective and efficient processing of visual information. Individuals can attend to specific visual information either overtly, by making an eye movement to an object of interest, or covertly, without moving their eyes. We review behavioral, neuropsychological, neurophysiological, and computational evidence of presaccadic attentional modulations that occur while preparing saccadic eye movements, and highlight their differences from those of covert spatial endogenous (voluntary) and exogenous (involuntary) attention. We discuss recent studies and experimental procedures on how these different types of attention impact visual performance, alter appearance, differentially modulate the featural representation of basic visual dimensions (orientation and spatial frequency), engage different neural computations, and recruit partially distinct neural substrates. We conclude that presaccadic attention and covert attention are dissociable.

Action affects perception through modulation of attention

The present study explored the origin of perceptual changes repeatedly observed in the context of actions. In Experiment 1, participants tried to hit a circular target with a stylus movement under restricted feedback conditions. We measured the perception of target size during action planning and observed larger estimates for larger movement distances. In Experiment 2, we then tested the hypothesis that this action specific influence on perception is due to changes in the allocation of spatial attention. For this purpose, we replaced the hitting task by conditions of focused and distributed attention and measured the perception of the former target stimulus. The results revealed changes in the perceived stimulus size very similar to those observed in Experiment 1. These results indicate that action’s effects on perception root in changes of spatial attention.

Covert Attention Increases the Gain of Stimulus-Evoked Population Codes

Covert spatial attention has a variety of effects on the responses of individual neurons. However, relatively little is known about the net effect of these changes on sensory population codes, even though perception ultimately depends on population activity. Here, we measured the EEG in human observers (male and female), and isolated stimulus-evoked activity that was phase-locked to the onset of attended and ignored visual stimuli. Using an encoding model, we reconstructed spatially selective population tuning functions from the pattern of stimulus-evoked activity across the scalp. Our EEG-based approach allowed us to measure very early visually evoked responses occurring ∼100 ms after stimulus onset. In Experiment 1, we found that covert attention increased the amplitude of spatially tuned population responses at this early stage of sensory processing. In Experiment 2, we parametrically varied stimulus contrast to test how this effect scaled with stimulus contrast. We found that the effect of attention on the amplitude of spatially tuned responses increased with stimulus contrast, and was well described by an increase in response gain (i.e., a multiplicative scaling of the population response). Together, our results show that attention increases the gain of spatial population codes during the first wave of visual processing.SIGNIFICANCE STATEMENT We know relatively little about how attention improves population codes, even though perception is thought to critically depend on population activity. In this study, we used an encoding-model approach to test how attention modulates the spatial tuning of stimulus-evoked population responses measured with EEG. We found that attention multiplicatively scales the amplitude of spatially tuned population responses. Furthermore, this effect was present within 100 ms of stimulus onset. Thus, our results show that attention improves spatial population codes by increasing their gain at this early stage of processing.

The size of attentional focus modulates the perception of object location

The present study examined how the size of attended area affects the repulsion of perceived object location from the focus of attention reported previously (attentional repulsion effect). We induced sustained changes in the size of attentional focus and tested the impact of this experimental variation on the perception of object location. The results of three experiments revealed reliable repulsion effects for each size of attentional focus. However, the magnitude of the effect decreased substantially with an increase in focus size. This outcome extends the knowledge about how spatial attention affects visual perception.

Foveal pRF properties in the visual cortex depend on the extent of stimulated visual field

Previous studies demonstrated that alterations in functional MRI derived receptive field (pRF) properties in cortical projection zones of retinal lesions can erroneously be mistaken for cortical large-scale reorganization in response to visual system pathologies. We tested, whether such confounds are also evident in the normal cortical projection zone of the fovea for simulated peripheral visual field defects. We applied fMRI-based visual field mapping of the central visual field at 3 T in eight controls to compare the pRF properties of the central visual field of a reference condition (stimulus radius: 14°) and two conditions with simulated peripheral visual field defect, i.e., with a peripheral gray mask, stimulating only the central 7° or 4° radius. We quantified, for the cortical representation of the actually stimulated visual field, the changes in the position and size of the pRFs associated with reduced peripheral stimulation using conventional and advanced pRF modeling. We found foveal pRF-positions (≤3°) to be significantly shifted towards the periphery (p<0.05, corrected). These pRF-shifts were largest for the 4° condition [visual area (mean eccentricity shift): V1 (0.9°), V2 (0.9°), V3 (1.0°)], but also evident for the 7° condition [V1 (0.5°), V2 (0.5°), V3 (0.9°)]. Further, an overall enlargement of pRF-sizes was observed. These findings indicate the dependence of foveal pRF parameters on the spatial extent of the stimulated visual field and are likely associated with methodological biases and/or physiological mechanisms. Consequently, our results imply that, previously reported similar findings in patients with actual peripheral scotomas need to be interpreted with caution and indicate the need for adequate control conditions in investigations of visual cortex reorganization.

Mexican Hat Modulation of Visual Acuity Following an Exogenous Cue

Classical models of exogenous attention suggest that attentional enhancement at the focus of attention degrades gradually with distance from the attended location. On the other hand, the Attentional Attraction Field (AAF) model (Baruch and Yeshurun, 2014) suggests that the shift of receptive fields toward the attended location, reported by several physiological studies, leads to a decreased density of RFs at the attentional surrounds and hence the model predicts that the modulation of performance by spatial attention may have the shape of a Mexican Hat. Motivated by these theories, this study presents behavioral evidence in support of a Mexican Hat shaped modulation in exogenous spatial tasks that appears only at short latencies. In two experiments participants had to decide the location of a small gap in a target circle that was preceded by a non-informative attention capturing cue. The distance between cue and target and the latency between their onsets were varied. At short SOAs the performance curves were cubic and only at longer SOAs- this trend turned linear. Our results suggest that a rapid Mexican Hat modulation is an inherent property of the mechanism underlying exogenous attention and that a monotonically degrading trend, such as advocated by classical models, develops only at later stages of processing. The involvements of bottom-up processes such as the attraction of RFs to the focus of attention are further discussed.

Attention strengthens across-trial pre-stimulus phase coherence in visual cortex, enhancing stimulus processing

Attention selectively routes the most behaviorally relevant information from the stream of sensory inputs through the hierarchy of cortical areas. Previous studies have shown that visual attention depends on the phase of oscillatory brain activities. These studies mainly focused on the stimulus presentation period, rather than the pre-stimulus period. Here, we hypothesize that selective attention controls the phase of oscillatory neural activities to efficiently process relevant information. We document an attentional modulation of pre-stimulus inter-trial phase coherence (a measure of deviation between instantaneous phases of trials) of low frequency local field potentials (LFP) in visual area MT of macaque monkeys. Our data reveal that phase coherence increases following a spatial cue deploying attention towards the receptive field of the recorded neural population. We further show that the attentional enhancement of phase coherence is positively correlated with the modulation of the stimulus-induced firing rate, and importantly, a higher phase coherence is associated with a faster behavioral response. These results suggest a functional utilization of intrinsic neural oscillatory activities for an enhanced processing of upcoming stimuli.

On Why Objects Appear Smaller in the Visual Periphery

An object appears smaller in the periphery than in the center of the visual field. In two experiments ( N = 24), we demonstrated that visuospatial attention contributes substantially to this perceptual distortion. Participants judged the size of central and peripheral target objects after a transient, exogenous cue directed their attention to either the central or the peripheral location. Peripheral target objects were judged to be smaller following a central cue, whereas this effect disappeared completely when the peripheral target was cued. This outcome suggests that objects appear smaller in the visual periphery not only because of the structural properties of the visual system but also because of a lack of spatial attention.

Strategic deployment of feature-based attentional gain in primate visual cortex

Attending to visual stimuli enhances the gain of those neurons in primate visual cortex that preferentially respond to the matching locations and features (on-target gain). Although this is well suited to enhance the neuronal representation of attended stimuli, it is nonoptimal under difficult discrimination conditions, as in the presence of similar distractors. In such cases, directing attention to neighboring neuronal populations (off-target gain) has been shown to be the most efficient strategy, but although such a strategic deployment of attention has been shown behaviorally, its underlying neural mechanisms are unknown. Here, we investigated how attention affects the population responses of neurons in the middle temporal (MT) visual area of rhesus monkeys to bidirectional movement inside the neurons' receptive field (RF). The monkeys were trained to focus their attention onto the fixation spot or to detect a direction or speed change in one of the motion directions (the "target"), ignoring the distractor motion. Population activity profiles were determined by systematically varying the patterns' directions while maintaining a constant angle between them. As expected, the response profiles show a peak for each of the 2 motion directions. Switching spatial attention from the fixation spot into the RF enhanced the peak representing the attended stimulus and suppressed the distractor representation. Importantly, the population data show a direction-dependent attentional modulation that does not peak at the target feature but rather along the slopes of the activity profile representing the target direction. Our results show that attentional gains are strategically deployed to optimize the discriminability of target stimuli, in line with an optimal gain mechanism proposed by Navalpakkam and Itti.

Spatial sampling in human visual cortex is modulated by both spatial and feature-based attention

Spatial attention changes the sampling of visual space. Behavioral studies suggest that feature-based attention modulates this resampling to optimize the attended feature's sampling. We investigate this hypothesis by estimating spatial sampling in visual cortex while independently varying both feature-based and spatial attention. Our results show that spatial and feature-based attention interacted: resampling of visual space depended on both the attended location and feature (color vs. temporal frequency). This interaction occurred similarly throughout visual cortex, regardless of an area's overall feature preference. However, the interaction did depend on spatial sampling properties of voxels that prefer the attended feature. These findings are parsimoniously explained by variations in the precision of an attentional gain field. Our results demonstrate that the deployment of spatial attention is tailored to the spatial sampling properties of units that are sensitive to the attended feature.

The attentional repulsion effect and relative size judgments

Rapid shifts of involuntary attention have been shown to induce mislocalizations of nearby objects. One pattern of mislocalization, termed the Attentional Repulsion Effect (ARE), occurs when the onset of peripheral pre-cues lead to perceived shifts of subsequently presented stimuli away from the cued location. While the standard ARE configuration utilizes vernier lines, to date, all previous ARE studies have only assessed distortions along one direction and tested one spatial dimension (i.e., position or shape). The present study assessed the magnitude of the ARE using a novel stimulus configuration. Across three experiments participants judged which of two rectangles on the left or right side of the display appeared wider or taller. Pre-cues were used in Experiments 1 and 2. Results show equivalent perceived expansions in the width and height of the pre-cued rectangle in addition to baseline asymmetries in left/right relative size under no-cue conditions. Altering cue locations led to shifts in the perceived location of the same rectangles, demonstrating distortions in perceived shape and location using the same stimuli and cues. Experiment 3 demonstrates that rectangles are perceived as larger in the periphery compared to fixation, suggesting that eye movements cannot account for results from Experiments 1 and 2. The results support the hypothesis that the ARE reflects a localized, symmetrical warping of visual space that impacts multiple aspects of spatial and object perception.

Optic tectal superficial interneurons detect motion in larval zebrafish

Detection of moving objects is an essential skill for animals to hunt prey, recognize conspecifics and avoid predators. The zebrafish, as a vertebrate model, primarily uses its elaborate visual system to distinguish moving objects against background scenes. The optic tectum (OT) receives and integrates inputs from various types of retinal ganglion cells (RGCs), including direction-selective (DS) RGCs and size-selective RGCs, and is required for both prey capture and predator avoidance. However, it remains largely unknown how motion information is processed within the OT. Here we performed in vivo whole-cell recording and calcium imaging to investigate the role of superficial interneurons (SINs), a specific type of optic tectal neurons, in motion detection of larval zebrafish. SINs mainly receive excitatory synaptic inputs, exhibit transient ON- or OFF-type of responses evoked by light flashes, and possess a large receptive field (RF). One fifth of SINs are DS and classified into two subsets with separate preferred directions. Furthermore, SINs show size-dependent responses to moving dots. They are efficiently activated by moving objects but not static ones, capable of showing sustained responses to moving objects and having less visual adaptation than periventricular neurons (PVNs), the principal tectal cells. Behaviorally, ablation of SINs impairs prey capture, which requires local motion detection, but not global looming-evoked escape. Finally, starvation enhances the gain of SINs' motion responses while maintaining their size tuning and DS. These results indicate that SINs serve as a motion detector for sensing and localizing sized moving objects in the visual field.

The Attentional Suppressive Surround: Eccentricity, Location-Based and Feature-Based Effects and Interactions

The Selective Tuning model of visual attention (Tsotsos, 1990) has proposed that the focus of attention is surrounded by an inhibitory zone, eliciting a center-surround attentional distribution. This attentional suppressive surround inhibits irrelevant information which is located close to attended information in physical space (e.g., Cutzu and Tsotsos, 2003; Hopf et al., 2010) or in feature space (e.g., Tombu and Tsotsos, 2008; Störmer and Alvarez, 2014; Bartsch et al., 2017). In Experiment 1, we investigate the interaction between location-based and feature-based surround suppression and hypothesize that the attentional surround suppression would be maximized when spatially adjacent stimuli are also represented closely within a feature map. Our results demonstrate that perceptual discrimination is worst when two similar orientations are presented in proximity to each other, suggesting the interplay of the two surround suppression mechanisms. The Selective Tuning model also predicts that the size of the attentional suppressive surround is determined by the receptive field size of the neuron which optimally processes the attended information. The receptive field size of the processing neurons is tightly associated with stimulus size and eccentricity. Therefore, Experiment 2 tested the hypothesis that the size of the attentional suppressive surround would become larger as stimulus size and eccentricity increase, corresponding to an increase in the neuron's receptive field size. We show that stimulus eccentricity but not stimulus size modulates the size of the attentional suppressive surround. These results are consistent for both low- and high-level features (e.g., orientation and human faces). Overall, the present study supports the existence of the attentional suppressive surround and reveals new properties of this selection mechanism.

How Visual Cortical Organization Is Altered by Ophthalmologic and Neurologic Disorders

Receptive fields are a core property of cortical organization. Modern neuroimaging allows routine access to visual population receptive fields (pRFs), enabling investigations of clinical disorders. Yet how the underlying neural circuitry operates is controversial. The controversy surrounds observations that measurements of pRFs can change in healthy adults as well as in patients with a range of ophthalmological and neurological disorders. The debate relates to the balance between plasticity and stability of the underlying neural circuitry. We propose that to move the debate forward, the field needs to define the implied mechanism. First, we review the pRF changes in both healthy subjects and those with clinical disorders. Then, we propose a computational model that describes how pRFs can change in healthy humans. We assert that we can correctly interpret the pRF changes in clinical disorders only if we establish the capabilities and limitations of pRF dynamics in healthy humans with mechanistic models that provide quantitative predictions.

Spatial receptive field shift by preceding cross-modal stimulation in the cat superior colliculus

KEY POINTS

It has been known for some time that sensory information of one type can bias the spatial perception of another modality. However, there is a lack of evidence of this occurring in individual neurons. In the present study, we found that the spatial receptive field of superior colliculus multisensory neurons could be dynamically shifted by a preceding stimulus in a different modality. The extent to which the receptive field shifted was dependent on both temporal and spatial gaps between the preceding and following stimuli, as well as the salience of the preceding stimulus. This result provides a neural mechanism that could underlie the process of cross-modal spatial calibration.

ABSTRACT

Psychophysical studies have shown that the different senses can be spatially entrained by each other. This can be observed in certain phenomena, such as ventriloquism, in which a visual stimulus can attract the perceived location of a spatially discordant sound. However, the neural mechanism underlying this cross-modal spatial recalibration has remained unclear, as has whether it takes place dynamically. We explored these issues in multisensory neurons of the cat superior colliculus (SC), a midbrain structure that involves both cross-modal and sensorimotor integration. Sequential cross-modal stimulation showed that the preceding stimulus can shift the receptive field (RF) of the lagging response. This cross-modal spatial calibration took place in both auditory and visual RFs, although auditory RFs shifted slightly more. By contrast, if a preceding stimulus was from the same modality, it failed to induce a similarly substantial RF shift. The extent of the RF shift was dependent on both temporal and spatial gaps between the preceding and following stimuli, as well as the salience of the preceding stimulus. A narrow time gap and high stimulus salience were able to induce larger RF shifts. In addition, when both visual and auditory stimuli were presented simultaneously, a substantial RF shift toward the location-fixed stimulus was also induced. These results, taken together, reveal an online cross-modal process and reflect the details of the organization of SC inter-sensory spatial calibration.

Attention Effects on Neural Population Representations for Shape and Location Are Stronger in the Ventral than Dorsal Stream

We examined how attention causes neural population representations of shape and location to change in ventral stream (AIT) and dorsal stream (LIP). Monkeys performed two identical delayed-match-to-sample (DMTS) tasks, attending either to shape or location. In AIT, shapes were more discriminable when directing attention to shape rather than location, measured by an increase in mean distance between population response vectors. In LIP, attending to location rather than shape did not increase the discriminability of different stimulus locations. Even when factoring out the change in mean vector response distance, multidimensional scaling (MDS) still showed a significant task difference in AIT, but not LIP, indicating that beyond increasing discriminability, attention also causes a nonlinear warping of representation space in AIT. Despite single-cell attentional modulations in both areas, our data show that attentional modulations of population representations are weaker in LIP, likely due to a need to maintain veridical representations for visuomotor control.

Analytic processing of distance

How does a human observer extract from the distance between two frontal points the component corresponding to an axis of a rectangular reference frame? To find out we had participants classify pairs of small circles, varying on the horizontal and vertical axes of a computer screen, in terms of the horizontal distance between them. A response signal controlled response time. The error rate depended on the irrelevant vertical as well as the relevant horizontal distance between the test circles with the relevant distance effect being larger than the irrelevant distance effect. The results implied that the horizontal distance between the test circles was imperfectly extracted from the overall distance between them. The results supported an account, derived from the Exemplar Based Random Walk model (Nosofsky & Palmieri, 1997), under which distance classification is based on the overall distance between the test circles, with relevant distance being extracted from overall distance to the extent that the relevant and irrelevant axes are differentially weighted so as to reduce the contribution of irrelevant distance to overall distance. The results did not support an account, derived from the General Recognition Theory (Ashby & Maddox, 1994), under which distance classification is based on the relevant distance between the test circles, with the irrelevant distance effect arising because a test circle's perceived location on the relevant axis depends on its location on the irrelevant axis, and with relevant distance being extracted from overall distance to the extent that this dependency is absent.

Deactivation of Association Cortices Disrupted the Congruence of Visual and Auditory Receptive Fields in Superior Colliculus Neurons

Physiological and behavioral studies in cats show that corticotectal inputs play a critical role in the information-processing capabilities of neurons in the deeper layers of the superior colliculus (SC). Among them, the sensory inputs from functionally related associational cortices are especially critical for SC multisensory integration. However, the underlying mechanism supporting this influence is still unclear. Here, results demonstrate that deactivation of relevant cortices can both dislocate SC visual and auditory spatial receptive fields (RFs) and decrease their overall size, resulting in reduced alignment. Further analysis demonstrated that this RF separation is significantly correlated with the decrement of neurons' multisensory enhancement and is most pronounced in low stimulus intensity conditions. In addition, cortical deactivation could influence the degree of stimulus effectiveness, thereby illustrating the means by which higher order cortices may modify the multisensory activity of SC.

New insights into the role of motion and form vision in neurodevelopmental disorders

A selective deficit in processing the global (overall) motion, but not form, of spatially extensive objects in the visual scene is frequently associated with several neurodevelopmental disorders, including preterm birth. Existing theories that proposed to explain the origin of this visual impairment are, however, challenged by recent research. In this review, we explore alternative hypotheses for why deficits in the processing of global motion, relative to global form, might arise. We describe recent evidence that has utilised novel tasks of global motion and global form to elucidate the underlying nature of the visual deficit reported in different neurodevelopmental disorders. We also examine the role of IQ and how the sex of an individual can influence performance on these tasks, as these are factors that are associated with performance on global motion tasks, but have not been systematically controlled for in previous studies exploring visual processing in clinical populations. Finally, we suggest that a new theoretical framework is needed for visual processing in neurodevelopmental disorders and present recommendations for future research.

Selective attention within the foveola

Efficient control of attentional resources and high-acuity vision are both fundamental for survival. Shifts in visual attention are known to covertly enhance processing at locations away from the center of gaze, where visual resolution is low. It is unknown, however, whether selective spatial attention operates where the observer already looks, i.e., within the high-acuity foveola, the small, yet disproportionally important rod-free region of the retina. Using new methods for precisely controlling retinal stimulation, here we show that covert attention flexibly improves and speeds-up both detection and discrimination at loci only a fraction of a degree apart within the foveola. These findings reveal a surprisingly precise control of attention and its involvement in fine spatial vision. They show that the commonly studied covert shifts of attention away from the fovea are the expression of a global mechanism that exerts its action across the entire visual field.

Altered visual cortical processing in a mouse model of MECP2 duplication syndrome

As an epigenetic modulator of gene expression, Methyl-CpG binding protein 2 (MeCP2) is essential for normal neurological function. Dysfunction of MeCP2 is associated with a variety of neurological disorders. MECP2 gene duplication in human causes neuropsychiatric symptoms such as mental retardation and autism. MeCP2 overexpression in mice results in neurobehavioural disorders, dendritic abnormalities, and synaptic defects. However, how gain of MeCP2 function influences cortical processing of sensory information remains unclear. In this study, we examined visual processing in a mouse model of MECP2 duplication syndrome (MECP2 Tg1 mouse) at 8 and 14 weeks, which were before and after the onset of behavioural symptoms, respectively. In vivo extracellular recordings from primary visual cortex (V1) showed that neurons in Tg1 mice at both adult ages preferred higher spatial frequencies (SFs) than those in wild-type (WT) littermate controls, and the semi-saturation contrasts of neurons were lower in Tg1 mice at 8 weeks but not at 14 weeks. Behavioural experiments showed that the performance for visual detection at high SFs and low contrasts was higher in MECP2 Tg1 mice. Thus, MeCP2 gain-of-function in mice leads to higher visual acuity and contrast sensitivity, both at the levels of cortical response and behavioural performance.

Temporally evolving gain mechanisms of attention in macaque area V4

Cognitive attention and perceptual saliency jointly govern our interaction with the environment. Yet, we still lack a universally accepted account of the interplay between attention and luminance contrast, a fundamental dimension of saliency. We measured the attentional modulation of V4 neurons' contrast response functions (CRFs) in awake, behaving macaque monkeys and applied a new approach that emphasizes the temporal dynamics of cell responses. We found that attention modulates CRFs via different gain mechanisms during subsequent epochs of visually driven activity: an early contrast-gain, strongly dependent on prestimulus activity changes (baseline shift); a time-limited stimulus-dependent multiplicative modulation, reaching its maximal expression around 150 ms after stimulus onset; and a late resurgence of contrast-gain modulation. Attention produced comparable time-dependent attentional gain changes on cells heterogeneously coding contrast, supporting the notion that the same circuits mediate attention mechanisms in V4 regardless of the form of contrast selectivity expressed by the given neuron. Surprisingly, attention was also sometimes capable of inducing radical transformations in the shape of CRFs. These findings offer important insights into the mechanisms that underlie contrast coding and attention in primate visual cortex and a new perspective on their interplay, one in which time becomes a fundamental factor.NEW & NOTEWORTHY We offer an innovative perspective on the interplay between attention and luminance contrast in macaque area V4, one in which time becomes a fundamental factor. We place emphasis on the temporal dynamics of attentional effects, pioneering the notion that attention modulates contrast response functions of V4 neurons via the sequential engagement of distinct gain mechanisms. These findings advance understanding of attentional influences on visual processing and help reconcile divergent results in the literature.

Spatial working memory alters the efficacy of input to visual cortex

Prefrontal cortex modulates sensory signals in extrastriate visual cortex, in part via its direct projections from the frontal eye field (FEF), an area involved in selective attention. We find that working memory-related activity is a dominant signal within FEF input to visual cortex. Although this signal alone does not evoke spiking responses in areas V4 and MT during memory, the gain of visual responses in these areas increases, and neuronal receptive fields expand and shift towards the remembered location, improving the stimulus representation by neuronal populations. These results provide a basis for enhancing the representation of working memory targets and implicate persistent FEF activity as a basis for the interdependence of working memory and selective attention.

Frontal eye field (FEF) is a visual prefrontal area involved in top-down attention. Here the authors report that FEF neurons projecting to V4/MT are persistently active during spatial working memory, and V4/MT neurons show changes in receptive field and gain at the location held in working memory.

Sites of overt and covert attention define simultaneous spatial reference centers for visuomotor response

The site of overt attention (fixation point) defines a spatial reference center that affects visuomotor response as indicated by the stimulus-response-compatibility (SRC) effect: When subjects press, e.g., a left key to report stimuli, their reaction time is shorter when stimuli appear to the left than to the right of the fixation. Covert attention to a peripheral site appears to define a similar reference center but previous studies did not control for confounding spatiotemporal factors or investigate the relationship between overt- and covert-attention-defined centers. Using an eye tracker to monitor fixation, we found an SRC effect relative to the site of covert attention induced by a flashed cue dot, and a concurrent reduction, but not elimination, of the overt-attention SRC effect. The two SRC effects jointly determined the overall motor reaction time. Since trials with different cue locations were randomly interleaved, the integration of the two reference centers must be updated online. When the cue was invalid and diminished covert attention, the covert-attention SRC effect disappeared and the overt-attention SRC effect retained full strength, excluding non-attention-based interpretations. We conclude that both covert- and overt-attention sites define visual reference centers that simultaneously contribute to motor response.

Visual and presaccadic activity in area 8Ar of the macaque monkey lateral prefrontal cortex

Common trends observed in many visual and oculomotor-related cortical areas include retinotopically organized receptive and movement fields exhibiting a Gaussian shape and increasing size with eccentricity. These trends are demonstrated in the frontal eye fields, many visual areas, and the superior colliculus but have not been thoroughly characterized in prearcuate area 8Ar of the prefrontal cortex. This is important since area 8Ar, located anterior to the frontal eye fields, is more cytoarchitectonically similar to prefrontal areas than premotor areas. Here we recorded the responses of 166 neurons in area 8Ar of two male macaques while the animals made visually guided saccades to a peripheral sine-wave grating stimulus positioned at 1 of 40 possible locations (8 angles along 5 eccentricities). To characterize the neurons' receptive and movement fields, we fit a bivariate Gaussian model to the baseline-subtracted average firing rate during stimulus presentation (early and late visual epochs) and before saccade onset (presaccadic epoch). One hundred twenty-one of one hundred sixty-six neurons showed spatially selective visual and presaccadic responses. Of the visually selective neurons, 76% preferred the contralateral visual hemifield, whereas 24% preferred the ipsilateral hemifield. The angular width of visual and movement-related fields scaled positively with increasing eccentricity. Moreover, responses of neurons with visual receptive fields were modulated by target contrast, exhibiting sigmoid tuning curves that resemble those of visual neurons in upstream areas such as MT and V4. Finally, we found that neurons with receptive fields at similar spatial locations were clustered within the area; however, this organization did not appear retinotopic.NEW & NOTEWORTHY We recorded the responses of neurons in lateral prefrontal area 8Ar of macaques during a visually guided saccade task using multielectrode arrays. Neurons have Gaussian-shaped visual and movement fields in both visual hemifields, with a bias toward the contralateral hemifield. Visual neurons show contrast response functions with sigmoid shapes. Visual neurons tend to cluster at similar locations within the cortical surface; however, this organization does not appear retinotopic.

Spatial Tuning Shifts Increase the Discriminability and Fidelity of Population Codes in Visual Cortex

Selective visual attention enables organisms to enhance the representation of behaviorally relevant stimuli by altering the encoding properties of single receptive fields (RFs). Yet we know little about how the attentional modulations of single RFs contribute to the encoding of an entire visual scene. Addressing this issue requires (1) measuring a group of RFs that tile a continuous portion of visual space, (2) constructing a population-level measurement of spatial representations based on these RFs, and (3) linking how different types of RF attentional modulations change the population-level representation. To accomplish these aims, we used fMRI to characterize the responses of thousands of voxels in retinotopically organized human cortex. First, we found that the response modulations of voxel RFs (vRFs) depend on the spatial relationship between the RF center and the visual location of the attended target. Second, we used two analyses to assess the spatial encoding quality of a population of voxels. We found that attention increased fine spatial discriminability and representational fidelity near the attended target. Third, we linked these findings by manipulating the observed vRF attentional modulations and recomputing our measures of the fidelity of population codes. Surprisingly, we discovered that attentional enhancements of population-level representations largely depend on position shifts of vRFs, rather than changes in size or gain. Our data suggest that position shifts of single RFs are a principal mechanism by which attention enhances population-level representations in visual cortex.SIGNIFICANCE STATEMENT Although changes in the gain and size of RFs have dominated our view of how attention modulates visual information codes, such hypotheses have largely relied on the extrapolation of single-cell responses to population responses. Here we use fMRI to relate changes in single voxel receptive fields (vRFs) to changes in population-level representations. We find that vRF position shifts contribute more to population-level enhancements of visual information than changes in vRF size or gain. This finding suggests that position shifts are a principal mechanism by which spatial attention enhances population codes for relevant visual information. This poses challenges for labeled line theories of information processing, suggesting that downstream regions likely rely on distributed inputs rather than single neuron-to-neuron mappings.

Attention Modifies Spatial Resolution According to Task Demands

How does visual attention affect spatial resolution? In texture-segmentation tasks, exogenous (involuntary) attention automatically increases resolution at the attended location, which improves performance where resolution is too low (at the periphery) but impairs performance where resolution is already too high (at central locations). Conversely, endogenous (voluntary) attention improves performance at all eccentricities, which suggests a more flexible mechanism. Here, using selective adaptation to spatial frequency, we investigated the mechanism by which endogenous attention benefits performance in resolution tasks. Participants detected a texture target that could appear at several eccentricities. Adapting to high or low spatial frequencies selectively affected performance in a manner consistent with changes in resolution. Moreover, adapting to high, but not low, frequencies mitigated the attentional benefit at central locations where resolution was too high; this shows that attention can improve performance by decreasing resolution. Altogether, our results indicate that endogenous attention benefits performance by modulating the contribution of high-frequency information in order to flexibly adjust spatial resolution according to task demands.