Attraction of Position Preference by Spatial Attention throughout Human Visual Cortex

Foveal pRF properties in the visual cortex depend on the extent of stimulated visual field

Previous studies demonstrated that alterations in functional MRI derived receptive field (pRF) properties in cortical projection zones of retinal lesions can erroneously be mistaken for cortical large-scale reorganization in response to visual system pathologies. We tested, whether such confounds are also evident in the normal cortical projection zone of the fovea for simulated peripheral visual field defects. We applied fMRI-based visual field mapping of the central visual field at 3 T in eight controls to compare the pRF properties of the central visual field of a reference condition (stimulus radius: 14°) and two conditions with simulated peripheral visual field defect, i.e., with a peripheral gray mask, stimulating only the central 7° or 4° radius. We quantified, for the cortical representation of the actually stimulated visual field, the changes in the position and size of the pRFs associated with reduced peripheral stimulation using conventional and advanced pRF modeling. We found foveal pRF-positions (≤3°) to be significantly shifted towards the periphery (p<0.05, corrected). These pRF-shifts were largest for the 4° condition [visual area (mean eccentricity shift): V1 (0.9°), V2 (0.9°), V3 (1.0°)], but also evident for the 7° condition [V1 (0.5°), V2 (0.5°), V3 (0.9°)]. Further, an overall enlargement of pRF-sizes was observed. These findings indicate the dependence of foveal pRF parameters on the spatial extent of the stimulated visual field and are likely associated with methodological biases and/or physiological mechanisms. Consequently, our results imply that, previously reported similar findings in patients with actual peripheral scotomas need to be interpreted with caution and indicate the need for adequate control conditions in investigations of visual cortex reorganization.

Mexican Hat Modulation of Visual Acuity Following an Exogenous Cue

Classical models of exogenous attention suggest that attentional enhancement at the focus of attention degrades gradually with distance from the attended location. On the other hand, the Attentional Attraction Field (AAF) model (Baruch and Yeshurun, 2014) suggests that the shift of receptive fields toward the attended location, reported by several physiological studies, leads to a decreased density of RFs at the attentional surrounds and hence the model predicts that the modulation of performance by spatial attention may have the shape of a Mexican Hat. Motivated by these theories, this study presents behavioral evidence in support of a Mexican Hat shaped modulation in exogenous spatial tasks that appears only at short latencies. In two experiments participants had to decide the location of a small gap in a target circle that was preceded by a non-informative attention capturing cue. The distance between cue and target and the latency between their onsets were varied. At short SOAs the performance curves were cubic and only at longer SOAs- this trend turned linear. Our results suggest that a rapid Mexican Hat modulation is an inherent property of the mechanism underlying exogenous attention and that a monotonically degrading trend, such as advocated by classical models, develops only at later stages of processing. The involvements of bottom-up processes such as the attraction of RFs to the focus of attention are further discussed.

The Role of Temporal and Spatial Attention in Size Adaptation

One of the most important tasks for the visual system is to construct an internal representation of the spatial properties of objects, including their size. Size perception includes a combination of bottom-up (retinal inputs) and top-down (e.g., expectations) information, which makes the estimates of object size malleable and susceptible to numerous contextual cues. For example, it has been shown that size perception is prone to adaptation: brief previous presentations of larger or smaller adapting stimuli at the same region of space changes the perceived size of a subsequent test stimulus. Large adapting stimuli cause the test to appear smaller than its veridical size and vice versa. Here, we investigated whether size adaptation is susceptible to attentional modulation. First, we measured the magnitude of adaptation aftereffects for a size discrimination task. Then, we compared these aftereffects (on average 15–20%) with those measured while participants were engaged, during the adaptation phase, in one of the two highly demanding central visual tasks: Multiple Object Tracking (MOT) or Rapid Serial Visual Presentation (RSVP). Our results indicate that deploying visual attention away from the adapters did not significantly affect the distortions of perceived size induced by adaptation, with accuracy and precision in the discrimination task being almost identical in all experimental conditions. Taken together, these results suggest that visual attention does not play a key role in size adaptation, in line with the idea that this phenomenon can be accounted for by local gain control mechanisms within area V1.

Flexible top-down modulation in human ventral temporal cortex

Visual neuroscientists have long characterized attention as inducing a scaling or additive effect on fixed parametric functions describing neural responses (e.g., contrast response functions). Here, we instead propose that top-down effects are more complex and manifest in ways that depend not only on attention but also other cognitive processes involved in executing a task. To substantiate this theory, we analyze fMRI responses in human ventral temporal cortex (VTC) in a study where stimulus eccentricity and cognitive task are varied. We find that as stimuli are presented farther into the periphery, bottom-up stimulus-driven responses decline but top-down attentional enhancement increases substantially. This disproportionate enhancement of weak responses cannot be easily explained by conventional models of attention. Furthermore, we find that attentional effects depend on the specific cognitive task performed by the subject, indicating the influence of additional cognitive processes other than attention (e.g., decision-making). The effects we observe replicate in an independent experiment from the same study, and also generalize to a separate study involving different stimulus manipulations (contrast and phase coherence). Our results suggest that a quantitative understanding of top-down modulation requires more nuanced characterization of the multiple cognitive factors involved in completing a perceptual task.

Stimulus visibility and uncertainty mediate the influence of attention on response bias and visual contrast appearance

Although attention is known to improve the efficacy of sensory processing, the impact of attention on subjective visual appearance is still a matter of debate. Although recent studies suggest that attention can alter the appearance of stimulus contrast, others argue that these changes reflect response bias induced by attention cues. Here, we provide evidence that attention has effects on both appearance and response bias. In a comparative judgment task in which subjects reported whether the attended or unattended visual stimulus had a higher perceived contrast, attention induced substantial baseline-offset response bias as well as small but significant changes in subjective contrast appearance when subjects viewed near-threshold stimuli. However, when subjects viewed suprathreshold stimuli, baseline-offset response bias decreased and attention primarily changed contrast appearance. To address the possibility that these changes in appearance might be influenced by uncertainty due to the attended and unattended stimuli having similar physical contrasts, subjects performed an equality judgment task in which they reported if the contrast of the two stimuli was the same or different. We found that, although there were still attention-induced changes in contrast appearance at lower contrasts, the robust changes in contrast appearance at higher contrasts observed in the comparative judgment task were diminished in the equality judgment task. Together, these results suggest that attention can impact both response bias and appearance, and these two types of attention effects are differentially mediated by stimulus visibility and uncertainty. Collectively, these findings help constrain arguments about the cognitive penetrability of perception.

Mapping sequences can bias population receptive field estimates

Population receptive field (pRF) modelling is a common technique for estimating the stimulus-selectivity of populations of neurons using neuroimaging. Here, we aimed to address if pRF properties estimated with this method depend on the spatio-temporal structure and the predictability of the mapping stimulus. We mapped the polar angle preference and tuning width of voxels in visual cortex (V1-V4) of healthy, adult volunteers. We compared sequences sweeping orderly through the visual field or jumping from location to location employing stimuli of different width (45° vs 6°) and cycles of variable duration (8s vs 60s). While we did not observe any systematic influence of stimulus predictability, the temporal structure of the sequences significantly affected tuning width estimates. Ordered designs with large wedges and short cycles produced systematically smaller estimates than random sequences. Interestingly, when we used small wedges and long cycles, we obtained larger tuning width estimates for ordered than random sequences. We suggest that ordered and random mapping protocols show different susceptibility to other design choices such as stimulus type and duration of the mapping cycle and can produce significantly different pRF results.

Modulating the global orientation bias of the visual system changes population receptive field elongations

The topographical structure of the visual system in individual subjects can be visualized using fMRI. Recently, a radial bias for the long axis of population receptive fields (pRF) has been shown using fMRI. It has been theorized that the elongation of receptive fields pointing toward the fovea results from horizontal local connections bundling orientation selective units mostly parallel to their polar position within the visual field. In order to investigate whether there is a causal relationship between orientation selectivity and pRF elongation the current study employed a global orientation adapter to modulate the orientation bias for the visual system while measuring spatial pRF characteristics. The hypothesis was that the orientation tuning change of neural populations would alter pRF elongations toward the fovea particularly at axial positions parallel and orthogonal to the affected orientation. The results indeed show a different amount of elongation of pRF units and their orientation at parallel and orthogonal axial positions relative to the adapter orientation. Within the lower left hemifield, pRF radial bias and elongation showed an increase during adaptation to a 135° grating while both parameters decreased during the presentation of a 45° adapter stimulus. The lower right visual field showed the reverse pattern. No modulation of the pRF topographies were observed in the upper visual field probably due to a vertical visual field asymmetry of sensitivity toward the low contrast spatial frequency pattern of the adapter stimulus. These data suggest a direct relationship between orientation selectivity and elongation of population units within the visual cortex.

The Importance of Considering Model Choices When Interpreting Results in Computational Neuroimaging

Model-based analyses open exciting opportunities for understanding neural information processing. In a commentary published in eNeuro, Gardner and Liu (2019) discuss the role of model specification in interpreting results derived from complex models of neural data. As a case study, they suggest that one such analysis, the inverted encoding model (IEM), should not be used to assay properties of "stimulus representations" because the ability to apply linear transformations at various stages of the analysis procedure renders results "arbitrary." Here, we argue that the specification of all models is arbitrary to the extent that an experimenter makes choices based on current knowledge of the model system. However, the results derived from any given model, such as the reconstructed channel response profiles obtained from an IEM analysis, are uniquely defined and are arbitrary only in the sense that changes in the model can predictably change results. IEM-based channel response profiles should therefore not be considered arbitrary when the model is clearly specified and guided by our best understanding of neural population representations in the brain regions being analyzed. Intuitions derived from this case study are important to consider when interpreting results from all model-based analyses, which are similarly contingent upon the specification of the models used.

Studying Cortical Plasticity in Ophthalmic and Neurological Disorders: From Stimulus-Driven to Cortical Circuitry Modeling Approaches

Unsolved questions in computational visual neuroscience research are whether and how neurons and their connecting cortical networks can adapt when normal vision is compromised by a neurodevelopmental disorder or damage to the visual system. This question on neuroplasticity is particularly relevant in the context of rehabilitation therapies that attempt to overcome limitations or damage, through either perceptual training or retinal and cortical implants. Studies on cortical neuroplasticity have generally made the assumption that neuronal population properties and the resulting visual field maps are stable in healthy observers. Consequently, differences in the estimates of these properties between patients and healthy observers have been taken as a straightforward indication for neuroplasticity. However, recent studies imply that the modeled neuronal properties and the cortical visual maps vary substantially within healthy participants, e.g., in response to specific stimuli or under the influence of cognitive factors such as attention. Although notable advances have been made to improve the reliability of stimulus-driven approaches, the reliance on the visual input remains a challenge for the interpretability of the obtained results. Therefore, we argue that there is an important role in the study of cortical neuroplasticity for approaches that assess intracortical signal processing and circuitry models that can link visual cortex anatomy, function, and dynamics.

Micro-probing enables fine-grained mapping of neuronal populations using fMRI

The characterization of receptive field (RF) properties is fundamental to understanding the neural basis of sensory and cognitive behaviour. The combination of non-invasive imaging, such as fMRI, with biologically inspired neural modelling has enabled the estimation of population RFs directly in humans. However, current approaches require making numerous a priori assumptions, so these cannot reveal unpredicted properties, such as fragmented RFs or subpopulations. This is a critical limitation in studies on adaptation, pathology or reorganization. Here, we introduce micro-probing (MP), a technique for fine-grained and largely assumption free characterization of multiple pRFs within a voxel. It overcomes many limitations of current approaches by enabling detection of unexpected RF shapes, properties and subpopulations, by enhancing the spatial detail with which we analyze the data. MP is based on tiny, fixed-size, Gaussian models that efficiently sample the entire visual space and create fine-grained probe maps. Subsequently, we derived population receptive fields (pRFs) from these maps. We demonstrate the scope of our method through simulations and by mapping the visual fields of healthy participants and of a patient group with highly abnormal RFs due to a congenital pathway disorder. Without using specific stimuli or adapted models, MP mapped the bilateral pRFs characteristic of observers with albinism. In healthy observers, MP revealed that voxels may capture the activity of multiple subpopulations RFs that sample distinct regions of the visual field. Thus, MP provides a versatile framework to visualize, analyze and model, without restrictions, the diverse RFs of cortical subpopulations in health and disease.

Comparison of human population receptive field estimates between scanners and the effect of temporal filtering

Background: Population receptive field (pRF) analysis with functional magnetic resonance imaging (fMRI) is an increasingly popular method for mapping visual field representations and estimating the spatial selectivity of voxels in human visual cortex. However, the multitude of experimental setups and processing methods used makes comparisons of results between studies difficult. Methods: Here, we compared pRF maps acquired in the same three individuals using comparable scanning parameters on a 1.5 and a 3 Tesla scanner located in two different countries. We also tested the effect of low-pass filtering of the time series on pRF estimates. Results: As expected, the signal-to-noise ratio for the 3 Tesla data was superior; critically, however, estimates of pRF size and cortical magnification did not reveal any systematic differences between the sites. Unsurprisingly, low-pass filtering enhanced goodness-of-fit, presumably by removing high-frequency noise. However, there was no substantial increase in the number of voxels containing meaningful retinotopic signals after low-pass filtering. Importantly, filtering also increased estimates of pRF size in the early visual areas which could substantially skew interpretations of spatial tuning properties. Conclusion: Our results therefore suggest that pRF estimates are generally comparable between scanners of different field strengths, but temporal filtering should be used with caution.

Comparison of human population receptive field estimates between scanners and the effect of temporal filtering

Background: Population receptive field (pRF) analysis with functional magnetic resonance imaging (fMRI) is an increasingly popular method for mapping visual field representations and estimating the spatial selectivity of voxels in human visual cortex. However, the multitude of experimental setups and processing methods used makes comparisons of results between studies difficult. Methods: Here, we compared pRF maps acquired in the same three individuals using comparable scanning parameters on a 1.5 and a 3 Tesla scanner located in two different countries. We also tested the effect of low-pass filtering of the time series on pRF estimates. Results: As expected, the signal-to-noise ratio for the 3 Tesla data was superior; critically, however, estimates of pRF size and cortical magnification did not reveal any systematic differences between the sites. Unsurprisingly, low-pass filtering enhanced goodness-of-fit, presumably by removing high-frequency noise. However, there was no substantial increase in the number of voxels containing meaningful retinotopic signals after low-pass filtering. Importantly, filtering also increased estimates of pRF size in the early visual areas which could substantially skew interpretations of spatial tuning properties. Conclusion: Our results therefore suggest that pRF estimates are generally comparable between scanners of different field strengths, but temporal filtering should be used with caution.

A flexible readout mechanism of human sensory representations

Attention can both enhance and suppress cortical sensory representations. However, changing sensory representations can also be detrimental to behavior. Behavioral consequences can be avoided by flexibly changing sensory readout, while leaving the representations unchanged. Here, we asked human observers to attend to and report about either one of two features which control the visibility of motion while making concurrent measurements of cortical activity with BOLD imaging (fMRI). We extend a well-established linking model to account for the relationship between these measurements and find that changes in sensory representation during directed attention are insufficient to explain perceptual reports. Adding a flexible downstream readout is necessary to best explain our data. Such a model implies that observers should be able to recover information about ignored features, a prediction which we confirm behaviorally. Thus, flexible readout is a critical component of the cortical implementation of human adaptive behavior.

Functional MRI and EEG Index Complementary Attentional Modulations

Functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) are two noninvasive methods commonly used to study neural mechanisms supporting visual attention in humans. Studies using these tools, which have complementary spatial and temporal resolutions, implicitly assume they index similar underlying neural modulations related to external stimulus and internal attentional manipulations. Accordingly, they are often used interchangeably for constraining understanding about the impact of bottom-up and top-down factors on neural modulations. To test this core assumption, we simultaneously manipulated bottom-up sensory inputs by varying stimulus contrast and top-down cognitive modulations by changing the focus of spatial attention. Each of the male and female subjects participated in both fMRI and EEG sessions performing the same experimental paradigm. We found categorically different patterns of attentional modulation on fMRI activity in early visual cortex and early stimulus-evoked potentials measured via EEG (e.g., the P1 component and steady-state visually-evoked potentials): fMRI activation scaled additively with attention, whereas evoked EEG components scaled multiplicatively with attention. However, across longer time scales, a contralateral negative-going potential and oscillatory EEG signals in the alpha band revealed additive attentional modulation patterns like those observed with fMRI. These results challenge prior assumptions that fMRI and early stimulus-evoked potentials measured with EEG can be interchangeably used to index the same neural mechanisms of attentional modulations at different spatiotemporal scales. Instead, fMRI measures of attentional modulations are more closely linked with later EEG components and alpha-band oscillations. Considered together, hemodynamic and electrophysiological signals can jointly constrain understanding of the neural mechanisms supporting cognition.SIGNIFICANCE STATEMENT fMRI and EEG have been used as tools to measure the location and timing of attentional modulations in visual cortex and are often used interchangeably for constraining computational models under the assumption that they index similar underlying neural processes. However, by varying attentional and stimulus parameters, we found differential patterns of attentional modulations of fMRI activity in early visual cortex and commonly used stimulus-evoked potentials measured via EEG. Instead, across longer time scales, a contralateral negative-going potential and EEG oscillations in the alpha band exhibited attentional modulations similar to those observed with fMRI. Together, these results suggest that different physiological processes assayed by these complementary techniques must be jointly considered when making inferences about the neural underpinnings of cognitive operations.

Population receptive field estimates for motion-defined stimuli

The processing of motion changes throughout the visual hierarchy, from spatially restricted ‘local motion’ in early visual cortex to more complex large-field ‘global motion’ at later stages. Here we used functional magnetic resonance imaging (fMRI) to examine spatially selective responses in these areas related to the processing of random-dot stimuli defined by differences in motion. We used population receptive field (pRF) analyses to map retinotopic cortex using bar stimuli comprising coherently moving dots. In the first experiment, we used three separate background conditions: no background dots (dot-defined bar-only), dots moving coherently in the opposite direction to the bar (kinetic boundary) and dots moving incoherently in random directions (global motion). Clear retinotopic maps were obtained for the bar-only and kinetic-boundary conditions across visual areas V1–V3 and in higher dorsal areas. For the global-motion condition, retinotopic maps were much weaker in early areas and became clear only in higher areas, consistent with the emergence of global-motion processing throughout the visual hierarchy. However, in a second experiment we demonstrate that this pattern is not specific to motion-defined stimuli, with very similar results for a transparent-motion stimulus and a bar defined by a static low-level property (dot size) that should have driven responses particularly in V1. We further exclude explanations based on stimulus visibility by demonstrating that the observed differences in pRF properties do not follow the ability of observers to localise or attend to these bar elements. Rather, our findings indicate that dorsal extrastriate retinotopic maps may primarily be determined by the visibility of the neural responses to the bar relative to the background response (i.e. neural signal-to-noise ratios) and suggests that claims about stimulus selectivity from pRF experiments must be interpreted with caution.

Attentional Modulation of Visual Spatial Integration: Psychophysical Evidence Supported by Population Coding Modeling

Two prominent strategies that the human visual system uses to reduce incoming information are spatial integration and selective attention. Whereas spatial integration summarizes and combines information over the visual field, selective attention can single it out for scrutiny. The way in which these well-known mechanisms-with rather opposing effects-interact remains largely unknown. To address this, we had observers perform a gaze-contingent search task that nudged them to deploy either spatial or feature-based attention to maximize performance. We found that, depending on the type of attention employed, visual spatial integration strength changed either in a strong and localized or a more modest and global manner compared with a baseline condition. Population code modeling revealed that a single mechanism can account for both observations: Attention acts beyond the neuronal encoding stage to tune the spatial integration weights of neural populations. Our study shows how attention and integration interact to optimize the information flow through the brain.

Spatial sampling in human visual cortex is modulated by both spatial and feature-based attention

Spatial attention changes the sampling of visual space. Behavioral studies suggest that feature-based attention modulates this resampling to optimize the attended feature's sampling. We investigate this hypothesis by estimating spatial sampling in visual cortex while independently varying both feature-based and spatial attention. Our results show that spatial and feature-based attention interacted: resampling of visual space depended on both the attended location and feature (color vs. temporal frequency). This interaction occurred similarly throughout visual cortex, regardless of an area's overall feature preference. However, the interaction did depend on spatial sampling properties of voxels that prefer the attended feature. These findings are parsimoniously explained by variations in the precision of an attentional gain field. Our results demonstrate that the deployment of spatial attention is tailored to the spatial sampling properties of units that are sensitive to the attended feature.

Temporal Dynamics and Response Modulation across the Human Visual System in a Spatial Attention Task: An ECoG Study

The selection of behaviorally relevant information from cluttered visual scenes (often referred to as "attention") is mediated by a cortical large-scale network consisting of areas in occipital, temporal, parietal, and frontal cortex that is organized into a functional hierarchy of feedforward and feedback pathways. In the human brain, little is known about the temporal dynamics of attentional processing from studies at the mesoscopic level of electrocorticography (ECoG), that combines millisecond temporal resolution with precise anatomical localization of recording sites. We analyzed high-frequency broadband responses (HFB) responses from 626 electrodes implanted in 8 epilepsy patients who performed a spatial attention task. Electrode locations were reconstructed using a probabilistic atlas of the human visual system. HFB responses showed high spatial selectivity and tuning, constituting ECoG response fields (RFs), within and outside the topographic visual system. In accordance with monkey physiology studies, both RF widths and onset latencies increased systematically across the visual processing hierarchy. We used the spatial specificity of HFB responses to quantitatively study spatial attention effects and their temporal dynamics to probe a hierarchical top-down model suggesting that feedback signals back propagate the visual processing hierarchy. Consistent with such a model, the strengths of attentional modulation were found to be greater and modulation latencies to be shorter in posterior parietal cortex, middle temporal cortex and ventral extrastriate cortex compared with early visual cortex. However, inconsistent with such a model, attention effects were weaker and more delayed in anterior parietal and frontal cortex.SIGNIFICANCE STATEMENT In the human brain, visual attention has been predominantly studied using methods with high spatial, but poor temporal resolution such as fMRI, or high temporal, but poor spatial resolution such as EEG/MEG. Here, we investigate temporal dynamics and attention effects across the human visual system at a mesoscopic level that combines precise spatial and temporal measurements by using electrocorticography in epilepsy patients performing a classical spatial attention task. Electrode locations were reconstructed using a probabilistic atlas of the human visual system, thereby relating them to topography and processing hierarchy. We demonstrate regional differences in temporal dynamics across the attention network. Our findings do not fully support a top-down model that promotes influences on visual cortex by reversing the processing hierarchy.

How Ocular Dominance and Binocularity Are Reflected by the Population Receptive Field Properties

Purpose

The neural substrate of binocularity and sighting ocular dominance in humans is not clear. By utilizing the population receptive field (pRF) modeling technique, we explored whether these phenomena are associated with amplitude and pRF size differences.

Methods

The visual field maps of 13 subjects were scanned (3-T Skyra) while viewing drifting bar stimuli. Both eyes (binocular condition), the dominant eye and the nondominant eye (two monocular conditions) were stimulated in separate sessions. For each condition, pRF size and amplitude were assessed. Binocular summation ratios were calculated by dividing binocular by mean monocular values (amplitude and pRF size).

Results

No differences in pRF size were seen between the viewing conditions within each region, that is, either between binocular and monocular or between dominant and nondominant viewing conditions. Binocular amplitudes were higher than the monocular amplitudes, but similar among the dominant and nondominant eyes. Binocular summation ratios derived from amplitudes were significantly higher than one (∼1.2), while those ratios derived from pRF size were not. These effects were found in all studied areas along the visual hierarchy, starting in V1.

Conclusions

Neither the amplitude nor the pRF size show intereye difference and therefore cannot explain the different roles of the dominant and the nondominant eyes. Binocular, as compared to monocular vision, resulted in higher amplitudes, while receptive fields' sizes were similar, suggesting increased binocular response intensity as the basis for the binocular summation phenomenon. Our results could be applicable in imaging studies of monocular disease and studies that deal with nondisparity binocularity effects.

Attention alters spatial resolution by modulating second-order processing

Endogenous and exogenous visuospatial attention both alter spatial resolution, but they operate via distinct mechanisms. In texture segmentation tasks, exogenous attention inflexibly increases resolution even when detrimental for the task at hand and does so by modulating second-order processing. Endogenous attention is more flexible and modulates resolution to benefit performance according to task demands, but it is unknown whether it also operates at the second-order level. To answer this question, we measured performance on a second-order texture segmentation task while independently manipulating endogenous and exogenous attention. Observers discriminated a second-order texture target at several eccentricities. We found that endogenous attention improved performance uniformly across eccentricity, suggesting a flexible mechanism that can increase or decrease resolution based on task demands. In contrast, exogenous attention improved performance in the periphery but impaired it at central retinal locations, consistent with an inflexible resolution enhancement. Our results reveal that endogenous and exogenous attention both alter spatial resolution by differentially modulating second-order processing.

How Visual Cortical Organization Is Altered by Ophthalmologic and Neurologic Disorders

Receptive fields are a core property of cortical organization. Modern neuroimaging allows routine access to visual population receptive fields (pRFs), enabling investigations of clinical disorders. Yet how the underlying neural circuitry operates is controversial. The controversy surrounds observations that measurements of pRFs can change in healthy adults as well as in patients with a range of ophthalmological and neurological disorders. The debate relates to the balance between plasticity and stability of the underlying neural circuitry. We propose that to move the debate forward, the field needs to define the implied mechanism. First, we review the pRF changes in both healthy subjects and those with clinical disorders. Then, we propose a computational model that describes how pRFs can change in healthy humans. We assert that we can correctly interpret the pRF changes in clinical disorders only if we establish the capabilities and limitations of pRF dynamics in healthy humans with mechanistic models that provide quantitative predictions.

The functional neuroanatomy of face perception: from brain measurements to deep neural networks

A central goal in neuroscience is to understand how processing within the ventral visual stream enables rapid and robust perception and recognition. Recent neuroscientific discoveries have significantly advanced understanding of the function, structure and computations along the ventral visual stream that serve as the infrastructure supporting this behaviour. In parallel, significant advances in computational models, such as hierarchical deep neural networks (DNNs), have brought machine performance to a level that is commensurate with human performance. Here, we propose a new framework using the ventral face network as a model system to illustrate how increasing the neural accuracy of present DNNs may allow researchers to test the computational benefits of the functional architecture of the human brain. Thus, the review (i) considers specific neural implementational features of the ventral face network, (ii) describes similarities and differences between the functional architecture of the brain and DNNs, and (iii) provides a hypothesis for the computational value of implementational features within the brain that may improve DNN performance. Importantly, this new framework promotes the incorporation of neuroscientific findings into DNNs in order to test the computational benefits of fundamental organizational features of the visual system.

Dissociable signatures of visual salience and behavioral relevance across attentional priority maps in human cortex

Computational models posit that visual attention is guided by activity within spatial maps that index the image-computable salience and the behavioral relevance of objects in the scene. These spatial maps are theorized to be instantiated as activation patterns across a series of retinotopic visual regions in occipital, parietal, and frontal cortex. Whereas previous research has identified sensitivity to either the behavioral relevance or the image-computable salience of different scene elements, the simultaneous influence of these factors on neural "attentional priority maps" in human cortex is not well understood. We tested the hypothesis that visual salience and behavioral relevance independently impact the activation profile across retinotopically organized cortical regions by quantifying attentional priority maps measured in human brains using functional MRI while participants attended one of two differentially salient stimuli. We found that the topography of activation in priority maps, as reflected in the modulation of region-level patterns of population activity, independently indexed the physical salience and behavioral relevance of each scene element. Moreover, salience strongly impacted activation patterns in early visual areas, whereas later visual areas were dominated by relevance. This suggests that prioritizing spatial locations relies on distributed neural codes containing graded representations of salience and relevance across the visual hierarchy. NEW & NOTEWORTHY We tested a theory which supposes that neural systems represent scene elements according to both their salience and their relevance in a series of "priority maps" by measuring functional MRI activation patterns across human brains and reconstructing spatial maps of the visual scene. We found that different regions indexed either the salience or the relevance of scene items, but not their interaction, suggesting an evolving representation of salience and relevance across different visual areas.

Development differentially sculpts receptive fields across early and high-level human visual cortex

Receptive fields (RFs) processing information in restricted parts of the visual field are a key property of visual system neurons. However, how RFs develop in humans is unknown. Using fMRI and population receptive field (pRF) modeling in children and adults, we determine where and how pRFs develop across the ventral visual stream. Here we report that pRF properties in visual field maps, from the first visual area, V1, through the first ventro-occipital area, VO1, are adult-like by age 5. However, pRF properties in face-selective and character-selective regions develop into adulthood, increasing the foveal coverage bias for faces in the right hemisphere and words in the left hemisphere. Eye-tracking indicates that pRF changes are related to changing fixation patterns on words and faces across development. These findings suggest a link between face and word viewing behavior and the differential development of pRFs across visual cortex, potentially due to competition on foveal coverage.

Population receptive fields (pRFs) in the visual system are key information-processors, but how they develop is unknown. Here, authors use fMRI and pRF modeling in children and adults to show that in the ventral stream only pRFs in face- and word-selective regions continue to develop, mirroring changes in viewing behavior.

Simultaneous estimation of population receptive field and hemodynamic parameters from single point BOLD responses using Metropolis-Hastings sampling

We introduce a new approach to Bayesian pRF model estimation using Markov Chain Monte Carlo (MCMC) sampling for simultaneous estimation of pRF and hemodynamic parameters. To obtain high performance on commonly accessible hardware we present a novel heuristic consisting of interpolation between precomputed responses for predetermined stimuli and a large cross-section of receptive field parameters. We investigate the validity of the proposed approach with respect to MCMC convergence, tuning and biases. We compare different combinations of pRF - Compressive Spatial Summation (CSS), Dumoulin-Wandell (DW) and hemodynamic (5-parameter and 3-parameter Balloon-Windkessel) models within our framework with and without the usage of the new heuristic. We evaluate estimation consistency and log probability across models. We perform as well a comparison of one model with and without lookup table within the RStan framework using its No-U-Turn Sampler. We present accelerated computation of whole-ROI parameters for one subject. Finally, we discuss risks and limitations associated with the usage of the new heuristic as well as the means of resolving them. We found that the new algorithm is a valid sampling approach to joint pRF/hemodynamic parameter estimation and that it exhibits very high performance.

Extensive Tonotopic Mapping across Auditory Cortex Is Recapitulated by Spectrally Directed Attention and Systematically Related to Cortical Myeloarchitecture

Auditory selective attention is vital in natural soundscapes. But it is unclear how attentional focus on the primary dimension of auditory representation—acoustic frequency—might modulate basic auditory functional topography during active listening. In contrast to visual selective attention, which is supported by motor-mediated optimization of input across saccades and pupil dilation, the primate auditory system has fewer means of differentially sampling the world. This makes spectrally-directed endogenous attention a particularly crucial aspect of auditory attention. Using a novel functional paradigm combined with quantitative MRI, we establish in male and female listeners that human frequency-band-selective attention drives activation in both myeloarchitectonically estimated auditory core, and across the majority of tonotopically mapped nonprimary auditory cortex. The attentionally driven best-frequency maps show strong concordance with sensory-driven maps in the same subjects across much of the temporal plane, with poor concordance in areas outside traditional auditory cortex. There is significantly greater activation across most of auditory cortex when best frequency is attended, versus ignored; the same regions do not show this enhancement when attending to the least-preferred frequency band. Finally, the results demonstrate that there is spatial correspondence between the degree of myelination and the strength of the tonotopic signal across a number of regions in auditory cortex. Strong frequency preferences across tonotopically mapped auditory cortex spatially correlate with R₁-estimated myeloarchitecture, indicating shared functional and anatomical organization that may underlie intrinsic auditory regionalization.

SIGNIFICANCE STATEMENT Perception is an active process, especially sensitive to attentional state. Listeners direct auditory attention to track a violin's melody within an ensemble performance, or to follow a voice in a crowded cafe. Although diverse pathologies reduce quality of life by impacting such spectrally directed auditory attention, its neurobiological bases are unclear. We demonstrate that human primary and nonprimary auditory cortical activation is modulated by spectrally directed attention in a manner that recapitulates its tonotopic sensory organization. Further, the graded activation profiles evoked by single-frequency bands are correlated with attentionally driven activation when these bands are presented in complex soundscapes. Finally, we observe a strong concordance in the degree of cortical myelination and the strength of tonotopic activation across several auditory cortical regions.

Decoding spoken phonemes from sensorimotor cortex with high-density ECoG grids

For people who cannot communicate due to severe paralysis or involuntary movements, technology that decodes intended speech from the brain may offer an alternative means of communication. If decoding proves to be feasible, intracranial Brain-Computer Interface systems can be developed which are designed to translate decoded speech into computer generated speech or to instructions for controlling assistive devices. Recent advances suggest that such decoding may be feasible from sensorimotor cortex, but it is not clear how this challenge can be approached best. One approach is to identify and discriminate elements of spoken language, such as phonemes. We investigated feasibility of decoding four spoken phonemes from the sensorimotor face area, using electrocorticographic signals obtained with high-density electrode grids. Several decoding algorithms including spatiotemporal matched filters, spatial matched filters and support vector machines were compared. Phonemes could be classified correctly at a level of over 75% with spatiotemporal matched filters. Support Vector machine analysis reached a similar level, but spatial matched filters yielded significantly lower scores. The most informative electrodes were clustered along the central sulcus. Highest scores were achieved from time windows centered around voice onset time, but a 500 ms window before onset time could also be classified significantly. The results suggest that phoneme production involves a sequence of robust and reproducible activity patterns on the cortical surface. Importantly, decoding requires inclusion of temporal information to capture the rapid shifts of robust patterns associated with articulator muscle group contraction during production of a phoneme. The high classification scores are likely to be enabled by the use of high density grids, and by the use of discrete phonemes. Implications for use in Brain-Computer Interfaces are discussed.

The Functional Neuroanatomy of Human Face Perception

Face perception is critical for normal social functioning and is mediated by a network of regions in the ventral visual stream. In this review, we describe recent neuroimaging findings regarding the macro- and microscopic anatomical features of the ventral face network, the characteristics of white matter connections, and basic computations performed by population receptive fields within face-selective regions composing this network. We emphasize the importance of the neural tissue properties and white matter connections of each region, as these anatomical properties may be tightly linked to the functional characteristics of the ventral face network. We end by considering how empirical investigations of the neural architecture of the face network may inform the development of computational models and shed light on how computations in the face network enable efficient face perception.

Invariance of surface color representations across illuminant changes in the human cortex

A central problem in color vision is that the light reaching the eye from a given surface can vary dramatically depending on the illumination. Despite this, our color percept, the brain's estimate of surface reflectance, remains remarkably stable. This phenomenon is called color constancy. Here we investigated which human brain regions represent surface color in a way that is invariant with respect to illuminant changes. We used physically realistic rendering methods to display natural yet abstract 3D scenes that were displayed under three distinct illuminants. The scenes embedded, in different conditions, surfaces that differed in their surface color (i.e. in their reflectance property). We used multivariate fMRI pattern analysis to probe neural coding of surface reflectance and illuminant, respectively. While all visual regions encoded surface color when viewed under the same illuminant, we found that only in V1 and V4α surface color representations were invariant to illumination changes. Along the visual hierarchy there was a gradient from V1 to V4α to increasingly encode surface color rather than illumination. Finally, effects of a stimulus manipulation on individual behavioral color constancy indices correlated with neural encoding of the illuminant in hV4. This provides neural evidence for the Equivalent Illuminant Model. Our results provide a principled characterization of color constancy mechanisms across the visual hierarchy, and demonstrate complementary contributions in early and late processing stages.

Sites of overt and covert attention define simultaneous spatial reference centers for visuomotor response

The site of overt attention (fixation point) defines a spatial reference center that affects visuomotor response as indicated by the stimulus-response-compatibility (SRC) effect: When subjects press, e.g., a left key to report stimuli, their reaction time is shorter when stimuli appear to the left than to the right of the fixation. Covert attention to a peripheral site appears to define a similar reference center but previous studies did not control for confounding spatiotemporal factors or investigate the relationship between overt- and covert-attention-defined centers. Using an eye tracker to monitor fixation, we found an SRC effect relative to the site of covert attention induced by a flashed cue dot, and a concurrent reduction, but not elimination, of the overt-attention SRC effect. The two SRC effects jointly determined the overall motor reaction time. Since trials with different cue locations were randomly interleaved, the integration of the two reference centers must be updated online. When the cue was invalid and diminished covert attention, the covert-attention SRC effect disappeared and the overt-attention SRC effect retained full strength, excluding non-attention-based interpretations. We conclude that both covert- and overt-attention sites define visual reference centers that simultaneously contribute to motor response.

The impact of ultra-high field MRI on cognitive and computational neuroimaging

The ability to measure functional brain responses non-invasively with ultra high field MRI (7 T and above) represents a unique opportunity in advancing our understanding of the human brain. Compared to lower fields (3 T and below), ultra high field MRI has an increased sensitivity, which can be used to acquire functional images with greater spatial resolution, and greater specificity of the blood oxygen level dependent (BOLD) signal to the underlying neuronal responses. Together, increased resolution and specificity enable investigating brain functions at a submillimeter scale, which so far could only be done with invasive techniques. At this mesoscopic spatial scale, perception, cognition and behavior can be probed at the level of fundamental units of neural computations, such as cortical columns, cortical layers, and subcortical nuclei. This represents a unique and distinctive advantage that differentiates ultra high from lower field imaging and that can foster a tighter link between fMRI and computational modeling of neural networks. So far, functional brain mapping at submillimeter scale has focused on the processing of sensory information and on well-known systems for which extensive information is available from invasive recordings in animals. It remains an open challenge to extend this methodology to uniquely human functions and, more generally, to systems for which animal models may be problematic. To succeed, the possibility to acquire high-resolution functional data with large spatial coverage, the availability of computational models of neural processing as well as accurate biophysical modeling of neurovascular coupling at mesoscopic scale all appear necessary.

Spatial Tuning Shifts Increase the Discriminability and Fidelity of Population Codes in Visual Cortex

Selective visual attention enables organisms to enhance the representation of behaviorally relevant stimuli by altering the encoding properties of single receptive fields (RFs). Yet we know little about how the attentional modulations of single RFs contribute to the encoding of an entire visual scene. Addressing this issue requires (1) measuring a group of RFs that tile a continuous portion of visual space, (2) constructing a population-level measurement of spatial representations based on these RFs, and (3) linking how different types of RF attentional modulations change the population-level representation. To accomplish these aims, we used fMRI to characterize the responses of thousands of voxels in retinotopically organized human cortex. First, we found that the response modulations of voxel RFs (vRFs) depend on the spatial relationship between the RF center and the visual location of the attended target. Second, we used two analyses to assess the spatial encoding quality of a population of voxels. We found that attention increased fine spatial discriminability and representational fidelity near the attended target. Third, we linked these findings by manipulating the observed vRF attentional modulations and recomputing our measures of the fidelity of population codes. Surprisingly, we discovered that attentional enhancements of population-level representations largely depend on position shifts of vRFs, rather than changes in size or gain. Our data suggest that position shifts of single RFs are a principal mechanism by which attention enhances population-level representations in visual cortex.SIGNIFICANCE STATEMENT Although changes in the gain and size of RFs have dominated our view of how attention modulates visual information codes, such hypotheses have largely relied on the extrapolation of single-cell responses to population responses. Here we use fMRI to relate changes in single voxel receptive fields (vRFs) to changes in population-level representations. We find that vRF position shifts contribute more to population-level enhancements of visual information than changes in vRF size or gain. This finding suggests that position shifts are a principal mechanism by which spatial attention enhances population codes for relevant visual information. This poses challenges for labeled line theories of information processing, suggesting that downstream regions likely rely on distributed inputs rather than single neuron-to-neuron mappings.

The Distributed Nature of Working Memory

Studies in humans and non-human primates have provided evidence for storage of working memory contents in multiple regions ranging from sensory to parietal and prefrontal cortex. We discuss potential explanations for these distributed representations: (i) features in sensory regions versus prefrontal cortex differ in the level of abstractness and generalizability; and (ii) features in prefrontal cortex reflect representations that are transformed for guidance of upcoming behavioral actions. We propose that the propensity to produce persistent activity is a general feature of cortical networks. Future studies may have to shift focus from asking where working memory can be observed in the brain to how a range of specialized brain areas together transform sensory information into a delayed behavioral response.

Attention Modifies Spatial Resolution According to Task Demands

How does visual attention affect spatial resolution? In texture-segmentation tasks, exogenous (involuntary) attention automatically increases resolution at the attended location, which improves performance where resolution is too low (at the periphery) but impairs performance where resolution is already too high (at central locations). Conversely, endogenous (voluntary) attention improves performance at all eccentricities, which suggests a more flexible mechanism. Here, using selective adaptation to spatial frequency, we investigated the mechanism by which endogenous attention benefits performance in resolution tasks. Participants detected a texture target that could appear at several eccentricities. Adapting to high or low spatial frequencies selectively affected performance in a manner consistent with changes in resolution. Moreover, adapting to high, but not low, frequencies mitigated the attentional benefit at central locations where resolution was too high; this shows that attention can improve performance by decreasing resolution. Altogether, our results indicate that endogenous attention benefits performance by modulating the contribution of high-frequency information in order to flexibly adjust spatial resolution according to task demands.

Visual attention spreads broadly but selects information locally

Visual attention spreads over a range around the focus as the spotlight metaphor describes. Spatial spread of attentional enhancement and local selection/inhibition are crucial factors determining the profile of the spatial attention. Enhancement and ignorance/suppression are opposite effects of attention, and appeared to be mutually exclusive. Yet, no unified view of the factors has been provided despite their necessity for understanding the functions of spatial attention. This report provides electroencephalographic and behavioral evidence for the attentional spread at an early stage and selection/inhibition at a later stage of visual processing. Steady state visual evoked potential showed broad spatial tuning whereas the P3 component of the event related potential showed local selection or inhibition of the adjacent areas. Based on these results, we propose a two-stage model of spatial attention with broad spread at an early stage and local selection at a later stage.

Separate spatial and temporal frequency tuning to visual motion in human MT+ measured with ECoG

The human middle temporal complex (hMT+) has a crucial biological relevance for the processing and detection of direction and speed of motion in visual stimuli. Here, we characterized how neuronal populations in hMT+ encode the speed of moving visual stimuli. We evaluated human intracranial electrocorticography (ECoG) responses elicited by square-wave dartboard moving stimuli with different spatial and temporal frequency to investigate whether hMT+ neuronal populations encode the stimulus speed directly, or whether they separate motion into its spatial and temporal components. We extracted two components from the ECoG responses: (1) the power in the high-frequency band (HFB: 65-95 Hz) as a measure of the neuronal population spiking activity and (2) a specific spectral component that followed the frequency of the stimulus's contrast reversals (SCR responses). Our results revealed that HFB neuronal population responses to visual motion stimuli exhibit distinct and independent selectivity for spatial and temporal frequencies of the visual stimuli rather than direct speed tuning. The SCR responses did not encode the speed or the spatiotemporal frequency of the visual stimuli. We conclude that the neuronal populations measured in hMT+ are not directly tuned to stimulus speed, but instead encode speed through separate and independent spatial and temporal frequency tuning. Hum Brain Mapp 38:293-307, 2017. © 2016 Wiley Periodicals, Inc.

Eyetracker-based gaze correction for robust mapping of population receptive fields

Functional MRI enables the acquisition of a retinotopic map that relates regions of the visual field to neural populations in the visual cortex. During such a "population receptive field" (PRF) experiment, stable gaze fixation is of utmost importance in order to correctly link the presented stimulus patterns to stimulated retinal regions and the resulting Blood Oxygen Level Dependent (BOLD) response of the appropriate region within the visual cortex. A method is described that compensates for unstable gaze fixation by recording gaze position via an eyetracker and subsequently modifies the input stimulus underlying the PRF analysis according to the eyetracking measures. Here we show that PRF maps greatly improve when the method is applied to data acquired with either saccadic or smooth eye movements. We conclude that the technique presented herein is useful for studies involving subjects with unstable gaze fixation, particularly elderly patient populations.

Does Congenital Deafness Affect the Structural and Functional Architecture of Primary Visual Cortex?

Deafness results in greater reliance on the remaining senses. It is unknown whether the cortical architecture of the intact senses is optimized to compensate for lost input. Here we performed widefield population receptive field (pRF) mapping of primary visual cortex (V1) with functional magnetic resonance imaging (fMRI) in hearing and congenitally deaf participants, all of whom had learnt sign language after the age of 10 years. We found larger pRFs encoding the peripheral visual field of deaf compared to hearing participants. This was likely driven by larger facilitatory center zones of the pRF profile concentrated in the near and far periphery in the deaf group. pRF density was comparable between groups, indicating pRFs overlapped more in the deaf group. This could suggest that a coarse coding strategy underlies enhanced peripheral visual skills in deaf people. Cortical thickness was also decreased in V1 in the deaf group. These findings suggest deafness causes structural and functional plasticity at the earliest stages of visual cortex.

Attention and multisensory modulation argue against total encapsulation

Firestone & Scholl (F&S) postulate that vision proceeds without any direct interference from cognition. We argue that this view is extreme and not in line with the available evidence. Specifically, we discuss two well-established counterexamples: Attention directly affects core aspects of visual processing, and multisensory modulations of vision originate on multiple levels, some of which are unlikely to fall "within perception."

Visual motion transforms visual space representations similarly throughout the human visual hierarchy

Several studies demonstrate that visual stimulus motion affects neural receptive fields and fMRI response amplitudes. Here we unite results of these two approaches and extend them by examining the effects of visual motion on neural position preferences throughout the hierarchy of human visual field maps. We measured population receptive field (pRF) properties using high-field fMRI (7T), characterizing position preferences simultaneously over large regions of the visual cortex. We measured pRFs properties using sine wave gratings in stationary apertures, moving at various speeds in either the direction of pRF measurement or the orthogonal direction. We find direction- and speed-dependent changes in pRF preferred position and size in all visual field maps examined, including V1, V3A, and the MT+ map TO1. These effects on pRF properties increase up the hierarchy of visual field maps. However, both within and between visual field maps the extent of pRF changes was approximately proportional to pRF size. This suggests that visual motion transforms the representation of visual space similarly throughout the visual hierarchy. Visual motion can also produce an illusory displacement of perceived stimulus position. We demonstrate perceptual displacements using the same stimulus configuration. In contrast to effects on pRF properties, perceptual displacements show only weak effects of motion speed, with far larger speed-independent effects. We describe a model where low-level mechanisms could underlie the observed effects on neural position preferences. We conclude that visual motion induces similar transformations of visuo-spatial representations throughout the visual hierarchy, which may arise through low-level mechanisms.