Computational neuroimaging and population receptive fields

Visuospatial computations vary by category and stream and continue to develop in adolescence

Reading, face recognition, and navigation are supported by visuospatial computations in category-selective regions across ventral, lateral, and dorsal visual streams. However, the nature of visuospatial computations across streams and their development in adolescence remain unknown. Using fMRI and population receptive field (pRF) modeling in adolescents and adults, we estimate pRFs in high-level visual cortex and determine their development. Results reveal that pRF location, size, and visual field coverage vary across category, stream, and hemisphere in both adolescents and adults. While pRF location is mature by adolescence, pRF size and visual field coverage continue to develop - increasing in face-selective and decreasing in place-selective regions - alongside similar development of category selectivity. These findings provide a timeline for differential development of visual functions and suggest that visuospatial computations in high-level visual cortex continue to be optimized to accommodate both category and stream demands through adolescence.

An Eccentricity Gradient Reversal across High-Level Visual Cortex

Human visual cortex contains regions selectively involved in perceiving and recognizing ecologically important visual stimuli such as people and places. Located in the ventral temporal lobe, these regions are organized consistently relative to cortical folding, a phenomenon thought to be inherited from how centrally or peripherally these stimuli are viewed with the retina. While this eccentricity theory of visual cortex has been one of the best descriptions of its functional organization, whether or not it accurately describes visual processing in all category-selective regions is not yet clear. Through a combination of behavioral and functional MRI measurements in 27 participants (17 females), we demonstrate that a limb-selective region neighboring well-studied face-selective regions shows tuning for the visual periphery in a cortical region originally thought to be centrally biased. We demonstrate that the spatial computations performed by the limb-selective region are consistent with visual experience and in doing so, make the novel observation that there may in fact be two eccentricity gradients, forming an eccentricity reversal across high-level visual cortex. These data expand the current theory of cortical organization to provide a unifying principle that explains the broad functional features of many visual regions, showing that viewing experience interacts with innate wiring principles to drive the location of cortical specialization.

Multiple loci for foveolar vision in macaque monkey visual cortex

In humans and nonhuman primates, the central 1° of vision is processed by the foveola, a retinal structure that comprises a high density of photoreceptors and is crucial for primate-specific high-acuity vision, color vision and gaze-directed visual attention. Here, we developed high-spatial-resolution ultrahigh-field 7T functional magnetic resonance imaging methods for functional mapping of the foveolar visual cortex in awake monkeys. In the ventral pathway (visual areas V1-V4 and the posterior inferior temporal cortex), viewing of a small foveolar spot elicits a ring of multiple (eight) foveolar representations per hemisphere. This ring surrounds an area called the 'foveolar core', which is populated by millimeter-scale functional domains sensitive to fine stimuli and high spatial frequencies, consistent with foveolar visual acuity, color and achromatic information and motion. Thus, this elaborate rerepresentation of central vision coupled with a previously unknown foveolar core area signifies a cortical specialization for primate foveation behaviors.

Retinotopic cortical mapping in objective functional monitoring of macular therapy

BACKGROUND/AIMS

To determine the suitability of functional MRI (fMRI) as an objective measure of macular function following therapeutic intervention; conventional psychophysical measures rely heavily on patient compliance.

METHODS

Twenty patients with neovascular age-related macular degeneration (nAMD) were studied with high-resolution fMRI, visual acuity, reading accuracy and speed, contrast sensitivity (CS) and microperimetry (MP) before and after 3 monthly intravitreal injections of ranibizumab. Population-receptive field retinotopic maps calculated from fMRI data were compared with psychophysical measures and optical coherence tomography.

RESULTS

Best-corrected visual acuity (BCVA) responders (≥5 letters) showed an increase of 29.5% in activated brain area, while non-responders showed a decrease of 0.8%. Radial histograms over eccentricity allowed quantification of the absolute number of significant voxels and thus differences before and after treatment. Responders showed increases in foveal (α<0.5°) activation, while non-responders did not. Absence of intraretinal fluid and preservation of outer retinal layers was associated with higher numbers of active V1 voxels and better BCVA. Higher voxel numbers were associated with improved reading performance and, less marked, with BCVA, CS and MP.

CONCLUSION

The data show that retinotopic mapping using fMRI can successfully be applied objectively to evaluate the therapeutic response in nAMD patients treated with anti-vascular endothelial growth factor therapy. This demonstrates the ability of retinotopic mapping to provide an objective assessment of functional recovery at a cortical level; the technique can therefore be applied, in other degenerative macular diseases, to the assessment of potential therapeutic interventions such as gene therapy or cell replacement therapy.

Are neuronal mechanisms of attention universal across human sensory and motor brain maps?

One's experience of shifting attention from the color to the smell to the act of picking a flower seems like a unitary process applied, at will, to one modality after another. Yet, the unique and separable experiences of sight versus smell versus movement might suggest that the neural mechanisms of attention have been separately optimized to employ each modality to its greatest advantage. Moreover, addressing the issue of universality can be particularly difficult due to a paucity of existing cross-modal comparisons and a dearth of neurophysiological methods that can be applied equally well across disparate modalities. Here we outline some of the conceptual and methodological issues related to this problem and present an instructive example of an experimental approach that can be applied widely throughout the human brain to permit detailed, quantitative comparison of attentional mechanisms across modalities. The ultimate goal is to spur efforts across disciplines to provide a large and varied database of empirical observations that will either support the notion of a universal neural substrate for attention or more clearly identify the degree to which attentional mechanisms are specialized for each modality.

Rapid effects of tryptamine psychedelics on perceptual distortions and early visual cortical population receptive fields

N, N-dimethyltryptamine (DMT) is a psychedelic tryptamine acting on 5-HT2A serotonin receptors, which is associated with intense visual hallucinatory phenomena and perceptual changes such as distortions in visual space. The neural underpinnings of these effects remain unknown. We hypothesised that changes in population receptive field (pRF) properties in the primary visual cortex (V1) might underlie visual perceptual experience. We tested this hypothesis using magnetic resonance imaging (MRI) in a within-subject design. We used a technique called pRF mapping, which measures neural population visual response properties and retinotopic maps in early visual areas. We show that in the presence of visual effects, as documented by the Hallucinogen Rating Scale (HRS), the mean pRF sizes in V1 significantly increase in the peripheral visual field for active condition (inhaled DMT) compared to the control. Eye and head movement differences were absent across conditions. This evidence for short-term effects of DMT in pRF may explain perceptual distortions induced by psychedelics such as field blurring, tunnel vision (peripheral vision becoming blurred while central vision remains sharp) and the enlargement of nearby visual space, particularly at the visual locations surrounding the fovea. Our findings are also consistent with a mechanistic framework whereby gain control of ongoing and evoked activity in the visual cortex is controlled by activation of 5-HT2A receptors.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Visual tracking brain-computer interface

Brain-computer interfaces (BCIs) offer a way to interact with computers without relying on physical movements. Non-invasive electroencephalography-based visual BCIs, known for efficient speed and calibration ease, face limitations in continuous tasks due to discrete stimulus design and decoding methods. To achieve continuous control, we implemented a novel spatial encoding stimulus paradigm and devised a corresponding projection method to enable continuous modulation of decoded velocity. Subsequently, we conducted experiments involving 17 participants and achieved Fitt's information transfer rate (ITR) of 0.55 bps for the fixed tracking task and 0.37 bps for the random tracking task. The proposed BCI with a high Fitt's ITR was then integrated into two applications, including painting and gaming. In conclusion, this study proposed a visual BCI based-control method to go beyond discrete commands, allowing natural continuous control based on neural activity.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Hierarchical Context-Based Emotion Recognition With Scene Graphs

For a better intention inference, we often try to figure out the emotional states of other people in social communications. Many studies on affective computing have been carried out to infer emotions through perceiving human states, i.e., facial expression and body posture. Such methods are skillful in a controlled environment. However, it often leads to misestimation due to the deficiency of effective inputs in unconstrained circumstances, that is, where context-aware emotion recognition appeared. We take inspiration from the advanced reasoning pattern of humans in perceived emotion recognition and propose the hierarchical context-based emotion recognition method with scene graphs. We propose to extract three contexts from the image, i.e., the entity context, the global context, and the scene context. The scene context contains abstract information about entity labels and their relationships. It is similar to the information processing of the human visual sensing mechanism. After that, these contexts are further fused to perform emotion recognition. We carried out a bunch of experiments on the widely used context-aware emotion datasets, i.e., CAER-S, EMOTIC, and BOdy Language Dataset (BoLD). We demonstrate that the hierarchical contexts can benefit emotion recognition by improving the accuracy of the SOTA score from 84.82% to 90.83% on CAER-S. The ablation experiments show that hierarchical contexts provide complementary information. Our method improves the F1 score of the SOTA result from 29.33% to 30.24% (C-F1) on EMOTIC. We also build the image-based emotion recognition task with BoLD-Img from BoLD and obtain a better emotion recognition score (ERS) score of 0.2153.

Local field potentials, spiking activity, and receptive fields in human visual cortex

The concept of receptive field (RF) is central to sensory neuroscience. Neuronal RF properties have been substantially studied in animals, while those in humans remain nearly unexplored. Here, we measured neuronal RFs with intracranial local field potentials (LFPs) and spiking activity in human visual cortex (V1/V2/V3). We recorded LFPs via macro-contacts and discovered that RF sizes estimated from low-frequency activity (LFA, 0.5-30 Hz) were larger than those estimated from low-gamma activity (LGA, 30-60 Hz) and high-gamma activity (HGA, 60-150 Hz). We then took a rare opportunity to record LFPs and spiking activity via microwires in V1 simultaneously. We found that RF sizes and temporal profiles measured from LGA and HGA closely matched those from spiking activity. In sum, this study reveals that spiking activity of neurons in human visual cortex could be well approximated by LGA and HGA in RF estimation and temporal profile measurement, implying the pivotal functions of LGA and HGA in early visual information processing.

A CODE model bridging crowding in sparse and dense displays

Visual crowding is arguably the strongest limitation imposed on extrafoveal vision, and is a relatively well-understood phenomenon. However, most investigations and theories are based on sparse displays consisting of a target and at most a handful of flanker objects. Recent findings suggest that the laws thought to govern crowding may not hold for densely cluttered displays, and that grouping and nearest neighbour effects may be more important. Here we present a computational model that accounts for crowding effects in both sparse and dense displays. The model is an adaptation and extension of an earlier model that has previously successfully accounted for spatial clustering, numerosity and object-based attention phenomena. Our model combines grouping by proximity and similarity with a nearest neighbour rule, and defines crowding as the extent to which target and flankers fail to segment. We show that when the model is optimized for explaining crowding phenomena in classic, sparse displays, it also does a good job in capturing novel crowding patterns in dense displays, in both existing and new data sets. The model thus ties together different principles governing crowding, specifically Bouma's law, grouping, and nearest neighbour similarity effects.

Characterizing Spatiotemporal Population Receptive Fields in Human Visual Cortex with fMRI

The use of fMRI and computational modeling has advanced understanding of spatial characteristics of population receptive fields (pRFs) in human visual cortex. However, we know relatively little about the spatiotemporal characteristics of pRFs because neurons' temporal properties are one to two orders of magnitude faster than fMRI BOLD responses. Here, we developed an image-computable framework to estimate spatiotemporal pRFs from fMRI data. First, we developed a simulation software that predicts fMRI responses to a time-varying visual input given a spatiotemporal pRF model and solves the model parameters. The simulator revealed that ground-truth spatiotemporal parameters can be accurately recovered at the millisecond resolution from synthesized fMRI responses. Then, using fMRI and a novel stimulus paradigm, we mapped spatiotemporal pRFs in individual voxels across human visual cortex in 10 participants (both females and males). We find that a compressive spatiotemporal (CST) pRF model better explains fMRI responses than a conventional spatial pRF model across visual areas spanning the dorsal, lateral, and ventral streams. Further, we find three organizational principles of spatiotemporal pRFs: (1) from early to later areas within a visual stream, spatial and temporal windows of pRFs progressively increase in size and show greater compressive nonlinearities, (2) later visual areas show diverging spatial and temporal windows across streams, and (3) within early visual areas (V1-V3), both spatial and temporal windows systematically increase with eccentricity. Together, this computational framework and empirical results open exciting new possibilities for modeling and measuring fine-grained spatiotemporal dynamics of neural responses using fMRI.

Computational model links normalization to chemoarchitecture in the human visual system

A goal of cognitive neuroscience is to provide computational accounts of brain function. Canonical computations-mathematical operations used by the brain in many contexts-fulfill broad information-processing needs by varying their algorithmic parameters. A key question concerns the identification of biological substrates for these computations and their algorithms. Chemoarchitecture-the spatial distribution of neurotransmitter receptor densities-shapes brain function. Here, we propose that local variations in specific receptor densities implement algorithmic modulations of canonical computations. To test this hypothesis, we combine mathematical modeling of brain responses with chemoarchitecture data. We compare parameters of divisive normalization obtained from 7-tesla functional magnetic resonance imaging with receptor density maps obtained from positron emission tomography. We find evidence that serotonin and γ-aminobutyric acid receptor densities are the biological substrate for algorithmic modulations of divisive normalization in the human visual system. Our model links computational and biological levels of vision, explaining how canonical computations allow the brain to fulfill broad information-processing needs.

Hippocampal mechanisms resolve competition in memory and perception

Behaving adaptively requires selection of relevant memories and sensations and suppression of competing ones. We hypothesized that these mechanisms are linked, such that hippocampal computations that resolve competition in memory also shape the precision of sensory representations to guide selective attention. We leveraged f MRI-based pattern similarity, receptive field modeling, and eye tracking to test this hypothesis in humans performing a memory-dependent visual search task. In the hippocampus, differentiation of competing memories predicted the precision of memory-guided eye movements. In visual cortex, preparatory coding of remembered target locations predicted search successes, whereas preparatory coding of competing locations predicted search failures due to interference. These effects were linked: stronger hippocampal memory differentiation was associated with lower competitor activation in visual cortex, yielding more precise preparatory representations. These results demonstrate a role for memory differentiation in shaping the precision of sensory representations, highlighting links between mechanisms that overcome competition in memory and perception.

Neural population dynamics of human working memory

The activity of neurons in macaque prefrontal cortex (PFC) persists during working memory (WM) delays, providing a mechanism for memory.1,2,3,4,5,6,7,8,9,10,11 Although theory,11,12 including formal network models,13,14 assumes that WM codes are stable over time, PFC neurons exhibit dynamics inconsistent with these assumptions.15,16,17,18,19 Recently, multivariate reanalyses revealed the coexistence of both stable and dynamic WM codes in macaque PFC.20,21,22,23 Human EEG studies also suggest that WM might contain dynamics.24,25 Nonetheless, how WM dynamics vary across the cortical hierarchy and which factors drive dynamics remain unknown. To elucidate WM dynamics in humans, we decoded WM content from fMRI responses across multiple cortical visual field maps.26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48 We found coexisting stable and dynamic neural representations of WM during a memory-guided saccade task. Geometric analyses of neural subspaces revealed that early visual cortex exhibited stronger dynamics than high-level visual and frontoparietal cortex. Leveraging models of population receptive fields, we visualized and made the neural dynamics interpretable. We found that during WM delays, V1 population initially encoded a narrowly tuned bump of activation centered on the peripheral memory target. Remarkably, this bump then spread inward toward foveal locations, forming a vector along the trajectory of the forthcoming memory-guided saccade. In other words, the neural code transformed into an abstraction of the stimulus more proximal to memory-guided behavior. Therefore, theories of WM must consider both sensory features and their task-relevant abstractions because changes in the format of memoranda naturally drive neural dynamics.

Statistical inference on representational geometries

Neuroscience has recently made much progress, expanding the complexity of both neural activity measurements and brain-computational models. However, we lack robust methods for connecting theory and experiment by evaluating our new big models with our new big data. Here, we introduce new inference methods enabling researchers to evaluate and compare models based on the accuracy of their predictions of representational geometries: A good model should accurately predict the distances among the neural population representations (e.g. of a set of stimuli). Our inference methods combine novel 2-factor extensions of crossvalidation (to prevent overfitting to either subjects or conditions from inflating our estimates of model accuracy) and bootstrapping (to enable inferential model comparison with simultaneous generalization to both new subjects and new conditions). We validate the inference methods on data where the ground-truth model is known, by simulating data with deep neural networks and by resampling of calcium-imaging and functional MRI data. Results demonstrate that the methods are valid and conclusions generalize correctly. These data analysis methods are available in an open-source Python toolbox (rsatoolbox.readthedocs.io).

The Spatial Precision of Contextual Feedback Signals in Human V1

Neurons in the primary visual cortex (V1) receive sensory inputs that describe small, local regions of the visual scene and cortical feedback inputs from higher visual areas processing the global scene context. Investigating the spatial precision of this visual contextual modulation will contribute to our understanding of the functional role of cortical feedback inputs in perceptual computations. We used human functional magnetic resonance imaging (fMRI) to test the spatial precision of contextual feedback inputs to V1 during natural scene processing. We measured brain activity patterns in the stimulated regions of V1 and in regions that we blocked from direct feedforward input, receiving information only from non-feedforward (i.e., feedback and lateral) inputs. We measured the spatial precision of contextual feedback signals by generalising brain activity patterns across parametrically spatially displaced versions of identical images using an MVPA cross-classification approach. We found that fMRI activity patterns in cortical feedback signals predicted our scene-specific features in V1 with a precision of approximately 4 degrees. The stimulated regions of V1 carried more precise scene information than non-stimulated regions; however, these regions also contained information patterns that generalised up to 4 degrees. This result shows that contextual signals relating to the global scene are similarly fed back to V1 when feedforward inputs are either present or absent. Our results are in line with contextual feedback signals from extrastriate areas to V1, describing global scene information and contributing to perceptual computations such as the hierarchical representation of feature boundaries within natural scenes.

Characterizing spatiotemporal population receptive fields in human visual cortex with fMRI

The use of fMRI and computational modeling has advanced understanding of spatial characteristics of population receptive fields (pRFs) in human visual cortex. However, we know relatively little about the spatiotemporal characteristics of pRFs because neurons' temporal properties are one to two orders of magnitude faster than fMRI BOLD responses. Here, we developed an image-computable framework to estimate spatiotemporal pRFs from fMRI data. First, we developed a simulation software that predicts fMRI responses to a time varying visual input given a spatiotemporal pRF model and solves the model parameters. The simulator revealed that ground-truth spatiotemporal parameters can be accurately recovered at the millisecond resolution from synthesized fMRI responses. Then, using fMRI and a novel stimulus paradigm, we mapped spatiotemporal pRFs in individual voxels across human visual cortex in 10 participants. We find that a compressive spatiotemporal (CST) pRF model better explains fMRI responses than a conventional spatial pRF model across visual areas spanning the dorsal, lateral, and ventral streams. Further, we find three organizational principles of spatiotemporal pRFs: (i) from early to later areas within a visual stream, spatial and temporal integration windows of pRFs progressively increase in size and show greater compressive nonlinearities, (ii) later visual areas show diverging spatial and temporal integration windows across streams, and (iii) within early visual areas (V1-V3), both spatial and temporal integration windows systematically increase with eccentricity. Together, this computational framework and empirical results open exciting new possibilities for modeling and measuring fine-grained spatiotemporal dynamics of neural responses in the human brain using fMRI.

Independent spatiotemporal effects of spatial attention and background clutter on human object location representations

Spatial attention helps us to efficiently localize objects in cluttered environments. However, the processing stage at which spatial attention modulates object location representations remains unclear. Here we investigated this question identifying processing stages in time and space in an EEG and fMRI experiment respectively. As both object location representations and attentional effects have been shown to depend on the background on which objects appear, we included object background as an experimental factor. During the experiments, human participants viewed images of objects appearing in different locations on blank or cluttered backgrounds while either performing a task on fixation or on the periphery to direct their covert spatial attention away or towards the objects. We used multivariate classification to assess object location information. Consistent across the EEG and fMRI experiment, we show that spatial attention modulated location representations during late processing stages (>150 ms, in middle and high ventral visual stream areas) independent of background condition. Our results clarify the processing stage at which attention modulates object location representations in the ventral visual stream and show that attentional modulation is a cognitive process separate from recurrent processes related to the processing of objects on cluttered backgrounds.

Comparing retinotopic maps of children and adults reveals a late-stage change in how V1 samples the visual field

Adult visual performance differs with angular location -it is better for stimuli along the horizontal than vertical, and lower than upper vertical meridian of the visual field. These perceptual asymmetries are paralleled by asymmetries in cortical surface area in primary visual cortex (V1). Children, unlike adults, have similar visual performance at the lower and upper vertical meridian. Do children have similar V1 surface area representing the upper and lower vertical meridian? Using MRI, we measure the surface area of retinotopic maps (V1-V3) in children and adults. Many features of the maps are similar between groups, including greater V1 surface area for the horizontal than vertical meridian. However, unlike adults, children have a similar amount of V1 surface area representing the lower and upper vertical meridian. These data reveal a late-stage change in V1 organization that may relate to the emergence of the visual performance asymmetry along the vertical meridian by adulthood.

Comparison of Stimulus Types for Retinotopic Cortical Mapping of Macular Disease

Purpose

Retinotopic maps acquired using functional magnetic resonance imaging (fMRI) provide a valuable adjunct in the assessment of macular function at the level of the visual cortex. The present study quantitatively assessed the performance of different visual stimulation approaches for mapping visual field coverage.

Methods

Twelve patients with geographic atrophy (GA) secondary to age-related macular degeneration (AMD) were examined using high-resolution ultra-high field fMRI (Siemens Magnetom 7T) and microperimetry (MP; Nidek MP-3). The population receptive field (pRF)-based coverage maps obtained with two different stimulus techniques (moving bars, and rotating wedges and expanding rings) were compared with the results of MP. Correspondence between MP and pRF mapping was quantified by calculating the simple matching coefficient (SMC).

Results

Stimulus choice is shown to bias the spatial distribution of pRF centers and eccentricity values with pRF sizes obtained from wedge/ring or bar stimulation showing systematic differences. Wedge/ring stimulation results show a higher number of pRF centers in foveal areas and strongly reduced pRF sizes compared to bar stimulation runs. A statistical comparison shows significantly higher pRF center numbers in the foveal 2.5 degrees region of the visual field for wedge/ring compared to bar stimuli. However, these differences do not significantly influence SMC values when compared to MP (bar <2.5 degrees: 0.88 ± 0.13; bar >2.5 degrees: 0.88 ± 0.11; wedge/ring <2.5 degrees: 0.89 ± 0.12 wedge/ring; >2.5 degrees: 0.86 ± 0.10) for the peripheral visual field.

Conclusions

Both visual stimulation designs examined can be applied successfully in patients with GA. Although the two designs show systematic differences in the distribution of pRF center locations, this variability has minimal impact on the SMC when compared to the MP outcome.

Multitask representations in the human cortex transform along a sensory-to-motor hierarchy

Human cognition recruits distributed neural processes, yet the organizing computational and functional architectures remain unclear. Here, we characterized the geometry and topography of multitask representations across the human cortex using functional magnetic resonance imaging during 26 cognitive tasks in the same individuals. We measured the representational similarity across tasks within a region and the alignment of representations between regions. Representational alignment varied in a graded manner along the sensory-association-motor axis. Multitask dimensionality exhibited compression then expansion along this gradient. To investigate computational principles of multitask representations, we trained multilayer neural network models to transform empirical visual-to-motor representations. Compression-then-expansion organization in models emerged exclusively in a rich training regime, which is associated with learning optimized representations that are robust to noise. This regime produces hierarchically structured representations similar to empirical cortical patterns. Together, these results reveal computational principles that organize multitask representations across the human cortex to support multitask cognition.

Macromolecular tissue volume mapping of lateral geniculate nucleus subdivisions in living human brains

The lateral geniculate nucleus (LGN) is a key thalamic nucleus in the visual system, which has an important function in relaying retinal visual input to the visual cortex. The human LGN is composed mainly of magnocellular (M) and parvocellular (P) subdivisions, each of which has different stimulus selectivity in neural response properties. Previous studies have discussed the potential relationship between LGN subdivisions and visual disorders based on psychophysical data on specific types of visual stimuli. However, these relationships remain speculative because non-invasive measurements of these subdivisions are difficult due to the small size of the LGN. Here we propose a method to identify these subdivisions by combining two structural MR measures: high-resolution proton-density weighted images and macromolecular tissue volume (MTV) maps. We defined the M and P subdivisions based on MTV fraction data and tested the validity of the definition by (1) comparing the data with that from human histological studies, (2) comparing the data with functional magnetic resonance imaging measurements on stimulus selectivity, and (3) analyzing the test-retest reliability. The findings demonstrated that the spatial organization of the M and P subdivisions was consistent across subjects and in line with LGN subdivisions observed in human histological data. Moreover, the difference in stimulus selectivity between the subdivisions identified using MTV was consistent with previous physiology literature. The definition of the subdivisions based on MTV was shown to be robust over measurements taken on different days. These results suggest that MTV mapping is a promising approach for evaluating the tissue properties of LGN subdivisions in living humans. This method potentially will enable neuroscientific and clinical hypotheses about the human LGN subdivisions to be tested.

Evaluating the efficacy of multi-echo ICA denoising on model-based fMRI

fMRI is an indispensable tool for neuroscience investigation, but this technique is limited by multiple sources of physiological and measurement noise. These noise sources are particularly problematic for analysis techniques that require high signal-to-noise ratio for stable model fitting, such as voxel-wise modeling. Multi-echo data acquisition in combination with echo-time dependent ICA denoising (ME-ICA) represents one promising strategy to mitigate physiological and hardware-related noise sources as well as motion-related artifacts. However, most studies employing ME-ICA to date are resting-state fMRI studies, and therefore we have a limited understanding of the impact of ME-ICA on complex task or model-based fMRI paradigms. Here, we addressed this knowledge gap by comparing data quality and model fitting performance of data acquired during a visual population receptive field (pRF) mapping (N = 13 participants) experiment after applying one of three preprocessing procedures: ME-ICA, optimally combined multi-echo data without ICA-denoising, and typical single echo processing. As expected, multi-echo fMRI improved temporal signal-to-noise compared to single echo fMRI, with ME-ICA amplifying the improvement compared to optimal combination alone. However, unexpectedly, this boost in temporal signal-to-noise did not directly translate to improved model fitting performance: compared to single echo acquisition, model fitting was only improved after ICA-denoising. Specifically, compared to single echo acquisition, ME-ICA resulted in improved variance explained by our pRF model throughout the visual system, including anterior regions of the temporal and parietal lobes where SNR is typically low, while optimal combination without ICA did not. ME-ICA also improved reliability of parameter estimates compared to single echo and optimally combined multi-echo data without ICA-denoising. Collectively, these results suggest that ME-ICA is effective for denoising task-based fMRI data for modeling analyzes and maintains the integrity of the original data. Therefore, ME-ICA may be beneficial for complex fMRI experiments, including voxel-wise modeling and naturalistic paradigms.

Bayesian connective field modeling using a Markov Chain Monte Carlo approach

The majority of neurons in the human brain process signals from neurons elsewhere in the brain. Connective Field (CF) modelling is a biologically-grounded method to describe this essential aspect of the brain's circuitry. It allows characterizing the response of a population of neurons in terms of the activity in another part of the brain. CF modelling translates the concept of the receptive field (RF) into the domain of connectivity by assessing, at the voxel level, the spatial dependency between signals in distinct cortical visual field areas. Thus, the approach enables to characterize the functional cortical circuitry of the human cortex. While already very useful, the present CF modelling approach has some intrinsic limitations due to the fact that it only estimates the model's explained variance and not the probability distribution associated with the estimated parameters. If we could resolve this, CF modelling would lend itself much better for statistical comparisons at the level of single voxels and individuals. This is important when trying to gain a detailed understanding of the neurobiology and pathophysiology of the visual cortex, notably in rare cases. To enable this, we present a Bayesian approach to CF modeling (bCF). Using a Markov Chain Monte Carlo (MCMC) procedure, it estimates the posterior probability distribution underlying the CF parameters. Based on this, bCF quantifies, at the voxel level, the uncertainty associated with each parameter estimate. This information can be used in various ways to increase confidence in the CF model predictions. We applied bCF to BOLD responses recorded in the early human visual cortex using 3T fMRI. We estimated both the CF parameters and their associated uncertainties and show they are only weakly correlated. Moreover, we show how bCF facilitates the use of effect size (beta) as a data-driven parameter that can be used to select the most reliable voxels for further analysis. Finally, to further illustrate the functionality gained by bCF, we apply it to perform a voxel-level comparison of a single, circular symmetric, Gaussian versus a Difference-of-Gaussian model. We conclude that our bCF framework provides a comprehensive tool to study human functional cortical circuitry in health and disease.

Perception and memory have distinct spatial tuning properties in human visual cortex

Reactivation of earlier perceptual activity is thought to underlie long-term memory recall. Despite evidence for this view, it is unclear whether mnemonic activity exhibits the same tuning properties as feedforward perceptual activity. Here, we leverage population receptive field models to parameterize fMRI activity in human visual cortex during spatial memory retrieval. Though retinotopic organization is present during both perception and memory, large systematic differences in tuning are also evident. Whereas there is a three-fold decline in spatial precision from early to late visual areas during perception, this pattern is not observed during memory retrieval. This difference cannot be explained by reduced signal-to-noise or poor performance on memory trials. Instead, by simulating top-down activity in a network model of cortex, we demonstrate that this property is well explained by the hierarchical structure of the visual system. Together, modeling and empirical results suggest that computational constraints imposed by visual system architecture limit the fidelity of memory reactivation in sensory cortex.

Trial-by-trial predictions of subjective time from human brain activity

Human experience of time exhibits systematic, context-dependent deviations from clock time; for example, time is experienced differently at work than on holiday. Here we test the proposal that differences from clock time in subjective experience of time arise because time estimates are constructed by accumulating the same quantity that guides perception: salient events. Healthy human participants watched naturalistic, silent videos of up to 24 seconds in duration and estimated their duration while fMRI was acquired. We were able to reconstruct trial-by-trial biases in participants’ duration reports, which reflect subjective experience of duration, purely from salient events in their visual cortex BOLD activity. By contrast, salient events in neither of two control regions–auditory and somatosensory cortex–were predictive of duration biases. These results held despite being able to (trivially) predict clock time from all three brain areas. Our results reveal that the information arising during perceptual processing of a dynamic environment provides a sufficient basis for reconstructing human subjective time duration.

Our perception of time isn’t like a clock; it varies depending on other aspects of experience, such as what we see and hear in that moment. Previous studies have shown that differences in simple features, such as an image being larger or smaller, or brighter or dimmer, can change how we perceive time for those experiences. But in everyday life, the properties of these simple features can change frequently, presenting a challenge to understanding real-world time perception based on simple lab experiments. To overcome this problem, we developed a computational model of human time perception based on tracking changes in neural activity across brain regions involved in sensory processing (using non-invasive brain imaging). By measuring changes in brain activity patterns across these regions, our approach accommodates the different and changing feature combinations present in natural scenarios, such as walking on a busy street. Our model reproduces people’s duration reports for natural videos (up to almost half a minute long) and, most importantly, predicts whether a person reports a scene as relatively shorter or longer–the biases in time perception that reflect how natural experience of time deviates from clock time.

Highly accurate retinotopic maps of the physiological blind spot in human visual cortex

The physiological blind spot is a naturally occurring scotoma corresponding with the optic disc in the retina of each eye. Even during monocular viewing, observers are usually oblivious to the scotoma, in part because the visual system extrapolates information from the surrounding area. Unfortunately, studying this visual field region with neuroimaging has proven difficult, as it occupies only a small part of retinotopic cortex. Here, we used functional magnetic resonance imaging and a novel data-driven method for mapping the retinotopic organization in and around the blind spot representation in V1. Our approach allowed for highly accurate reconstructions of the extent of an observer's blind spot, and out-performed conventional model-based analyses. This method opens exciting opportunities to study the plasticity of receptive fields after visual field loss, and our data add to evidence suggesting that the neural circuitry responsible for impressions of perceptual completion across the physiological blind spot most likely involves regions of extrastriate cortex-beyond V1.

Self-supervised Natural Image Reconstruction and Large-scale Semantic Classification from Brain Activity

Reconstructing natural images and decoding their semantic category from fMRI brain recordings is challenging. Acquiring sufficient pairs of images and their corresponding fMRI responses, which span the huge space of natural images, is prohibitive. We present a novel self-supervised approach that goes well beyond the scarce paired data, for achieving both: (i) state-of-the art fMRI-to-image reconstruction, and (ii) first-ever large-scale semantic classification from fMRI responses. By imposing cycle consistency between a pair of deep neural networks (from image-to-fMRI & from fMRI-to-image), we train our image reconstruction network on a large number of "unpaired" natural images (images without fMRI recordings) from many novel semantic categories. This enables to adapt our reconstruction network to a very rich semantic coverage without requiring any explicit semantic supervision. Specifically, we find that combining our self-supervised training with high-level perceptual losses, gives rise to new reconstruction & classification capabilities. In particular, this perceptual training enables to classify well fMRIs of never-before-seen semantic classes, without requiring any class labels during training. This gives rise to: (i) Unprecedented image-reconstruction from fMRI of never-before-seen images (evaluated by image metrics and human testing), and (ii) Large-scale semantic classification of categories that were never-before-seen during network training. Such large-scale (1000-way) semantic classification from fMRI recordings has never been demonstrated before. Finally, we provide evidence for the biological consistency of our learned model.

Gradient-Guided and Multi-Scale Feature Network for Image Super-Resolution

Recently, deep-learning-based image super-resolution methods have made remarkable progress. However, most of these methods do not fully exploit the structural feature of the input image, as well as the intermediate features from the intermediate layers, which hinders the ability of detail recovery. To deal with this issue, we propose a gradient-guided and multi-scale feature network for image super-resolution (GFSR). Specifically, a dual-branch structure network is proposed, including the trunk branch and the gradient one, where the latter is used to extract the gradient feature map as structural prior to guide the image reconstruction process. Then, to absorb features from different layers, two effective multi-scale feature extraction modules, namely residual of residual inception block (RRIB) and residual of residual receptive field block (RRRFB), are proposed and embedded in different network layers. In our RRIB and RRRFB structures, an adaptive weighted residual feature fusion block (RFFB) is investigated to fuse the intermediate features to generate more beneficial representations, and an adaptive channel attention block (ACAB) is introduced to effectively explore the dependencies between channel features to further boost the feature representation capacity. Experimental results on several benchmark datasets demonstrate that our method achieves superior performance against state-of-the-art methods in terms of both subjective visual quality and objective quantitative metrics.

Mapping spatial frequency preferences across human primary visual cortex

Neurons in primate visual cortex (area V1) are tuned for spatial frequency, in a manner that depends on their position in the visual field. Several studies have examined this dependency using functional magnetic resonance imaging (fMRI), reporting preferred spatial frequencies (tuning curve peaks) of V1 voxels as a function of eccentricity, but their results differ by as much as two octaves, presumably owing to differences in stimuli, measurements, and analysis methodology. Here, we characterize spatial frequency tuning at a millimeter resolution within the human primary visual cortex, across stimulus orientation and visual field locations. We measured fMRI responses to a novel set of stimuli, constructed as sinusoidal gratings in log-polar coordinates, which include circular, radial, and spiral geometries. For each individual stimulus, the local spatial frequency varies inversely with eccentricity, and for any given location in the visual field, the full set of stimuli span a broad range of spatial frequencies and orientations. Over the measured range of eccentricities, the preferred spatial frequency is well-fit by a function that varies as the inverse of the eccentricity plus a small constant. We also find small but systematic effects of local stimulus orientation, defined in both absolute coordinates and relative to visual field location. Specifically, peak spatial frequency is higher for pinwheel than annular stimuli and for horizontal than vertical stimuli.

The spatiotemporal neural dynamics of object location representations in the human brain

To interact with objects in complex environments, we must know what they are and where they are in spite of challenging viewing conditions. Here, we investigated where, how and when representations of object location and category emerge in the human brain when objects appear on cluttered natural scene images using a combination of functional magnetic resonance imaging, electroencephalography and computational models. We found location representations to emerge along the ventral visual stream towards lateral occipital complex, mirrored by gradual emergence in deep neural networks. Time-resolved analysis suggested that computing object location representations involves recurrent processing in high-level visual cortex. Object category representations also emerged gradually along the ventral visual stream, with evidence for recurrent computations. These results resolve the spatiotemporal dynamics of the ventral visual stream that give rise to representations of where and what objects are present in a scene under challenging viewing conditions.

Comparing BOLD and VASO-CBV population receptive field estimates in human visual cortex

Vascular Space Occupancy (VASO) is an alternative fMRI approach based on changes in Cerebral Blood Volume (CBV). VASO-CBV fMRI can provide higher spatial specificity than the blood oxygenation level-dependent (BOLD) method because the CBV response is thought to be limited to smaller vessels. To investigate how this technique compares to BOLD fMRI for cognitive neuroscience applications, we compared population receptive field (pRF) mapping estimates between BOLD and VASO-CBV. We hypothesized that VASO-CBV would elicit distinct pRF properties compared to BOLD. Specifically, since pRF size estimates also depend on vascular sources, we hypothesized that reduced vascular blurring might yield narrower pRFs for VASO-CBV measurements. We used a VASO sequence with a double readout 3D EPI sequence at 7T to simultaneously measure VASO-CBV and BOLD responses in the visual cortex while participants viewed conventional pRF mapping stimuli. Both VASO-CBV and BOLD images show similar eccentricity and polar angle maps across all participants. Compared to BOLD-based measurements, VASO-CBV yielded lower tSNR and variance explained. The pRF size changed with eccentricity similarly for VASO-CBV and BOLD, and the pRF size estimates were similar for VASO-CBV and BOLD, even when we equate variance explained between VASO-CBV and BOLD. This result suggests that the vascular component of the pRF size is not dominating in either VASO-CBV or BOLD.

Spatial Attention Enhances Crowded Stimulus Encoding Across Modeled Receptive Fields by Increasing Redundancy of Feature Representations

Any visual system, biological or artificial, must make a trade-off between the number of units used to represent the visual environment and the spatial resolution of the sampling array. Humans and some other animals are able to allocate attention to spatial locations to reconfigure the sampling array of receptive fields (RFs), thereby enhancing the spatial resolution of representations without changing the overall number of sampling units. Here, we examine how representations of visual features in a fully convolutional neural network interact and interfere with each other in an eccentricity-dependent RF pooling array and how these interactions are influenced by dynamic changes in spatial resolution across the array. We study these feature interactions within the framework of visual crowding, a well-characterized perceptual phenomenon in which target objects in the visual periphery that are easily identified in isolation are much more difficult to identify when flanked by similar nearby objects. By separately simulating effects of spatial attention on RF size and on the density of the pooling array, we demonstrate that the increase in RF density due to attention is more beneficial than changes in RF size for enhancing target classification for crowded stimuli. Furthermore, by varying target/flanker spacing, as well as the spatial extent of attention, we find that feature redundancy across RFs has more influence on target classification than the fidelity of the feature representations themselves. Based on these findings, we propose a candidate mechanism by which spatial attention relieves visual crowding through enhanced feature redundancy that is mostly due to increased RF density.

A population receptive field model of the magnetoencephalography response

Computational models which predict the neurophysiological response from experimental stimuli have played an important role in human neuroimaging. One type of computational model, the population receptive field (pRF), has been used to describe cortical responses at the millimeter scale using functional magnetic resonance imaging (fMRI) and electrocorticography (ECoG). However, pRF models are not widely used for non-invasive electromagnetic field measurements (EEG/MEG), because individual sensors pool responses originating from several centimeter of cortex, containing neural populations with widely varying spatial tuning. Here, we introduce a forward-modeling approach in which pRFs estimated from fMRI data are used to predict MEG sensor responses. Subjects viewed contrast-reversing bar stimuli sweeping across the visual field in separate fMRI and MEG sessions. Individual subject's pRFs were modeled on the cortical surface at the millimeter scale using the fMRI data. We then predicted cortical time series and projected these predictions to MEG sensors using a biophysical MEG forward model, accounting for the pooling across cortex. We compared the predicted MEG responses to observed visually evoked steady-state responses measured in the MEG session. We found that pRF parameters estimated by fMRI could explain a substantial fraction of the variance in steady-state MEG sensor responses (up to 60% in individual sensors). Control analyses in which we artificially perturbed either pRF size or pRF position reduced MEG prediction accuracy, indicating that MEG data are sensitive to pRF properties derived from fMRI. Our model provides a quantitative approach to link fMRI and MEG measurements, thereby enabling advances in our understanding of spatiotemporal dynamics in human visual field maps.

Cross-dataset reproducibility of human retinotopic maps

Population receptive field (pRF) models fit to fMRI data are used to non-invasively measure retinotopic maps in human visual cortex, and these maps are a fundamental component of visual neuroscience experiments. Here, we examined the reproducibility of retinotopic maps across two datasets: a newly acquired retinotopy dataset from New York University (NYU) (n = 44) and a public dataset from the Human Connectome Project (HCP) (n = 181). Our goal was to assess the degree to which pRF properties are similar across datasets, despite substantial differences in their experimental protocols. The two datasets simultaneously differ in their stimulus apertures, participant pool, fMRI protocol, MRI field strength, and preprocessing pipeline. We assessed the cross-dataset reproducibility of the two datasets in terms of the similarity of vertex-wise pRF estimates and in terms of large-scale polar angle asymmetries in cortical magnification. Within V1, V2, V3, and hV4, the group-median NYU and HCP vertex-wise polar angle estimates were nearly identical. Both eccentricity and pRF size estimates were also strongly correlated between the two datasets, but with a slope different from 1; the eccentricity and pRF size estimates were systematically greater in the NYU data. Next, to compare large-scale map properties, we quantified two polar angle asymmetries in V1 cortical magnification previously identified in the HCP data. The NYU dataset confirms earlier reports that more cortical surface area represents horizontal than vertical visual field meridian, and lower than upper vertical visual field meridian. Together, our findings show that the retinotopic properties of V1, V2, V3, and hV4 can be reliably measured across two datasets, despite numerous differences in their experimental design. fMRI-derived retinotopic maps are reproducible because they rely on an explicit computational model of the fMRI response. In the case of pRF mapping, the model is grounded in physiological evidence of how visual receptive fields are organized, allowing one to quantitatively characterize the BOLD signal in terms of stimulus properties (i.e., location and size). The new NYU Retinotopy Dataset will serve as a useful benchmark for testing hypotheses about the organization of visual areas and for comparison to the HCP 7T Retinotopy Dataset.

Divisive normalization unifies disparate response signatures throughout the human visual hierarchy

A canonical neural computation is a mathematical operation applied by the brain in a wide variety of contexts and capable of explaining and unifying seemingly unrelated neural and perceptual phenomena. Here, we use a combination of state-of-the-art experiments (ultra-high-field functional MRI) and mathematical methods (population receptive field [pRF] modeling) to uniquely demonstrate the role of divisive normalization (DN) as the canonical neural computation underlying visuospatial responses throughout the human visual hierarchy. The DN pRF model provides a tool to investigate and interpret the computational processes underlying neural responses in human and animal recordings, but also in clinical and cognitive dimensions.

Neural processing is hypothesized to apply the same mathematical operations in a variety of contexts, implementing so-called canonical neural computations. Divisive normalization (DN) is considered a prime candidate for a canonical computation. Here, we propose a population receptive field (pRF) model based on DN and evaluate it using ultra-high-field functional MRI (fMRI). The DN model parsimoniously captures seemingly disparate response signatures with a single computation, superseding existing pRF models in both performance and biological plausibility. We observe systematic variations in specific DN model parameters across the visual hierarchy and show how they relate to differences in response modulation and visuospatial information integration. The DN model delivers a unifying framework for visuospatial responses throughout the human visual hierarchy and provides insights into its underlying information-encoding computations. These findings extend the role of DN as a canonical computation to neuronal populations throughout the human visual hierarchy.

Population receptive fields in non-human primates from whole-brain fMRI and large-scale neurophysiology in visual cortex

Population receptive field (pRF) modeling is a popular fMRI method to map the retinotopic organization of the human brain. While fMRI-based pRF maps are qualitatively similar to invasively recorded single-cell receptive fields in animals, it remains unclear what neuronal signal they represent. We addressed this question in awake nonhuman primates comparing whole-brain fMRI and large-scale neurophysiological recordings in areas V1 and V4 of the visual cortex. We examined the fits of several pRF models based on the fMRI blood-oxygen-level-dependent (BOLD) signal, multi-unit spiking activity (MUA), and local field potential (LFP) power in different frequency bands. We found that pRFs derived from BOLD-fMRI were most similar to MUA-pRFs in V1 and V4, while pRFs based on LFP gamma power also gave a good approximation. fMRI-based pRFs thus reliably reflect neuronal receptive field properties in the primate brain. In addition to our results in V1 and V4, the whole-brain fMRI measurements revealed retinotopic tuning in many other cortical and subcortical areas with a consistent increase in pRF size with increasing eccentricity, as well as a retinotopically specific deactivation of default mode network nodes similar to previous observations in humans.

The role of neural tuning in quantity perception

Perception of quantities, such as numerosity, timing, and size, is essential for behavior and cognition. Accumulating evidence demonstrates neurons processing quantities are tuned, that is, have a preferred quantity amount, not only for numerosity, but also other quantity dimensions and sensory modalities. We argue that quantity-tuned neurons are fundamental to understanding quantity perception. We illustrate how the properties of quantity-tuned neurons can underlie a range of perceptual phenomena. Furthermore, quantity-tuned neurons are organized in distinct but overlapping topographic maps. We suggest that this overlap in tuning provides the neural basis for perceptual interactions between different quantities, without the need for a common neural representational code.

Comparing Object Recognition in Humans and Deep Convolutional Neural Networks-An Eye Tracking Study

Deep convolutional neural networks (DCNNs) and the ventral visual pathway share vast architectural and functional similarities in visual challenges such as object recognition. Recent insights have demonstrated that both hierarchical cascades can be compared in terms of both exerted behavior and underlying activation. However, these approaches ignore key differences in spatial priorities of information processing. In this proof-of-concept study, we demonstrate a comparison of human observers (N = 45) and three feedforward DCNNs through eye tracking and saliency maps. The results reveal fundamentally different resolutions in both visualization methods that need to be considered for an insightful comparison. Moreover, we provide evidence that a DCNN with biologically plausible receptive field sizes called vNet reveals higher agreement with human viewing behavior as contrasted with a standard ResNet architecture. We find that image-specific factors such as category, animacy, arousal, and valence have a direct link to the agreement of spatial object recognition priorities in humans and DCNNs, while other measures such as difficulty and general image properties do not. With this approach, we try to open up new perspectives at the intersection of biological and computer vision research.

Spared perilesional V1 activity underlies training-induced recovery of luminance detection sensitivity in cortically-blind patients

Damage to the primary visual cortex (V1) causes homonymous visual-field loss long considered intractable. Multiple studies now show that perceptual training can restore visual functions in chronic cortically-induced blindness (CB). A popular hypothesis is that training can harness residual visual functions by recruiting intact extrageniculostriate pathways. Training may also induce plastic changes within spared regions of the damaged V1. Here, we link changes in luminance detection sensitivity with retinotopic fMRI activity before and after visual discrimination training in eleven patients with chronic, stroke-induced CB. We show that spared V1 activity representing perimetrically-blind locations prior to training predicts the amount of training-induced recovery of luminance detection sensitivity. Additionally, training results in an enlargement of population receptive fields in perilesional V1, which increases blind-field coverage and may support further recovery with subsequent training. These findings uncover fundamental changes in perilesional V1 cortex underlying training-induced restoration of conscious luminance detection sensitivity in CB.

In humans, stroke damage to V1 causes large visual field defects. Spared V1 activity prior to training predicts the amount of training-induced recovery in luminance detection sensitivity. Moreover, visual training changes population receptive field properties within residual V1 circuits.

The relationship between transcription and eccentricity in human V1

Gene expression gradients radiating from regions of primary sensory cortices have recently been described and are thought to underlie the large-scale organization of the human cerebral cortex. However, the role of transcription in the functional layout of a single region within the adult brain has yet to be clarified, likely owing to the difficulty of identifying a brain region anatomically consistent enough across individuals with dense enough tissue sampling. Overcoming these hurdles in human primary visual cortex (V1), we show a relationship between differential gene expression and the cortical layout of eccentricity in human V1. Interestingly, these genes are unique from those previously identified that contribute to the positioning of cortical areas in the visual processing hierarchy. Enrichment analyses show that a subset of the identified genes encode for structures related to inhibitory interneurons, ion channels, as well as cellular projections, and are expressed more in foveal compared to peripheral portions of human V1. These findings predict that tissue density should be higher in foveal compared to peripheral V1. Using a histological pipeline, we validate this prediction using Nissl-stained sections of postmortem occipital cortex. We discuss these findings relative to previous studies in non-human primates, as well as in the context of an organizational pattern in which the adult human brain employs transcription gradients at multiple spatial scales: across the cerebral cortex, across areas within processing hierarchies, and within single cortical areas.

Mapping Visual Field Defects With fMRI - Impact of Approach and Experimental Conditions

Current initiatives to restore vision emphasize the need for objective assessments of visual field (VF) defects as pursued with functional magnetic resonance imaging (fMRI) approaches. Here, we compared population receptive field (pRF) mapping-based VF reconstructions to an fMRI method that uses more robust visual stimulation (on-off block design) in combination with individualized anatomy-driven retinotopic atlas-information (atlas-based VF). We investigated participants with sizable peripheral VF-deficits due to advanced glaucoma (n = 4) or retinitis pigmentosa (RP; n = 2) and controls (n = 6) with simulated scotoma. We obtained (1) standard automated perimetry (SAP) data as reference VFs and 3T fMRI data for (2) pRF-mapping [8-direction bar stimulus, fixation color change task] and (3) block-design full-field stimulation [8-direction drifting contrast patterns during (a) passive viewing (PV) and (b) one-back-task (OBT; reporting successions of identical motion directions) to probe the impact of previously reported task-related unspecific visual cortex activations]. Correspondence measures between the SAP and fMRI-based VFs were accuracy, assisted by sensitivity and specificity. We found an accuracy of pRF-based VF from V1 in patients [median: 0.62] that was similar to previous reports and increased by adding V2 and V3 to the analysis [0.74]. In comparison to the pRF-based VF, equivalent accuracies were obtained for the atlas-based VF for both PV [0.67] and, unexpectedly, the OBT [0.59], where, however, unspecific cortical activations were reflected by a reduction in sensitivity [0.71 (PV) and 0.35 (OBT)]. In conclusion, in patients with peripheral VF-defects, we demonstrate that previous fMRI procedures to obtain VF-estimates might be enhanced by: (1) pooling V1-V3 to enhance accuracy; (2) reporting sensitivity and specificity measures to increase transparency of the VF-reconstruction metric; (3) applying atlas-based procedures, if pRF-based VFs are not available or difficult to obtain; and (4) giving, counter-intuitively, preference to PV. These findings are expected to provide guidance to overcome current limitations of translating fMRI-based methods to a clinical work-up.

Transcranial magnetic stimulation entrains alpha oscillatory activity in occipital cortex

Parieto-occipital alpha rhythms (8-12 Hz) underlie cortical excitability and influence visual performance. Whether the synchrony of intrinsic alpha rhythms in the occipital cortex can be entrained by transcranial magnetic stimulation (TMS) is an open question. We applied 4-pulse, 10-Hz rhythmic TMS to entrain intrinsic alpha oscillators targeting right V1/V2, and tested four predictions with concurrent electroencephalogram (EEG): (1) progressive enhancement of entrainment across time windows, (2) output frequency specificity, (3) dependence on the intrinsic oscillation phase, and (4) input frequency specificity to individual alpha frequency (IAF) in the neural signatures. Two control conditions with an equal number of pulses and duration were arrhythmic-active and rhythmic-sham stimulation. The results confirmed the first three predictions. Rhythmic TMS bursts significantly entrained local neural activity. Near the stimulation site, evoked oscillation amplitude and inter-trial phase coherence (ITPC) were increased for 2 and 3 cycles, respectively, after the last TMS pulse. Critically, ITPC following entrainment positively correlated with IAF rather than with the degree of similarity between IAF and the input frequency (10 Hz). Thus, we entrained alpha-band activity in occipital cortex for ~ 3 cycles (~ 300 ms), and IAF predicts the strength of entrained occipital alpha phase synchrony indexed by ITPC.

Lightweight Salient Object Detection via Hierarchical Visual Perception Learning

Recently, salient object detection (SOD) has witnessed vast progress with the rapid development of convolutional neural networks (CNNs). However, the improvement of SOD accuracy comes with the increase in network depth and width, resulting in large network size and heavy computational overhead. This prevents state-of-the-art SOD methods from being deployed into practical platforms, especially mobile devices. To promote the deployment of real-world SOD applications, we aim at developing a lightweight SOD model in this article. Our observation comes from that the primate visual system processes visual signals hierarchically with different receptive fields and eccentricities in different visual cortex areas. Inspired by this, we propose a hierarchical visual perception (HVP) module to imitate the primate visual cortex for hierarchical perception learning. With the HVP module incorporated, we design a lightweight SOD network, namely, HVPNet. Extensive experiments on popular benchmarks demonstrate that HVPNet achieves highly competitive accuracy compared with state-of-the-art SOD methods while running at 4.3 frames/s CPU speed and 333.2 frames/s GPU speed with only 1.23M parameters.

An audio-visual motor training improves audio spatial localization skills in individuals with scotomas due to retinal degenerative diseases

Several studies have shown that impairments in a sensory modality can induce perceptual deficits in tasks involving the remaining senses. For example, people with retinal degenerative diseases like Macular Degeneration (MD) and with central scotoma show biased auditory localization abilities towards the visual field's scotoma area. This result indicates an auditory spatial reorganization of cross-modal processing in people with scotoma when the visual information is impaired. Recent works showed that multisensory training could be beneficial to improve spatial perception. In line with this idea, here we hypothesize that audio-visual and motor training could improve people's spatial skills with retinal degenerative diseases. In the present study, we tested this hypothesis by testing two groups of scotoma patients in an auditory and visual localization task before and after a training or rest performance. The training group was tested before and after multisensory training, while the control group performed the two tasks twice after 10 min of break. The training was done with a portable device positioned on the finger, providing spatially and temporally congruent audio and visual feedback during arm movement. Our findings show improved audio and visual localization for the training group and not for the control group. These results suggest that integrating multiple spatial sensory cues can improve the spatial perception of scotoma patients. This finding ignites further research and applications for people with central scotoma for whom rehabilitation is classically focused on training visual modality only.

Holistic face recognition is an emergent phenomenon of spatial processing in face-selective regions

Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial processing in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical holistic processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. Here we show that in face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage are smaller and shifted downward in response to face inversion. From these measurements, we successfully predict the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial processing in face-selective regions may enable holistic perception. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.

Human primary visual cortex shows larger population receptive fields for binocular disparity-defined stimuli

The visual perception of 3D depth is underpinned by the brain's ability to combine signals from the left and right eyes to produce a neural representation of binocular disparity for perception and behaviour. Electrophysiological studies of binocular disparity over the past 2 decades have investigated the computational role of neurons in area V1 for binocular combination, while more recent neuroimaging investigations have focused on identifying specific roles for different extrastriate visual areas in depth perception. Here we investigate the population receptive field properties of neural responses to binocular information in striate and extrastriate cortical visual areas using ultra-high field fMRI. We measured BOLD fMRI responses while participants viewed retinotopic mapping stimuli defined by different visual properties: contrast, luminance, motion, correlated and anti-correlated stereoscopic disparity. By fitting each condition with a population receptive field model, we compared quantitatively the size of the population receptive field for disparity-specific stimulation. We found larger population receptive fields for disparity compared with contrast and luminance in area V1, the first stage of binocular combination, which likely reflects the binocular integration zone, an interpretation supported by modelling of the binocular energy model. A similar pattern was found in region LOC, where it may reflect the role of disparity as a cue for 3D shape. These findings provide insight into the binocular receptive field properties underlying processing for human stereoscopic vision.