The evolution of a disparity decision in human visual cortex

Estimating neural activity from visual areas using functionally defined EEG templates

Electroencephalography (EEG) is a common and inexpensive method to record neural activity in humans. However, it lacks spatial resolution making it difficult to determine which areas of the brain are responsible for the observed EEG response. Here we present a new easy-to-use method that relies on EEG topographical templates. Using MRI and fMRI scans of 50 participants, we simulated how the activity in each visual area appears on the scalp and averaged this signal to produce functionally defined EEG templates. Once created, these templates can be used to estimate how much each visual area contributes to the observed EEG activity. We tested this method on extensive simulations and on real data. The proposed procedure is as good as bespoke individual source localization methods, robust to a wide range of factors, and has several strengths. First, because it does not rely on individual brain scans, it is inexpensive and can be used on any EEG data set, past or present. Second, the results are readily interpretable in terms of functional brain regions and can be compared across neuroimaging techniques. Finally, this method is easy to understand, simple to use and expandable to other brain sources.

[Abnormal spontaneous brain functional activity in adult patients with amblyopia: a resting-state functional magnetic resonance imaging study]

Amblyopia is a visual development deficit caused by abnormal visual experience in early life, mainly manifesting as defected visual acuity and binocular visual impairment, which is considered to reflect abnormal development of the brain rather than organic lesions of the eye. Previous studies have reported abnormal spontaneous brain activity in patients with amblyopia. However, the location of abnormal spontaneous activity in patients with amblyopia and the association between abnormal brain function activity and clinical deficits remain unclear. The purpose of this study is to analyze spontaneous brain functional activity abnormalities in patients with amblyopia and their associations with clinical defects using resting-state functional magnetic resonance imaging (fMRI) data. In this study, 31 patients with amblyopia and 31 healthy controls were enrolled for resting-state fMRI scanning. The results showed that spontaneous activity in the right angular gyrus, left posterior cerebellum, and left cingulate gyrus were significantly lower in patients with amblyopia than in controls, and spontaneous activity in the right middle temporal gyrus was significantly higher in patients with amblyopia. In addition, the spontaneous activity of the left cerebellum in patients with amblyopia was negatively associated with the best-corrected visual acuity of the amblyopic eye, and the spontaneous activity of the right middle temporal gyrus was positively associated with the stereoacuity. This study found that adult patients with amblyopia showed abnormal spontaneous activity in the angular gyrus, cerebellum, middle temporal gyrus, and cingulate gyrus. Furthermore, the functional abnormalities in the cerebellum and middle temporal gyrus may be associated with visual acuity defects and stereopsis deficiency in patients with amblyopia. These findings help explain the neural mechanism of amblyopia, thus promoting the improvement of the treatment strategy for amblyopia.

Abnormal effective connectivity in visual cortices underlies stereopsis defects in amblyopia

•

Abnormal effective connectivity inherent stereopsis defects in amblyopia was studied.

•

A weakened connection from V2v to LO2 relates to stereopsis defects in amblyopia.

•

Higher-order visual cortices may serve as key nodes to the stereopsis defects.

•

An independent longitudinal dataset was used to validate the obtained results.

The neural basis underlying stereopsis defects in patients with amblyopia remains unclear, which hinders the development of clinical therapy. This study aimed to investigate visual network abnormalities in patients with amblyopia and their associations with stereopsis function. Spectral dynamic causal modeling methods were employed for resting-state functional magnetic resonance imaging data to investigate the effective connectivity (EC) among 14 predefined regions of interest in the dorsal and ventral visual pathways. We adopted two independent datasets, including a cross-sectional and a longitudinal dataset. In the cross-sectional dataset, we compared group differences in EC between 31 patients with amblyopia (mean age: 26.39 years old) and 31 healthy controls (mean age: 25.71 years old) and investigated the association between EC and stereoacuity. In addition, we explored EC changes after perceptual learning in a novel longitudinal dataset including 9 patients with amblyopia (mean age: 15.78 years old). We found consistent evidence from the two datasets indicating that the aberrant EC from V2v to LO2 is crucial for the stereoscopic deficits in the patients with amblyopia: it was weaker in the patients than in the controls, showed a positive linear relationship with the stereoscopic function, and increased after perceptual learning in the patients. In addition, higher-level dorsal (V3d, V3A, and V3B) and ventral areas (LO1 and LO2) were important nodes in the network of abnormal ECs associated with stereoscopic deficits in the patients with amblyopia. Our research provides insights into the neural mechanism underlying stereopsis deficits in patients with amblyopia and provides candidate targets for focused stimulus interventions to enhance the efficacy of clinical treatment for the improvement of stereopsis deficiency.

Ultra-High-Field Neuroimaging Reveals Fine-Scale Processing for 3D Perception

Binocular disparity provides critical information about three-dimensional (3D) structures to support perception and action. In the past decade significant progress has been made in uncovering human brain areas engaged in the processing of binocular disparity signals. Yet, the fine-scale brain processing underlying 3D perception remains unknown. Here, we use ultra-high-field (7T) functional imaging at submillimeter resolution to examine fine-scale BOLD fMRI signals involved in 3D perception. In particular, we sought to interrogate the local circuitry involved in disparity processing by sampling fMRI responses at different positions relative to the cortical surface (i.e., across cortical depths corresponding to layers). We tested for representations related to 3D perception by presenting participants (male and female, N = 8) with stimuli that enable stable stereoscopic perception [i.e., correlated random dot stereograms (RDS)] versus those that do not (i.e., anticorrelated RDS). Using multivoxel pattern analysis (MVPA), we demonstrate cortical depth-specific representations in areas V3A and V7 as indicated by stronger pattern responses for correlated than for anticorrelated stimuli in upper rather than deeper layers. Examining informational connectivity, we find higher feedforward layer-to-layer connectivity for correlated than anticorrelated stimuli between V3A and V7. Further, we observe disparity-specific feedback from V3A to V1 and from V7 to V3A. Our findings provide evidence for the role of V3A as a key nexus for disparity processing, which is implicated in feedforward and feedback signals related to the perceptual estimation of 3D structures.SIGNIFICANCE STATEMENT Binocular vision plays a significant role in supporting our interactions with the surrounding environment. The fine-scale neural mechanisms that underlie the brain's skill in extracting 3D structures from binocular signals are poorly understood. Here, we capitalize on recent advances in ultra-high-field functional imaging to interrogate human brain circuits involved in 3D perception at submillimeter resolution. We provide evidence for the role of area V3A as a key nexus for disparity processing, which is implicated in feedforward and feedback signals related to the perceptual estimation of 3D structures from binocular signals. These fine-scale measurements help bridge the gap between animal neurophysiology and human fMRI studies investigating cross-scale circuits, from micro circuits to global brain networks for 3D perception.

Disparity in Context: Understanding how monocular image content interacts with disparity processing in human visual cortex

Horizontal disparities between the two eyes' retinal images are the primary cue for depth. Commonly used random ot tereograms (RDS) intentionally camouflage the disparity cue, breaking the correlations between monocular image structure and the depth map that are present in natural images. Because of the nonlinear nature of visual processing, it is unlikely that simple computational rules derived from RDS will be sufficient to explain binocular vision in natural environments. In order to understand the interplay between natural scene structure and disparity encoding, we used a depth-image-based-rendering technique and a library of natural 3D stereo pairs to synthesize two novel stereogram types in which monocular scene content was manipulated independent of scene depth information. The half-images of the novel stereograms comprised either random-dots or scrambled natural scenes, each with the same depth maps as the corresponding natural scene stereograms. Using these stereograms in a simultaneous Event-Related Potential and behavioral discrimination task, we identified multiple disparity-contingent encoding stages between 100 ~ 500 msec. The first disparity sensitive evoked potential was observed at ~100 msec after an earlier evoked potential (between ~50-100 msec) that was sensitive to the structure of the monocular half-images but blind to disparity. Starting at ~150 msec, disparity responses were stereogram-specific and predictive of perceptual depth. Complex features associated with natural scene content are thus at least partially coded prior to disparity information, but these features and possibly others associated with natural scene content interact with disparity information only after an intermediate, 2D scene-independent disparity processing stage.

Specialized contributions of mid-tier stages of dorsal and ventral pathways to stereoscopic processing in macaque

The division of labor between the dorsal and ventral visual pathways has been well studied, but not often with direct comparison at the single-neuron resolution with matched stimuli. Here we directly compared how single neurons in MT and V4, mid-tier areas of the two pathways, process binocular disparity, a powerful cue for 3D perception and actions. We found that MT neurons transmitted disparity signals more quickly and robustly, whereas V4 or its upstream neurons transformed the signals into sophisticated representations more prominently. Therefore, signaling speed and robustness were traded for transformation between the dorsal and ventral pathways. The key factor in this tradeoff was disparity-tuning shape: V4 neurons had more even-symmetric tuning than MT neurons. Moreover, the tuning symmetry predicted the degree of signal transformation across neurons similarly within each area, implying a general role of tuning symmetry in the stereoscopic processing by the two pathways.

Decoding Neural Responses to Motion-in-Depth Using EEG

Two stereoscopic cues that underlie the perception of motion-in-depth (MID) are changes in retinal disparity over time (CD) and interocular velocity differences (IOVD). These cues have independent spatiotemporal sensitivity profiles, depend upon different low-level stimulus properties, and are potentially processed along separate cortical pathways. Here, we ask whether these MID cues code for different motion directions: do they give rise to discriminable patterns of neural signals, and is there evidence for their convergence onto a single "motion-in-depth" pathway? To answer this, we use a decoding algorithm to test whether, and when, patterns of electroencephalogram (EEG) signals measured from across the full scalp, generated in response to CD- and IOVD-isolating stimuli moving toward or away in depth can be distinguished. We find that both MID cue type and 3D-motion direction can be decoded at different points in the EEG timecourse and that direction decoding cannot be accounted for by static disparity information. Remarkably, we find evidence for late processing convergence: IOVD motion direction can be decoded relatively late in the timecourse based on a decoder trained on CD stimuli, and vice versa. We conclude that early CD and IOVD direction decoding performance is dependent upon fundamentally different low-level stimulus features, but that later stages of decoding performance may be driven by a central, shared pathway that is agnostic to these features. Overall, these data are the first to show that neural responses to CD and IOVD cues that move toward and away in depth can be decoded from EEG signals, and that different aspects of MID-cues contribute to decoding performance at different points along the EEG timecourse.

Neural dynamics of cue reliability in perceptual decisions

To extract meaningful information from scenes, the visual system must combine local cues that can vary greatly in their degree of reliability. Here, we asked whether cue reliability mostly affects visual or decision-related processes, using visual evoked potentials (VEPs) and a model-based approach to identify when and where stimulus-evoked brain activity reflects cue reliability. Participants performed a shape discrimination task on Gaborized ellipses, while we parametrically and independently, varied the reliability of contour or surface cues. We modeled the expected behavioral performance as a linear function of cue reliability and established at what latencies and electrodes VEP activity reflected behavioral sensitivity to cue reliability. We found that VEPs were linearly related to the individual behavioral predictors at around 400 ms post-stimulus, at electrodes over parietal and lateral temporal cortex. The observed cue reliability effects were similar for variations in contour and surface cues. Notably, effects of cue reliability were absent at earlier latencies where visual shape information is typically reported, and also in data time-locked to the behavioral response, suggesting the effects are not decision-related. These results indicate that reliability of visual cues is reflected in late distributed perceptual processes.

Dynamics of perceptual decisions about symmetry in visual cortex

Neuroimaging studies have identified multiple extra-striate visual areas that are sensitive to symmetry in planar images (Kohler et al., 2016; Sasaki et al., 2005). Here, we investigated which of these areas are directly involved in perceptual decisions about symmetry, by recording high-density EEG in participants (n = 25) who made rapid judgments about whether an exemplar image contained rotation symmetry or not. Stimulus-locked sensor-level analysis revealed symmetry-specific activity that increased with increasing order of rotation symmetry. Response-locked analysis identified activity occurring between 600 and 200 ms before the button-press, that was directly related to perceptual decision making. We then used fMRI-informed EEG source imaging to characterize the dynamics of symmetry-specific activity within an extended network of areas in visual cortex. The most consistent cortical source of the stimulus-locked activity was VO1, a topographically organized area in ventral visual cortex, that was highly sensitive to symmetry in a previous study (Kohler et al., 2016). Importantly, VO1 activity also contained a strong decision-related component, suggesting that this area plays a crucial role in perceptual decisions about symmetry. Other candidate areas, such as lateral occipital cortex, had weak stimulus-locked symmetry responses and no evidence of correlation with response timing.

Sparse EEG/MEG source estimation via a group lasso

Non-invasive recordings of human brain activity through electroencephalography (EEG) or magnetoencelphalography (MEG) are of value for both basic science and clinical applications in sensory, cognitive, and affective neuroscience. Here we introduce a new approach to estimating the intra-cranial sources of EEG/MEG activity measured from extra-cranial sensors. The approach is based on the group lasso, a sparse-prior inverse that has been adapted to take advantage of functionally-defined regions of interest for the definition of physiologically meaningful groups within a functionally-based common space. Detailed simulations using realistic source-geometries and data from a human Visual Evoked Potential experiment demonstrate that the group-lasso method has improved performance over traditional ℓ2 minimum-norm methods. In addition, we show that pooling source estimates across subjects over functionally defined regions of interest results in improvements in the accuracy of source estimates for both the group-lasso and minimum-norm approaches.

Stereopsis after anterior temporal lobectomy

Brain areas critical for stereopsis have been investigated in non-human primates but are largely unknown in the human brain. Microelectrode recordings and functional MRI (fMRI) studies in monkeys have shown that in monkeys the inferior temporal cortex is critically involved in 3D shape categorization. Furthermore, some human fMRI studies similarly suggest an involvement of visual areas in the temporal lobe in depth perception. We aimed to investigate the role of the human anterior temporal neocortex in stereopsis by assessing stereoscopic depth perception before and after anterior temporal lobectomy. Eighteen epilepsy surgery patients were tested, pre- and postoperatively, in 3 different depth discrimination tasks. Sensitivity for local and global disparity was tested in a near-far discrimination task and sensitivity for 3D curvature was assessed in a convex-concave discrimination task, where 3D shapes were presented at different positions in depth. We found no evidence that temporal lobe epilepsy surgery has a significant effect on stereopsis. In contrast with earlier findings, we conclude that local as well as global stereopsis is maintained after unilateral resection of the temporal pole in epilepsy surgery patients. Our findings, together with previous studies, suggest that in humans more posterior visual regions underlie depth perception.

Degraded attentional modulation of cortical neural populations in strabismic amblyopia

Behavioral studies have reported reduced spatial attention in amblyopia, a developmental disorder of spatial vision. However, the neural populations in the visual cortex linked with these behavioral spatial attention deficits have not been identified. Here, we use functional MRI–informed electroencephalography source imaging to measure the effect of attention on neural population activity in the visual cortex of human adult strabismic amblyopes who were stereoblind. We show that compared with controls, the modulatory effects of selective visual attention on the input from the amblyopic eye are substantially reduced in the primary visual cortex (V1) as well as in extrastriate visual areas hV4 and hMT+. Degraded attentional modulation is also found in the normal-acuity fellow eye in areas hV4 and hMT+ but not in V1. These results provide electrophysiological evidence that abnormal binocular input during a developmental critical period may impact cortical connections between the visual cortex and higher level cortices beyond the known amblyopic losses in V1 and V2, suggesting that a deficit of attentional modulation in the visual cortex is an important component of the functional impairment in amblyopia. Furthermore, we find that degraded attentional modulation in V1 is correlated with the magnitude of interocular suppression and the depth of amblyopia. These results support the view that the visual suppression often seen in strabismic amblyopia might be a form of attentional neglect of the visual input to the amblyopic eye.

Representation of Maximally Regular Textures in Human Visual Cortex

Naturalistic textures with an intermediate degree of statistical regularity can capture key structural features of natural images (Freeman and Simoncelli, 2011). V2 and later visual areas are sensitive to these features, while primary visual cortex is not (Freeman et al., 2013). Here we expand on this work by investigating a class of textures that have maximal formal regularity, the 17 crystallographic wallpaper groups (Fedorov, 1891). We used texture stimuli from four of the groups that differ in the maximum order of rotation symmetry they contain, and measured neural responses in human participants using functional MRI and high-density EEG. We found that cortical area V3 has a parametric representation of the rotation symmetries in the textures that is not present in either V1 or V2, the first discovery of a stimulus property that differentiates processing in V3 from that of lower-level areas. Parametric responses were also seen in higher-order ventral stream areas V4, VO1, and lateral occipital complex (LOC), but not in dorsal stream areas. The parametric response pattern was replicated in the EEG data, and source localization indicated that responses in V3 and V4 lead responses in LOC, which is consistent with a feedforward mechanism. Finally, we presented our stimuli to four well developed feedforward models and found that none of them were able to account for our results. Our results highlight structural regularity as an important stimulus dimension for distinguishing the early stages of visual processing, and suggest a previously unrecognized role for V3 in the visual form-processing hierarchy. Significance statement: Hierarchical processing is a fundamental organizing principle in visual neuroscience, with each successive processing stage being sensitive to increasingly complex stimulus properties. Here, we probe the encoding hierarchy in human visual cortex using a class of visual textures--wallpaper patterns--that are maximally regular. Through a combination of fMRI and EEG source imaging, we find specific responses to texture regularity that depend parametrically on the maximum order of rotation symmetry in the textures. These parametric responses are seen in several areas of the ventral visual processing stream, as well as in area V3, but not in V1 or V2. This is the first demonstration of a stimulus property that differentiates processing in V3 from that of lower-level visual areas.

The steady-state visual evoked potential in vision research: A review

Periodic visual stimulation and analysis of the resulting steady-state visual evoked potentials were first introduced over 80 years ago as a means to study visual sensation and perception. From the first single-channel recording of responses to modulated light to the present use of sophisticated digital displays composed of complex visual stimuli and high-density recording arrays, steady-state methods have been applied in a broad range of scientific and applied settings.The purpose of this article is to describe the fundamental stimulation paradigms for steady-state visual evoked potentials and to illustrate these principles through research findings across a range of applications in vision science.

How to use fMRI functional localizers to improve EEG/MEG source estimation

EEG and MEG have excellent temporal resolution, but the estimation of the neural sources that generate the signals recorded by the sensors is a difficult, ill-posed problem. The high spatial resolution of functional MRI makes it an ideal tool to improve the localization of the EEG/MEG sources using data fusion. However, the combination of the two techniques remains challenging, as the neural generators of the EEG/MEG and BOLD signals might in some cases be very different. Here we describe a data fusion approach that was developed by our team over the last decade in which fMRI is used to provide source constraints that are based on functional areas defined individually for each subject. This mini-review describes the different steps that are necessary to perform source estimation using this approach. It also provides a list of pitfalls that should be avoided when doing fMRI-informed EEG/MEG source imaging. Finally, it describes the advantages of using a ROI-based approach for group-level analysis and for the study of sensory systems.