Disparity-tuned population responses from human visual cortex

Distinct rich and diverse clubs regulate coarse and fine binocular disparity processing: Evidence from stereoscopic task-based fMRI

While cortical regions involved in processing binocular disparities have been studied extensively, little is known on how the human visual system adapts to changing disparity magnitudes. In this paper, we investigate causal mechanisms of coarse and fine binocular disparity processing using fMRI with a clinically validated, custom anaglyph-based stimulus. We make use of Granger causality and graph measures to reveal the existence of distinct rich and diverse clubs across different disparity magnitudes. We demonstrate that Middle Temporal area (MT) plays a specialized role with overlapping rich and diverse characteristics. Next, we show that subtle interhemispheric differences exist across various brain regions, despite an overall right hemisphere dominance. Finally, we pass the graph measures through the decision tree and found that the diverse clubs outperform rich clubs in decoding disparity magnitudes. Our study sets the stage for conducting further investigations on binocular disparity processing, particularly in the context of neuro-ophthalmic disorders with binocular impairments.

Two Disparity Channels in Human Visual Cortex With Different Contrast and Blur Sensitivity

Purpose

Our goal is to describe the contrast and blur sensitivity of multiple horizontal disparity subsystems and to relate them to the contrast and spatial sensitivities of their monocular inputs.

Methods

Steady-state visual evoked potential (SSVEP) amplitudes were recorded in response to dynamic random dot stereograms (DRDSs) alternating at 2 Hz between zero disparity and varying magnitudes of crossed disparity for disparity plane and disparity grating stimuli. Half-image contrasts ranged between 2.5% and 80% and over a range of Gaussian blurs from 1.4 to 12 arcmin. Separate experiments measured contrast and blur sensitivity for the monocular half-images.

Results

The first and second harmonics disparity responses were maximal for disparity gratings and for the disparity plane condition, respectively. The first harmonic of the disparity grating response was more affected by both contrast and blur than was the second harmonic of the disparity plane response, which had higher contrast sensitivity than the first harmonic.

Conclusions

The corrugation frequency, contrast, and blur tuning of the first harmonic suggest that it reflects activity of neurons tuned to higher luminance spatial frequencies that are selective for relative disparity, whereas the second harmonic reflects the activity of neurons sensitive to absolute disparity that are driven by low monocular spatial frequencies.

Translational Relevance

SSVEPs to DRDSs provide two objective neural measures of disparity processing, the first harmonic-whose stimulus preferences are similar to those of behavioral stereoacuity-and the second harmonic that represents an independent disparity-specific but not necessarily stereoscopic mechanism.

Steady state visual evoked potentials reveal a signature of the pitch-size crossmodal association in visual cortex

Crossmodal correspondences describe our tendency to associate sensory features from different modalities with each other, such as the pitch of a sound with the size of a visual object. While such crossmodal correspondences (or associations) are described in many behavioural studies their neurophysiological correlates remain unclear. Under the current working model of multisensory perception both a low- and a high-level account seem plausible. That is, the neurophysiological processes shaping these associations could commence in low-level sensory regions, or may predominantly emerge in high-level association regions of semantic and object identification networks. We exploited steady-state visual evoked potentials (SSVEP) to directly probe this question, focusing on the associations between pitch and the visual features of size, hue or chromatic saturation. We found that SSVEPs over occipital regions are sensitive to the congruency between pitch and size, and a source analysis pointed to an origin around primary visual cortices. We speculate that this signature of the pitch-size association in low-level visual cortices reflects the successful pairing of congruent visual and acoustic object properties and may contribute to establishing causal relations between multisensory objects. Besides this, our study also provides a paradigm can be exploited to study other crossmodal associations involving visual stimuli in the future.

Cognitive penetrability of scene representations based on horizontal image disparities

The structure of natural scenes is signaled by many visual cues. Principal amongst them are the binocular disparities created by the laterally separated viewpoints of the two eyes. Disparity cues are believed to be processed hierarchically, first in terms of local measurements of absolute disparity and second in terms of more global measurements of relative disparity that allow extraction of the depth structure of a scene. Psychophysical and oculomotor studies have suggested that relative disparities are particularly relevant to perception, whilst absolute disparities are not. Here, we compare neural responses to stimuli that isolate the absolute disparity cue with stimuli that contain additional relative disparity cues, using the high temporal resolution of EEG to determine the temporal order of absolute and relative disparity processing. By varying the observers' task, we assess the extent to which each cue is cognitively penetrable. We find that absolute disparity is extracted before relative disparity, and that task effects arise only at or after the extraction of relative disparity. Our results indicate a hierarchy of disparity processing stages leading to the formation of a proto-object representation upon which higher cognitive processes can act.

Dynamics of absolute and relative disparity processing in human visual cortex

Cortical processing of binocular disparity is believed to begin in V1 where cells are sensitive to absolute disparity, followed by the extraction of relative disparity in higher visual areas. While much is known about the cortical distribution and spatial tuning of disparity-selective neurons, the relationship between their spatial and temporal properties is less well understood. Here, we use steady-state Visual Evoked Potentials and dynamic random dot stereograms to characterize the temporal dynamics of spatial mechanisms in human visual cortex that are primarily sensitive to either absolute or relative disparity. Stereograms alternated between disparate and non-disparate states at 2 Hz. By varying the disparity-defined spatial frequency content of the stereograms from a planar surface to corrugated ones, we biased responses towards absolute vs. relative disparities. Reliable Components Analysis was used to derive two dominant sources from the 128 channel EEG records. The first component (RC1) was maximal over the occipital pole. In RC1, first harmonic responses were sustained, tuned for corrugation frequency, and sensitive to the presence of disparity references, consistent with prior psychophysical sensitivity measurements. By contrast, the second harmonic, associated with transient processing, was not spatially tuned and was indifferent to references, consistent with it being generated by an absolute disparity mechanism. Thus, our results reveal a duplex coding strategy in the disparity domain, where relative disparities are computed via sustained mechanisms and absolute disparities are computed via transient mechanisms.

Excitatory Contribution to Binocular Interactions in Human Visual Cortex Is Reduced in Strabismic Amblyopia

Binocular summation in strabismic amblyopia is typically reported as being absent or greatly reduced in behavioral studies and is thought to be because of a preferential loss of excitatory interactions between the eyes. Here, we studied how excitatory and suppressive interactions contribute to binocular contrast interactions along the visual cortical hierarchy of humans with strabismic and anisometropic amblyopia in both sexes, using source-imaged steady-state visual evoked potentials (SSVEP) over a wide range of relative contrast between the two eyes. Dichoptic parallel grating stimuli modulated at unique temporal frequencies in each eye allowed us to quantify spectral response components associated with monocular inputs (self-terms) and the response components because of interaction of the inputs of the two eyes [intermodulation (IM) terms]. Although anisometropic amblyopes revealed a similar pattern of responses to normal-vision observers, strabismic amblyopes exhibited substantially reduced IM responses across cortical regions of interest (V1, V3a, hV4, hMT+ and lateral occipital cortex), indicating reduced interocular interactions in visual cortex. A contrast gain control model that simultaneously fits self- and IM-term responses within each cortical area revealed different patterns of binocular interactions between individuals with normal and disrupted binocularity. Our model fits show that in strabismic amblyopia, the excitatory contribution to binocular interactions is significantly reduced in both V1 and extra-striate cortex, whereas suppressive contributions remain intact. Our results provide robust electrophysiological evidence supporting the view that disruption of binocular interactions in strabismus or amblyopia is because of preferential loss of excitatory interactions between the eyes.SIGNIFICANCE STATEMENT We studied how excitatory and suppressive interactions contribute to binocular contrast interactions along the visual cortical hierarchy of humans with normal and amblyopic vision, using source-imaged SSVEP and frequency-domain analysis of dichoptic stimuli over a wide range of relative contrast between the two eyes. A dichoptic contrast gain control model was used to characterize these interactions in amblyopia and provided a quantitative comparison to normal vision. Our model fits revealed different patterns of binocular interactions between normal and amblyopic vision. Strabismic amblyopia significantly reduced excitatory contributions to binocular interactions, whereas suppressive contributions remained intact. Our results provide robust evidence supporting the view that the preferential loss of excitatory interactions disrupts binocular interactions in strabismic amblyopia.

Ultra-High-Field Neuroimaging Reveals Fine-Scale Processing for 3D Perception

Binocular disparity provides critical information about three-dimensional (3D) structures to support perception and action. In the past decade significant progress has been made in uncovering human brain areas engaged in the processing of binocular disparity signals. Yet, the fine-scale brain processing underlying 3D perception remains unknown. Here, we use ultra-high-field (7T) functional imaging at submillimeter resolution to examine fine-scale BOLD fMRI signals involved in 3D perception. In particular, we sought to interrogate the local circuitry involved in disparity processing by sampling fMRI responses at different positions relative to the cortical surface (i.e., across cortical depths corresponding to layers). We tested for representations related to 3D perception by presenting participants (male and female, N = 8) with stimuli that enable stable stereoscopic perception [i.e., correlated random dot stereograms (RDS)] versus those that do not (i.e., anticorrelated RDS). Using multivoxel pattern analysis (MVPA), we demonstrate cortical depth-specific representations in areas V3A and V7 as indicated by stronger pattern responses for correlated than for anticorrelated stimuli in upper rather than deeper layers. Examining informational connectivity, we find higher feedforward layer-to-layer connectivity for correlated than anticorrelated stimuli between V3A and V7. Further, we observe disparity-specific feedback from V3A to V1 and from V7 to V3A. Our findings provide evidence for the role of V3A as a key nexus for disparity processing, which is implicated in feedforward and feedback signals related to the perceptual estimation of 3D structures.SIGNIFICANCE STATEMENT Binocular vision plays a significant role in supporting our interactions with the surrounding environment. The fine-scale neural mechanisms that underlie the brain's skill in extracting 3D structures from binocular signals are poorly understood. Here, we capitalize on recent advances in ultra-high-field functional imaging to interrogate human brain circuits involved in 3D perception at submillimeter resolution. We provide evidence for the role of area V3A as a key nexus for disparity processing, which is implicated in feedforward and feedback signals related to the perceptual estimation of 3D structures from binocular signals. These fine-scale measurements help bridge the gap between animal neurophysiology and human fMRI studies investigating cross-scale circuits, from micro circuits to global brain networks for 3D perception.

Distinct neural sources underlying visual word form processing as revealed by steady state visual evoked potentials (SSVEP)

EEG has been central to investigations of the time course of various neural functions underpinning visual word recognition. Recently the steady-state visual evoked potential (SSVEP) paradigm has been increasingly adopted for word recognition studies due to its high signal-to-noise ratio. Such studies, however, have been typically framed around a single source in the left ventral occipitotemporal cortex (vOT). Here, we combine SSVEP recorded from 16 adult native English speakers with a data-driven spatial filtering approach—Reliable Components Analysis (RCA)—to elucidate distinct functional sources with overlapping yet separable time courses and topographies that emerge when contrasting words with pseudofont visual controls. The first component topography was maximal over left vOT regions with a shorter latency (approximately 180 ms). A second component was maximal over more dorsal parietal regions with a longer latency (approximately 260 ms). Both components consistently emerged across a range of parameter manipulations including changes in the spatial overlap between successive stimuli, and changes in both base and deviation frequency. We then contrasted word-in-nonword and word-in-pseudoword to test the hierarchical processing mechanisms underlying visual word recognition. Results suggest that these hierarchical contrasts fail to evoke a unitary component that might be reasonably associated with lexical access.

Human primary visual cortex shows larger population receptive fields for binocular disparity-defined stimuli

The visual perception of 3D depth is underpinned by the brain's ability to combine signals from the left and right eyes to produce a neural representation of binocular disparity for perception and behaviour. Electrophysiological studies of binocular disparity over the past 2 decades have investigated the computational role of neurons in area V1 for binocular combination, while more recent neuroimaging investigations have focused on identifying specific roles for different extrastriate visual areas in depth perception. Here we investigate the population receptive field properties of neural responses to binocular information in striate and extrastriate cortical visual areas using ultra-high field fMRI. We measured BOLD fMRI responses while participants viewed retinotopic mapping stimuli defined by different visual properties: contrast, luminance, motion, correlated and anti-correlated stereoscopic disparity. By fitting each condition with a population receptive field model, we compared quantitatively the size of the population receptive field for disparity-specific stimulation. We found larger population receptive fields for disparity compared with contrast and luminance in area V1, the first stage of binocular combination, which likely reflects the binocular integration zone, an interpretation supported by modelling of the binocular energy model. A similar pattern was found in region LOC, where it may reflect the role of disparity as a cue for 3D shape. These findings provide insight into the binocular receptive field properties underlying processing for human stereoscopic vision.

Optimal Approach for Signal Detection in Steady-State Visual Evoked Potentials in Humans Using Single-Channel EEG and Stereoscopic Stimuli

In EEG studies, one of the most common ways to detect a weak periodic signal in the steady-state visual evoked potential (SSVEP) is spectral evaluation, a process that detects peaks of power present at notable temporal frequencies. However, the presence of noise decreases the signal-to-noise ratio (SNR), which in turn lowers the probability of successful detection of these spectral peaks. In this paper, using a single EEG channel, we compare the detection performance of four different metrics to analyse the SSVEP: two metrics that use spectral power density, and two other metrics that use phase coherency. We employ these metrics find weak signals with a known temporal frequency hidden in the SSVEP, using both simulation and real data from a stereoscopic apparent depth movement perception task. We demonstrate that out of these metrics, the phase coherency analysis is the most sensitive way to find weak signals in the SSVEP, provided that the phase information of the stimulus eliciting the SSVEP is preserved.

Decoding Neural Responses to Motion-in-Depth Using EEG

Two stereoscopic cues that underlie the perception of motion-in-depth (MID) are changes in retinal disparity over time (CD) and interocular velocity differences (IOVD). These cues have independent spatiotemporal sensitivity profiles, depend upon different low-level stimulus properties, and are potentially processed along separate cortical pathways. Here, we ask whether these MID cues code for different motion directions: do they give rise to discriminable patterns of neural signals, and is there evidence for their convergence onto a single "motion-in-depth" pathway? To answer this, we use a decoding algorithm to test whether, and when, patterns of electroencephalogram (EEG) signals measured from across the full scalp, generated in response to CD- and IOVD-isolating stimuli moving toward or away in depth can be distinguished. We find that both MID cue type and 3D-motion direction can be decoded at different points in the EEG timecourse and that direction decoding cannot be accounted for by static disparity information. Remarkably, we find evidence for late processing convergence: IOVD motion direction can be decoded relatively late in the timecourse based on a decoder trained on CD stimuli, and vice versa. We conclude that early CD and IOVD direction decoding performance is dependent upon fundamentally different low-level stimulus features, but that later stages of decoding performance may be driven by a central, shared pathway that is agnostic to these features. Overall, these data are the first to show that neural responses to CD and IOVD cues that move toward and away in depth can be decoded from EEG signals, and that different aspects of MID-cues contribute to decoding performance at different points along the EEG timecourse.

Functional specialization in human dorsal pathway for stereoscopic depth processing

Binocular disparity, a primary cue for stereoscopic depth perception, is widely represented in visual cortex. However, the functional specialization in the disparity processing network remains unclear. Using magnetic resonance imaging-guided transcranial magnetic stimulation, we studied the causal contributions of V3A and MT+ to stereoscopic depth perception. Subjects viewed random-dot stereograms forming transparent planes with various interplane disparities. Their smallest detectable disparity and largest detectable disparity were measured in two experiments. We found that the smallest detectable disparity was affected by V3A, but not MT+ , stimulation. On the other hand, the largest detectable disparity was affected by both V3A and MT+ stimulation. Our results suggest different roles of V3A and MT+ in stereoscopic depth processing.

Antero-Posterior vs. Lateral Vestibular Input Processing in Human Visual Cortex

Visuo-vestibular integration is crucial for locomotion, yet the cortical mechanisms involved remain poorly understood. We combined binaural monopolar galvanic vestibular stimulation (GVS) and functional magnetic resonance imaging (fMRI) to characterize the cortical networks activated during antero-posterior and lateral stimulations in humans. We focused on functional areas that selectively respond to egomotion-consistent optic flow patterns: the human middle temporal complex (hMT+), V6, the ventral intraparietal (VIP) area, the cingulate sulcus visual (CSv) area and the posterior insular cortex (PIC). Areas hMT+, CSv, and PIC were equivalently responsive during lateral and antero-posterior GVS while areas VIP and V6 were highly activated during antero-posterior GVS, but remained silent during lateral GVS. Using psychophysiological interaction (PPI) analyses, we confirmed that a cortical network including areas V6 and VIP is engaged during antero-posterior GVS. Our results suggest that V6 and VIP play a specific role in processing multisensory signals specific to locomotion during navigation.

Stereomotion Processing in the Nonhuman Primate Brain

The cortical areas that process disparity-defined motion-in-depth (i.e., cyclopean stereomotion [CSM]) were characterized with functional magnetic resonance imaging (fMRI) in two awake, behaving macaques. The experimental protocol was similar to previous human neuroimaging studies. We contrasted the responses to dynamic random-dot patterns that continuously changed their binocular disparity over time with those to a control condition that shared the same properties, except that the temporal frames were shuffled. A whole-brain voxel-wise analysis revealed that in all four cortical hemispheres, three areas showed consistent sensitivity to CSM. Two of them were localized respectively in the lower bank of the superior temporal sulcus (CSMSTS) and on the neighboring infero-temporal gyrus (CSMITG). The third area was situated in the posterior parietal cortex (CSMPPC). Additional regions of interest-based analyses within retinotopic areas defined in both animals indicated weaker but significant responses to CSM within the MT cluster (most notably in areas MSTv and FST). Altogether, our results are in agreement with previous findings in both human and macaque and suggest that the cortical areas that process CSM are relatively well preserved between the two primate species.

Contrast Normalization Accounts for Binocular Interactions in Human Striate and Extra-striate Visual Cortex

During binocular viewing, visual inputs from the two eyes interact at the level of visual cortex. Here we studied binocular interactions in human visual cortex, including both sexes, using source-imaged steady-state visual evoked potentials over a wide range of relative contrast between two eyes. The ROIs included areas V1, V3a, hV4, hMT⁺, and lateral occipital cortex. Dichoptic parallel grating stimuli in each eye modulated at distinct temporal frequencies allowed us to quantify spectral components associated with the individual stimuli from monocular inputs (self-terms) and responses due to interaction between the inputs from the two eyes (intermodulation [IM] terms). Data with self-terms revealed an interocular suppression effect, in which the responses to the stimulus in one eye were reduced when a stimulus was presented simultaneously to the other eye. The suppression magnitude varied depending on visual area, and the relative contrast between the two eyes. Suppression was strongest in V1 and V3a (50% reduction) and was least in lateral occipital cortex (20% reduction). Data with IM terms revealed another form of binocular interaction, compared with self-terms. IM response was strongest at V1 and was least in hV4. Fits of a family of divisive gain control models to both self- and IM-term responses within each cortical area indicated that both forms of binocular interaction shared a common gain control nonlinearity. However, our model fits revealed different patterns of binocular interaction along the cortical hierarchy, particularly in terms of excitatory and suppressive contributions.SIGNIFICANCE STATEMENT Using source-imaged steady-state visual evoked potentials and frequency-domain analysis of dichoptic stimuli, we measured two forms of binocular interactions: one is associated with the individual stimuli that represent interocular suppression from each eye, and the other is a direct measure of interocular interaction between inputs from the two eyes. We demonstrated that both forms of binocular interactions share a common gain control mechanism in striate and extra-striate cortex. Furthermore, our model fits revealed different patterns of binocular interaction along the visual cortical hierarchy, particularly in terms of excitatory and suppressive contributions.

Asymmetries in Global Perception Are Represented in Near- versus Far-Preferring Clusters in Human Visual Cortex

Human perception is more “global” when stimuli are viewed within the lower (rather than the upper) visual field. This phenomenon is typically considered as a 2-D phenomenon, likely due to differential neural processing within dorsal versus ventral cortical areas that represent lower versus upper visual fields, respectively. Here we test a novel hypothesis that this vertical asymmetry in global processing is a 3-D phenomenon associated with (1) higher ecological relevance of low-spatial frequency (SF) components in encoding near (compared with far) visual objects and (2) the fact that near objects are more frequently found in lower rather than upper visual fields. Using high-resolution fMRI, collected within an ultra-high-field (7 T) scanner, we found that the extent of vertical asymmetry in global visual processing in human subjects (n = 10) was correlated with the fMRI response evoked by disparity-varying stimuli in human cortical area V3A. We also found that near-preferring clusters in V3A, located within stereoselective cortical columns, responded more selectively than far-preferring clusters, to low-SF features. These findings support the hypothesis that vertical asymmetry in global processing is a 3-D (not a 2-D) phenomenon, associated with the function of the stereoselective columns within visual cortex, especially those located within visual area V3A.

SIGNIFICANCE STATEMENT Here we test and confirm a new hypothesis: fine-scale neural mechanisms underlying the vertical asymmetry in global visual processing. According to this hypothesis, the asymmetry in global visual processing is a 3-D (rather than a 2-D) phenomenon, reflected in the function of fine-scale cortical structures (clusters and columns) underlying depth perception. Our findings highlight the importance of considering these structures, as regions of interest, in clarifying the neural mechanisms underlying visual perception. The results also highlight the importance of statistics of natural scenes in shaping human visual perception.

Adaptation to Binocular Anticorrelation Results in Increased Neural Excitability

Throughout the brain, information from individual sources converges onto higher order neurons. For example, information from the two eyes first converges in binocular neurons in area V1. Some neurons are tuned to similarities between sources of information, which makes intuitive sense in a system striving to match multiple sensory signals to a single external cause-that is, establish causal inference. However, there are also neurons that are tuned to dissimilar information. In particular, some binocular neurons respond maximally to a dark feature in one eye and a light feature in the other. Despite compelling neurophysiological and behavioral evidence supporting the existence of these neurons [Katyal, S., Vergeer, M., He, S., He, B., & Engel, S. A. Conflict-sensitive neurons gate interocular suppression in human visual cortex. Scientific Reports, 8, 1239, 2018; Kingdom, F. A. A., Jennings, B. J., & Georgeson, M. A. Adaptation to interocular difference. Journal of Vision, 18, 9, 2018; Janssen, P., Vogels, R., Liu, Y., & Orban, G. A. At least at the level of inferior temporal cortex, the stereo correspondence problem is solved. Neuron, 37, 693-701, 2003; Tsao, D. Y., Conway, B. R., & Livingstone, M. S. Receptive fields of disparity-tuned simple cells in macaque V1. Neuron, 38, 103-114, 2003; Cumming, B. G., & Parker, A. J. Responses of primary visual cortical neurons to binocular disparity without depth perception. Nature, 389, 280-283, 1997], their function has remained opaque. To determine how neural mechanisms tuned to dissimilarities support perception, here we use electroencephalography to measure human observers' steady-state visually evoked potentials in response to change in depth after prolonged viewing of anticorrelated and correlated random-dot stereograms (RDS). We find that adaptation to anticorrelated RDS results in larger steady-state visually evoked potentials, whereas adaptation to correlated RDS has no effect. These results are consistent with recent theoretical work suggesting "what not" neurons play a suppressive role in supporting stereopsis [Goncalves, N. R., & Welchman, A. E. "What not" detectors help the brain see in depth. Current Biology, 27, 1403-1412, 2017]; that is, selective adaptation of neurons tuned to binocular mismatches reduces suppression resulting in increased neural excitability.

Identifying Cortical Substrates Underlying the Phenomenology of Stereopsis and Realness: A Pilot fMRI Study

Viewing a real scene or a stereoscopic image (e.g., 3D movies) with both eyes yields a vivid subjective impression of object solidity, tangibility, immersive negative space and sense of realness; something that is not experienced when viewing single pictures of 3D scenes normally with both eyes. This phenomenology, sometimes referred to as stereopsis, is conventionally ascribed to the derivation of depth from the differences in the two eye's images (binocular disparity). Here we report on a pilot study designed to explore if dissociable neural activity associated with the phenomenology of realness can be localized in the cortex. In order to dissociate subjective impression from disparity processing, we capitalized on the finding that the impression of realness associated with stereoscopic viewing can also be generated when viewing a single picture of a 3D scene with one eye through an aperture. Under a blocked fMRI design, subjects viewed intact and scrambled images of natural 3-D objects, and scenes under three viewing conditions: (1) single pictures viewed normally with both eyes (binocular); (2) single pictures viewed with one eye through an aperture (monocular-aperture); and (3) stereoscopic anaglyph images of the same scenes viewed with both eyes (binocular stereopsis). Fixed-effects GLM contrasts aimed at isolating the phenomenology of stereopsis demonstrated a selective recruitment of similar posterior parietal regions for both monocular and binocular stereopsis conditions. Our findings provide preliminary evidence that the cortical processing underlying the subjective impression of realness may be dissociable and distinct from the derivation of depth from disparity.

Dissociating neural activity associated with the subjective phenomenology of monocular stereopsis: An EEG study

The subjective phenomenology associated with stereopsis, of solid tangible objects separated by a palpable negative space, is conventionally thought to be a by-product of the derivation of depth from binocular disparity. However, the same qualitative impression has been reported in the absence of disparity, e.g., when viewing pictorial images monocularly through an aperture. Here we aimed to explore if we could identify dissociable neural activity associated with the qualitative impression of stereopsis in the absence of the processing of binocular disparities. We measured EEG activity while subjects viewed pictorial (non-stereoscopic) images of 2D and 3D geometric forms under four different viewing conditions (binocular, monocular, binocular aperture, monocular aperture). EEG activity was analysed by oscillatory source localization (beamformer technique) to examine power change in occipital and parietal regions across viewing and stimulus conditions in targeted frequency bands (alpha: 8-13 Hz & gamma: 60-90 Hz). We observed expected event-related gamma synchronization and alpha desynchronization in occipital cortex and predominant gamma synchronization in parietal cortex across viewing and stimulus conditions. However, only the viewing condition predicted to generate the strongest impression of stereopsis (monocular aperture) revealed significantly elevated gamma synchronization within the parietal cortex for the critical contrasts (3D vs. 2D form). These findings suggest dissociable neural processes specific to the qualitative impression of stereopsis as distinguished from disparity processing.

Disparity level identification using the voxel-wise Gabor model of fMRI data

Perceiving disparities is the intuitive basis for our understanding of the physical world. Although many electrophysiology studies have revealed the disparity-tuning characteristics of the neurons in the visual areas of the macaque brain, neuron population responses to disparity processing have seldom been investigated. Many disparity studies using functional magnetic resonance imaging (fMRI) have revealed the disparity-selective visual areas in the human brain. However, it is unclear how to characterize neuron population disparity-tuning responses using fMRI technique. In the present study, we constructed three voxel-wise encoding Gabor models to predict the voxel responses to novel disparity levels and used a decoding method to identify the new disparity levels from population responses in the cortex. Among the three encoding models, the fine-coarse model (FCM) that used fine/coarse disparities to fit the voxel responses to disparities outperformed the single model and uncrossed-crossed model. Moreover, the FCM demonstrated high accuracy in predicting voxel responses in V3A complex and high accuracy in identifying novel disparities from responses in V3A complex. Our results suggest that the FCM can better characterize the voxel responses to disparities than the other two models and V3A complex is a critical visual area for representing disparity information.

Microstructural properties of the vertical occipital fasciculus explain the variability in human stereoacuity

Seeing in the three-dimensional world—stereopsis—is an innate human ability, but it varies substantially among individuals. The neurobiological basis of this variability is not understood. We combined diffusion and quantitative MRI imaging with a psychophysical measurements, and found that variability in stereoacuity is associated with microstructural differences in the right vertical occipital fasciculus, a white matter tract connecting dorsal and ventral visual cortex. This result suggests that the microstructure of the pathways that support information transmission across dorsal and ventral visual areas plays an important role human stereopsis.

Stereopsis is a fundamental visual function that has been studied extensively. However, it is not clear why depth discrimination (stereoacuity) varies more significantly among people than other modalities. Previous studies have reported the involvement of both dorsal and ventral visual areas in stereopsis, implying that not only neural computations in cortical areas but also the anatomical properties of white matter tracts connecting those areas can impact stereopsis. Here, we studied how human stereoacuity relates to white matter properties by combining psychophysics, diffusion MRI (dMRI), and quantitative MRI (qMRI). We performed a psychophysical experiment to measure stereoacuity and, in the same participants, we analyzed the microstructural properties of visual white matter tracts on the basis of two independent measurements, dMRI (fractional anisotropy, FA) and qMRI (macromolecular tissue volume; MTV). Microstructural properties along the right vertical occipital fasciculus (VOF), a major tract connecting dorsal and ventral visual areas, were highly correlated with measures of stereoacuity. This result was consistent for both FA and MTV, suggesting that the behavioral–structural relationship reflects differences in neural tissue density, rather than differences in the morphological configuration of fibers. fMRI confirmed that binocular disparity stimuli activated the dorsal and ventral visual regions near VOF endpoints. No other occipital tracts explained the variance in stereoacuity. In addition, the VOF properties were not associated with differences in performance on a different psychophysical task (contrast detection). These series of experiments suggest that stereoscopic depth discrimination performance is, at least in part, constrained by dorso-ventral communication through the VOF.

Simple reaction times to cyclopean stimuli reveal that the binocular system is tuned to react faster to near than to far objects

Binocular depth perception is an important mechanism to segregate the visual scene for mapping relevant objects in our environment. Convergent evidence from psychophysical and neurophysiological studies have revealed asymmetries between the processing of near (crossed) and far (uncrossed) binocular disparities. The aim of the present study was to test if near or far objects are processed faster and with higher contrast sensitivity in the visual system. We therefore measured the relationship between binocular disparity and simple reaction time (RT) as well as contrast gain based on the contrast-RT function in young healthy adults. RTs were measured to suddenly appearing cyclopean target stimuli, which were checkerboard patterns encoded by depth in dynamic random dot stereograms (DRDS). The DRDS technique allowed us to selectively study the stereoscopic processing system by eliminating all monocular cues. The results showed that disparity and contrast had significant effects on RTs. RTs as a function of disparity followed a U-shaped tuning curve indicating an optimum at around 15 arc min, where RTs were minimal. Surprisingly, the disparity tuning of RT was much less pronounced for far disparities. At the optimal disparity, we measured advantages of about 80 ms and 30 ms for near disparities at low (10%) and high (90%) contrasts, respectively. High contrast always reduced RTs as well as the disparity dependent differences. Furthermore, RT-based contrast gains were higher for near disparities in the range of disparities where RTs were the shortest. These results show that the sensitivity of the human visual system is biased for near versus far disparities and near stimuli can result in faster motor responses, probably because they bear higher biological relevance.

Development of Relative Disparity Sensitivity in Human Visual Cortex

Stereopsis is the primary cue underlying our ability to make fine depth judgments. In adults, depth discriminations are supported largely by relative rather than absolute binocular disparity, and depth is perceived primarily for horizontal rather than vertical disparities. Although human infants begin to exhibit disparity-specific responses between 3 and 5 months of age, it is not known how relative disparity mechanisms develop. Here we show that the specialization for relative disparity is highly immature in 4- to 6-month-old infants but is adult-like in 4- to 7-year-old children. Disparity-tuning functions for horizontal and vertical disparities were measured using the visual evoked potential. Infant relative disparity thresholds, unlike those of adults, were equal for vertical and horizontal disparities. Their horizontal disparity thresholds were a factor of ∼10 higher than adults, but their vertical disparity thresholds differed by a factor of only ∼4. Horizontal relative disparity thresholds for 4- to 7-year-old children were comparable with those of adults at ∼0.5 arcmin. To test whether infant immaturity was due to spatial limitations or insensitivity to interocular correlation, highly suprathreshold horizontal and vertical disparities were presented in alternate regions of the display, and the interocular correlation of the interdigitated regions was varied from 0% to 100%. This manipulation regulated the availability of coarse-scale relative disparity cues. Adult and infant responses both increased with increasing interocular correlation by similar magnitudes, but adult responses increased much more for horizontal disparities, further evidence for qualitatively immature stereopsis based on relative disparity at 4-6 months of age.SIGNIFICANCE STATEMENT Stereopsis, our ability to sense depth from horizontal image disparity, is among the finest spatial discriminations made by the primate visual system. Fine stereoscopic depth discriminations depend critically on comparisons of disparity relationships in the image that are supported by relative disparity cues rather than the estimation of single, absolute disparities. Very young human and macaque infants are sensitive to absolute disparity, but no previous study has specifically studied the development of relative disparity sensitivity, a hallmark feature of adult stereopsis. Here, using high-density EEG recordings, we show that 4- to 6-month-old infants display both quantitative and qualitative response immaturities for relative disparity information. Relative disparity responses are adult-like no later than 4-7 years of age.

Cortical sources of Vernier acuity in the human visual system: An EEG-source imaging study

Vernier acuity determines the relative position of visual features with a precision better than the sampling resolution of cone receptors in the retina. Because Vernier displacement is thought to be mediated by orientation-tuned mechanisms, Vernier acuity is presumed to be processed in striate visual cortex (V1). However, there is considerable evidence suggesting that Vernier acuity is dependent not only on structures in V1 but also on processing in extrastriate cortical regions. Here we used functional magnetic resonance imaging–informed electroencephalogram source imaging to localize the cortical sources of Vernier acuity in observers with normal vision. We measured suprathreshold and near-threshold responses to Vernier onset/offset stimuli at different stages of the visual cortical hierarchy, including V1, hV4, lateral occipital cortex (LOC), and middle temporal cortex (hMT+). These responses were compared with responses to grating on/off stimuli, as well as to stimuli that control for lateral motion in the Vernier task. Our results show that all visual cortical regions of interest (ROIs) responded to both suprathreshold Vernier and grating stimuli. However, thresholds for Vernier displacement (Vernier acuity) were lowest in V1 and LOC compared with hV4 and hMT+, whereas all visual ROIs had identical thresholds for spatial frequency (grating acuity) and for relative motion. The cortical selectivity of sensitivity to Vernier displacement provides strong evidence that LOC, in addition to V1, is involved in Vernier acuity processing. The robust activation of LOC might be related to the sensitivity to the relative position of features, which is common to Vernier displacement and to some kinds of texture segmentation.

The Active Side of Stereopsis: Fixation Strategy and Adaptation to Natural Environments

Depth perception in near viewing strongly relies on the interpretation of binocular retinal disparity to obtain stereopsis. Statistical regularities of retinal disparities have been claimed to greatly impact on the neural mechanisms that underlie binocular vision, both to facilitate perceptual decisions and to reduce computational load. In this paper, we designed a novel and unconventional approach in order to assess the role of fixation strategy in conditioning the statistics of retinal disparity. We integrated accurate realistic three-dimensional models of natural scenes with binocular eye movement recording, to obtain accurate ground-truth statistics of retinal disparity experienced by a subject in near viewing. Our results evidence how the organization of human binocular visual system is finely adapted to the disparity statistics characterizing actual fixations, thus revealing a novel role of the active fixation strategy over the binocular visual functionality. This suggests an ecological explanation for the intrinsic preference of stereopsis for a close central object surrounded by a far background, as an early binocular aspect of the figure-ground segregation process.

The visual white matter: The application of diffusion MRI and fiber tractography to vision science

Visual neuroscience has traditionally focused much of its attention on understanding the response properties of single neurons or neuronal ensembles. The visual white matter and the long-range neuronal connections it supports are fundamental in establishing such neuronal response properties and visual function. This review article provides an introduction to measurements and methods to study the human visual white matter using diffusion MRI. These methods allow us to measure the microstructural and macrostructural properties of the white matter in living human individuals; they allow us to trace long-range connections between neurons in different parts of the visual system and to measure the biophysical properties of these connections. We also review a range of findings from recent studies on connections between different visual field maps, the effects of visual impairment on the white matter, and the properties underlying networks that process visual information supporting visual face recognition. Finally, we discuss a few promising directions for future studies. These include new methods for analysis of MRI data, open datasets that are becoming available to study brain connectivity and white matter properties, and open source software for the analysis of these data.

Degraded attentional modulation of cortical neural populations in strabismic amblyopia

Behavioral studies have reported reduced spatial attention in amblyopia, a developmental disorder of spatial vision. However, the neural populations in the visual cortex linked with these behavioral spatial attention deficits have not been identified. Here, we use functional MRI–informed electroencephalography source imaging to measure the effect of attention on neural population activity in the visual cortex of human adult strabismic amblyopes who were stereoblind. We show that compared with controls, the modulatory effects of selective visual attention on the input from the amblyopic eye are substantially reduced in the primary visual cortex (V1) as well as in extrastriate visual areas hV4 and hMT+. Degraded attentional modulation is also found in the normal-acuity fellow eye in areas hV4 and hMT+ but not in V1. These results provide electrophysiological evidence that abnormal binocular input during a developmental critical period may impact cortical connections between the visual cortex and higher level cortices beyond the known amblyopic losses in V1 and V2, suggesting that a deficit of attentional modulation in the visual cortex is an important component of the functional impairment in amblyopia. Furthermore, we find that degraded attentional modulation in V1 is correlated with the magnitude of interocular suppression and the depth of amblyopia. These results support the view that the visual suppression often seen in strabismic amblyopia might be a form of attentional neglect of the visual input to the amblyopic eye.

Representation of Maximally Regular Textures in Human Visual Cortex

Naturalistic textures with an intermediate degree of statistical regularity can capture key structural features of natural images (Freeman and Simoncelli, 2011). V2 and later visual areas are sensitive to these features, while primary visual cortex is not (Freeman et al., 2013). Here we expand on this work by investigating a class of textures that have maximal formal regularity, the 17 crystallographic wallpaper groups (Fedorov, 1891). We used texture stimuli from four of the groups that differ in the maximum order of rotation symmetry they contain, and measured neural responses in human participants using functional MRI and high-density EEG. We found that cortical area V3 has a parametric representation of the rotation symmetries in the textures that is not present in either V1 or V2, the first discovery of a stimulus property that differentiates processing in V3 from that of lower-level areas. Parametric responses were also seen in higher-order ventral stream areas V4, VO1, and lateral occipital complex (LOC), but not in dorsal stream areas. The parametric response pattern was replicated in the EEG data, and source localization indicated that responses in V3 and V4 lead responses in LOC, which is consistent with a feedforward mechanism. Finally, we presented our stimuli to four well developed feedforward models and found that none of them were able to account for our results. Our results highlight structural regularity as an important stimulus dimension for distinguishing the early stages of visual processing, and suggest a previously unrecognized role for V3 in the visual form-processing hierarchy. Significance statement: Hierarchical processing is a fundamental organizing principle in visual neuroscience, with each successive processing stage being sensitive to increasingly complex stimulus properties. Here, we probe the encoding hierarchy in human visual cortex using a class of visual textures--wallpaper patterns--that are maximally regular. Through a combination of fMRI and EEG source imaging, we find specific responses to texture regularity that depend parametrically on the maximum order of rotation symmetry in the textures. These parametric responses are seen in several areas of the ventral visual processing stream, as well as in area V3, but not in V1 or V2. This is the first demonstration of a stimulus property that differentiates processing in V3 from that of lower-level visual areas.

Visual adaptation provides objective electrophysiological evidence of facial identity discrimination

Discrimination of facial identities is a fundamental function of the human brain that is challenging to examine with macroscopic measurements of neural activity, such as those obtained with functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). Although visual adaptation or repetition suppression (RS) stimulation paradigms have been successfully implemented to this end with such recording techniques, objective evidence of an identity-specific discrimination response due to adaptation at the level of the visual representation is lacking. Here, we addressed this issue with fast periodic visual stimulation (FPVS) and EEG recording combined with a symmetry/asymmetry adaptation paradigm. Adaptation to one facial identity is induced through repeated presentation of that identity at a rate of 6 images per second (6 Hz) over 10 sec. Subsequently, this identity is presented in alternation with another facial identity (i.e., its anti-face, both faces being equidistant from an average face), producing an identity repetition rate of 3 Hz over a 20 sec testing sequence. A clear EEG response at 3 Hz is observed over the right occipito-temporal (ROT) cortex, indexing discrimination between the two facial identities in the absence of an explicit behavioral discrimination measure. This face identity discrimination occurs immediately after adaptation and disappears rapidly within 20 sec. Importantly, this 3 Hz response is not observed in a control condition without the single-identity 10 sec adaptation period. These results indicate that visual adaptation to a given facial identity produces an objective (i.e., at a pre-defined stimulation frequency) electrophysiological index of visual discrimination between that identity and another, and provides a unique behavior-free quantification of the effect of visual adaptation.

fMRI Activity in Posterior Parietal Cortex Relates to the Perceptual Use of Binocular Disparity for Both Signal-In-Noise and Feature Difference Tasks

Visually guided action and interaction depends on the brain’s ability to (a) extract and (b) discriminate meaningful targets from complex retinal inputs. Binocular disparity is known to facilitate this process, and it is an open question how activity in different parts of the visual cortex relates to these fundamental visual abilities. Here we examined fMRI responses related to performance on two different tasks (signal-in-noise “coarse” and feature difference “fine” tasks) that have been widely used in previous work, and are believed to differentially target the visual processes of signal extraction and feature discrimination. We used multi-voxel pattern analysis to decode depth positions (near vs. far) from the fMRI activity evoked while participants were engaged in these tasks. To look for similarities between perceptual judgments and brain activity, we constructed ‘fMR-metric’ functions that described decoding performance as a function of signal magnitude. Thereafter we compared fMR-metric and psychometric functions, and report an association between judged depth and fMRI responses in the posterior parietal cortex during performance on both tasks. This highlights common stages of processing during perceptual performance on these tasks.

fMRI Analysis-by-Synthesis Reveals a Dorsal Hierarchy That Extracts Surface Slant

The brain's skill in estimating the 3-D orientation of viewed surfaces supports a range of behaviors, from placing an object on a nearby table, to planning the best route when hill walking. This ability relies on integrating depth signals across extensive regions of space that exceed the receptive fields of early sensory neurons. Although hierarchical selection and pooling is central to understanding of the ventral visual pathway, the successive operations in the dorsal stream are poorly understood. Here we use computational modeling of human fMRI signals to probe the computations that extract 3-D surface orientation from binocular disparity. To understand how representations evolve across the hierarchy, we developed an inference approach using a series of generative models to explain the empirical fMRI data in different cortical areas. Specifically, we simulated the responses of candidate visual processing algorithms and tested how well they explained fMRI responses. Thereby we demonstrate a hierarchical refinement of visual representations moving from the representation of edges and figure-ground segmentation (V1, V2) to spatially extensive disparity gradients in V3A. We show that responses in V3A are little affected by low-level image covariates, and have a partial tolerance to the overall depth position. Finally, we show that responses in V3A parallel perceptual judgments of slant. This reveals a relatively short computational hierarchy that captures key information about the 3-D structure of nearby surfaces, and more generally demonstrates an analysis approach that may be of merit in a diverse range of brain imaging domains.

Functional architecture for disparity in macaque inferior temporal cortex and its relationship to the architecture for faces, color, scenes, and visual field

Binocular disparity is a powerful depth cue for object perception. The computations for object vision culminate in inferior temporal cortex (IT), but the functional organization for disparity in IT is unknown. Here we addressed this question by measuring fMRI responses in alert monkeys to stimuli that appeared in front of (near), behind (far), or at the fixation plane. We discovered three regions that showed preferential responses for near and far stimuli, relative to zero-disparity stimuli at the fixation plane. These "near/far" disparity-biased regions were located within dorsal IT, as predicted by microelectrode studies, and on the posterior inferotemporal gyrus. In a second analysis, we instead compared responses to near stimuli with responses to far stimuli and discovered a separate network of "near" disparity-biased regions that extended along the crest of the superior temporal sulcus. We also measured in the same animals fMRI responses to faces, scenes, color, and checkerboard annuli at different visual field eccentricities. Disparity-biased regions defined in either analysis did not show a color bias, suggesting that disparity and color contribute to different computations within IT. Scene-biased regions responded preferentially to near and far stimuli (compared with stimuli without disparity) and had a peripheral visual field bias, whereas face patches had a marked near bias and a central visual field bias. These results support the idea that IT is organized by a coarse eccentricity map, and show that disparity likely contributes to computations associated with both central (face processing) and peripheral (scene processing) visual field biases, but likely does not contribute much to computations within IT that are implicated in processing color.

7 tesla FMRI reveals systematic functional organization for binocular disparity in dorsal visual cortex

The binocular disparity between the views of the world registered by the left and right eyes provides a powerful signal about the depth structure of the environment. Despite increasing knowledge of the cortical areas that process disparity from animal models, comparatively little is known about the local architecture of stereoscopic processing in the human brain. Here, we take advantage of the high spatial specificity and image contrast offered by 7 tesla fMRI to test for systematic organization of disparity representations in the human brain. Participants viewed random dot stereogram stimuli depicting different depth positions while we recorded fMRI responses from dorsomedial visual cortex. We repeated measurements across three separate imaging sessions. Using a series of computational modeling approaches, we report three main advances in understanding disparity organization in the human brain. First, we show that disparity preferences are clustered and that this organization persists across imaging sessions, particularly in area V3A. Second, we observe differences between the local distribution of voxel responses in early and dorsomedial visual areas, suggesting different cortical organization. Third, using modeling of voxel responses, we show that higher dorsal areas (V3A, V3B/KO) have properties that are characteristic of human depth judgments: a simple model that uses tuning parameters estimated from fMRI data captures known variations in human psychophysical performance. Together, these findings indicate that human dorsal visual cortex contains selective cortical structures for disparity that may support the neural computations that underlie depth perception.

Orienting of visuo-spatial attention in complex 3D space: Search and detection

The ability to detect changes in the environment is necessary for appropriate interactions with the external world. Changes in the background go more unnoticed than foreground changes, possibly because attention prioritizes processing of foreground/near stimuli. Here, we investigated the detectability of foreground and background changes within natural scenes and the influence of stereoscopic depth cues on this. Using a flicker paradigm, we alternated a pair of images that were exactly same or differed for one single element (i.e., a color change of one object in the scene). The participants were asked to find the change that occurred either in a foreground or background object, while viewing the stimuli either with binocular and monocular cues (bmC) or monocular cues only (mC). The behavioral results showed faster and more accurate detections for foreground changes and overall better performance in bmC than mC conditions. The imaging results highlighted the involvement of fronto‐parietal attention controlling networks during active search and target detection. These attention networks did not show any differential effect as function of the presence/absence of the binocular cues, or the detection of foreground/background changes. By contrast, the lateral occipital cortex showed greater activation for detections in foreground compared to background, while area V3A showed a main effect of bmC vs. mC, specifically during search. These findings indicate that visual search with binocular cues does not impose any specific requirement on attention‐controlling fronto‐parietal networks, while the enhanced detection of front/near objects in the bmC condition reflects bottom‐up sensory processes in visual cortex. Hum Brain Mapp 36:2231–2247, 2015. © 2015 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc.

The steady-state visual evoked potential in vision research: A review

Periodic visual stimulation and analysis of the resulting steady-state visual evoked potentials were first introduced over 80 years ago as a means to study visual sensation and perception. From the first single-channel recording of responses to modulated light to the present use of sophisticated digital displays composed of complex visual stimuli and high-density recording arrays, steady-state methods have been applied in a broad range of scientific and applied settings.The purpose of this article is to describe the fundamental stimulation paradigms for steady-state visual evoked potentials and to illustrate these principles through research findings across a range of applications in vision science.

Vergence eye movements in patients with schizophrenia

Previous studies have shown that smooth pursuit eye movements are impaired in patients with schizophrenia. However, under normal viewing conditions, targets move not only in the frontoparallel plane but also in depth, and tracking them requires both smooth pursuit and vergence eye movements. Although previous studies in humans and non-human primates suggest that these two eye movement subsystems are relatively independent of one another, to our knowledge, there have been no prior studies of vergence tracking behavior in patients with schizophrenia. Therefore, we have investigated these eye movements in patients with schizophrenia and in healthy controls. We found that patients with schizophrenia exhibited substantially lower gains compared to healthy controls during vergence tracking at all tested speeds (e.g. 0.25 Hz vergence tracking mean gain of 0.59 vs. 0.86). Further, consistent with previous reports, patients with schizophrenia exhibited significantly lower gains than healthy controls during smooth pursuit at higher target speeds (e.g. 0.5 Hz smooth pursuit mean gain of 0.64 vs. 0.73). In addition, there was a modest (r≈0.5), but significant, correlation between smooth pursuit and vergence tracking performance in patients with schizophrenia. Our observations clearly demonstrate substantial vergence tracking deficits in patients with schizophrenia. In these patients, deficits for smooth pursuit and vergence tracking are partially correlated suggesting overlap in the central control of smooth pursuit and vergence eye movements.

How to use fMRI functional localizers to improve EEG/MEG source estimation

EEG and MEG have excellent temporal resolution, but the estimation of the neural sources that generate the signals recorded by the sensors is a difficult, ill-posed problem. The high spatial resolution of functional MRI makes it an ideal tool to improve the localization of the EEG/MEG sources using data fusion. However, the combination of the two techniques remains challenging, as the neural generators of the EEG/MEG and BOLD signals might in some cases be very different. Here we describe a data fusion approach that was developed by our team over the last decade in which fMRI is used to provide source constraints that are based on functional areas defined individually for each subject. This mini-review describes the different steps that are necessary to perform source estimation using this approach. It also provides a list of pitfalls that should be avoided when doing fMRI-informed EEG/MEG source imaging. Finally, it describes the advantages of using a ROI-based approach for group-level analysis and for the study of sensory systems.

Cortical responses to optic flow and motion contrast across patterns and speeds

Motion provides animals with fast and robust cues for navigation and object detection. In the first case, stereotyped patterns of optic flow inform a moving observer about the direction and speed of its own movement. In the case of object detection, regional differences in motion allow for the segmentation of figures from their background, even in the absence of color or shading cues. Previous research has investigated human electrophysiological responses to global motion across speeds, but only focused upon one type of optic flow pattern. Here, we compared steady-state visual evoked potential (SSVEP) responses across patterns and speeds, both for optic flow and for motion-defined figure patterns, to assess the extent to which the processes are pattern-general or pattern-specific. For optic flow, pattern and speed effects on response amplitudes varied substantially across channels, suggesting pattern-specific processing at slow speeds and pattern-general activity at fast speeds. Responses for coherence- and direction-defined figures were comparatively more uniform, with similar response profiles and spatial distributions. Self- and object-motion patterns activate some of the same circuits, but these data suggest differential sensitivity: not only across the two classes of motion, but also across the patterns within each class, and across speeds. Thus, the results demonstrate that cortical processing of global motion is complex and activates a distributed network.

The evolution of a disparity decision in human visual cortex

We used fMRI-informed EEG source-imaging in humans to characterize the dynamics of cortical responses during a disparity-discrimination task. After the onset of a disparity-defined target, decision-related activity was found within an extended cortical network that included several occipital regions of interest (ROIs): V4, V3A, hMT+ and the Lateral Occipital Complex (LOC). By using a response-locked analysis, we were able to determine the timing relationships in this network of ROIs relative to the subject's behavioral response. Choice-related activity appeared first in the V4 ROI almost 200 ms before the button press and then subsequently in the V3A ROI. Modeling of the responses in the V4 ROI suggests that this area provides an early contribution to disparity discrimination. Choice-related responses were also found after the button-press in ROIs V4, V3A, LOC and hMT+. Outside the visual cortex, choice-related activity was found in the frontal and temporal poles before the button-press. By combining the spatial resolution of fMRI-informed EEG source imaging with the ability to sort out neural activity occurring before, during and after the behavioral manifestation of the decision, our study is the first to assign distinct functional roles to the extra-striate ROIs involved in perceptual decisions based on disparity, the primary cue for depth.

Dynamics and cortical distribution of neural responses to 2D and 3D motion in human

The perception of motion-in-depth is important for avoiding collisions and for the control of vergence eye-movements and other motor actions. Previous psychophysical studies have suggested that sensitivity to motion-in-depth has a lower temporal processing limit than the perception of lateral motion. The present study used functional MRI-informed EEG source-imaging to study the spatiotemporal properties of the responses to lateral motion and motion-in-depth in human visual cortex. Lateral motion and motion-in-depth displays comprised stimuli whose only difference was interocular phase: monocular oscillatory motion was either in-phase in the two eyes (lateral motion) or in antiphase (motion-in-depth). Spectral analysis was used to break the steady-state visually evoked potentials responses down into even and odd harmonic components within five functionally defined regions of interest: V1, V4, lateral occipital complex, V3A, and hMT+. We also characterized the responses within two anatomically defined regions: the inferior and superior parietal cortex. Even harmonic components dominated the evoked responses and were a factor of approximately two larger for lateral motion than motion-in-depth. These responses were slower for motion-in-depth and were largely independent of absolute disparity. In each of our regions of interest, responses at odd-harmonics were relatively small, but were larger for motion-in-depth than lateral motion, especially in parietal cortex, and depended on absolute disparity. Taken together, our results suggest a plausible neural basis for reduced psychophysical sensitivity to rapid motion-in-depth.

Linking brain imaging signals to visual perception

The rapid advances in brain imaging technology over the past 20 years are affording new insights into cortical processing hierarchies in the human brain. These new data provide a complementary front in seeking to understand the links between perceptual and physiological states. Here we review some of the challenges associated with incorporating brain imaging data into such "linking hypotheses," highlighting some of the considerations needed in brain imaging data acquisition and analysis. We discuss work that has sought to link human brain imaging signals to existing electrophysiological data and opened up new opportunities in studying the neural basis of complex perceptual judgments. We consider a range of approaches when using human functional magnetic resonance imaging to identify brain circuits whose activity changes in a similar manner to perceptual judgments and illustrate these approaches by discussing work that has studied the neural basis of 3D perception and perceptual learning. Finally, we describe approaches that have sought to understand the information content of brain imaging data using machine learning and work that has integrated multimodal data to overcome the limitations associated with individual brain imaging approaches. Together these approaches provide an important route in seeking to understand the links between physiological and psychological states.

Linking brain to behavior for the visual perception of figures and objects

The dissociation of a figure from its background is an essential feat of visual perception, as it allows us to detect, recognize, and interact with shapes and objects in our environment. In order to understand how the human brain gives rise to the perception of figures, we here review experiments that explore the links between activity in visual cortex and performance of perceptual tasks related to figure perception. We organize our review according to a proposed model that attempts to contextualize figure processing within the more general framework of object processing in the brain. Overall, the current literature provides us with individual linking hypotheses as to cortical regions that are necessary for particular tasks related to figure perception. Attempts to reach a more complete understanding of how the brain instantiates figure and object perception, however, will have to consider the temporal interaction between the many regions involved, the details of which may vary widely across different tasks.

Using the kinetic Zollner illusion to quantify the interaction between form and motion information in depth

In the kinetic Zollner illusion a stimulus moving over a background of oriented lines appears tilted away from the line orientation. This "motion-tilt" illusion is a powerful demonstration of how form information can influence the computation of motion, particularly in signaling motion direction. In the present study, using a random dot stereogram of the kinetic Zollner illusion, we examined whether and how the degree of motion tilt is affected when form and motion components of the illusion are separated in depth. In Experiment 1 we showed that increasing the depth separation (by increasing binocular disparity) between the moving stimulus and oriented lines attenuated the motion-tilt effect. Motion tilt induction was observed for depth separations of -18 to 18 arcmin in uncrossed and crossed directions, but not at larger separations. In Experiment 2 we showed that motion tilt induction in the kinetic Zollner illusion was also observed when multiple oriented planes were presented in conjunction with a moving stimulus. However, the direction and extent of the illusory motion tilt was determined by the nearest oriented plane. Collectively, these findings show that the interaction of form and motion is dependent on depth and is optimally tuned for a small range of separations.

Attention selects informative neural populations in human V1

In a neural population driven by a simple grating stimulus, different subpopulations are maximally informative about changes to the grating's orientation and contrast. In theory, observers should attend to the optimal subpopulation when switching between orientation and contrast discrimination tasks. Here we used source-imaged, steady-state visual evoked potentials and visual psychophysics to determine whether this is the case. Observers fixated centrally while static targets were presented bilaterally along with a cue indicating task type (contrast or orientation modulation detection) and task location (left or right). Changes in neuronal activity were measured by quantifying frequency-tagged responses from flickering "reporter" gratings surrounding the targets. To determine the orientation tuning of attentionally modulated neurons, we measured responses for three different probe-reporter angles: 0, 20, and 45°. We estimated frequency-tagged cortical activity using a minimum norm inverse procedure combined with realistic MR-derived head models and retinotopically mapped visual areas. Estimates of neural activity from regions of interest centered on V1 showed that attention to a spatial location clearly increased the amplitude of the neural response in that location. More importantly, the pattern of modulation depended on the task. For orientation discrimination, attentional modulation showed a sharp peak in the population tuned 20° from the target orientation, whereas for contrast discrimination the enhancement was more broadly tuned. Similar tuning functions for orientation and contrast discrimination were obtained from psychophysical adaptation studies. These findings indicate that humans attend selectively to the most informative neural population and that these populations change depending on the nature of the task.

Cytoarchitectonic mapping of the human dorsal extrastriate cortex

The dorsal visual stream consists of several functionally specialized areas, but most of their cytoarchitectonic correlates have not yet been identified in the human brain. The cortex adjacent to Brodmann area 18/V2 was therefore analyzed in serial sections of ten human post-mortem brains using morphometrical and multivariate statistical analyses for the definition of areal borders. Two previously unknown cytoarchitectonic areas (hOc3d, hOc4d) were detected. They occupy the medial and, to a smaller extent, lateral surface of the occipital lobe. The larger area, hOc3d, is located dorso-lateral to area V2 in the region of superior and transverse occipital, as well as parieto-occipital sulci. Area hOc4d was identified rostral to hOc3d; it differed from the latter by larger pyramidal cells in lower layer III, thinner layers V and VI, and a sharp cortex-white-matter borderline. The delineated areas were superimposed in the anatomical MNI space, and probabilistic maps were calculated. They show a relatively high intersubject variability in volume and position. Based on their location and neighborhood relationship, areas hOc3d and hOc4d are putative anatomical substrates of functionally defined areas V3d and V3a, a hypothesis that can now be tested by comparing probabilistic cytoarchitectonic maps and activation studies of the living human brain.

The selectivity of task-dependent attention varies with surrounding context

Attention is thought to operate by enhancing the target of interest and suppressing the surroundings. We hypothesized that the spatial profile of attention depends on the surround's relationship to the target. Using high-density electroencephalographic measurements, we examined the spatial profile of attention to a grating target surrounded by an annular grating that was either coextensive with the target (unsegmented) or appeared segmented from it due to a gap or phase offset. We directly probed the spread of attention from the central target into the surround by flickering the surround and monitoring frequency-tagged steady-state visual-evoked potentials. Observers were required to detect a contrast increment that occurred only on the target. Successful detection of the increment required selecting the target and suppressing the surround, particularly when the target did not readily segment from the surround. The profile of attention was investigated in five visual regions of interest (ROIs) (V1, V4, V3A, lateral occipital complex, and human middle temporal area), mapped in a separate anatomical magnetic resonance imaging scan. We found that in most ROIs, attention to the target generated smaller responses from the surrounding annulus when it was contiguous compared with when it was clearly segmented. This result shows that the profile of attention depends on task demands and on surrounding context; attention is tightly focused when the target region needs to be isolated but loosely focused when the target region is clearly segmented.

Increasing the accuracy of electromagnetic inverses using functional area source correlation constraints

Estimating cortical current distributions from electroencephalographic (EEG) or magnetoencephalographic data is a difficult inverse problem whose solution can be improved by the addition of priors on the associated neural responses. In the context of visual activation studies, we propose a new approach that uses a functional area constrained estimator (FACE) to increase the accuracy of the reconstructions. It derives the source correlation matrix from a segmentation of the cortex into areas defined by retinotopic maps of the visual field or by functional localizers obtained independently by fMRI. These areas are computed once for each individual subject and the associated estimators can therefore be reused for any new study on the same participant. The resulting FACE reconstructions emphasize the activity of sources within these areas or enforce their intercorrelations. We used realistic Monte-Carlo simulations to demonstrate that this approach improved our estimates of a diverse set of source configurations. Reconstructions obtained from a real EEG dataset demonstrate that our priors improve the localization of the cortical areas involved in horizontal disparity processing.