At what stage of neural processing do perspective depth cues make a difference?

The human brain deals with violating general color or depth knowledge in different time courses

Utilizing the high temporal resolution of event-related potentials (ERPs), we compared the time course of processing incongruent color versus 3D-depth information. Participants were asked to judge whether the food color (color condition) or 3D structure (3D-depth condition) was congruent or incongruent with their previous knowledge and experience. The behavioral results showed that the reaction times in the congruent 3D-depth condition were slower than those in the congruent color condition. The reaction times in the incongruent 3D-depth condition were slower than those in the incongruent color condition. The ERP results showed that incongruent color stimuli induced a larger N270, larger P300, and smaller N400 components in the fronto-central region than the congruent color stimuli. Incongruent 3D-depth stimuli induced a smaller N1 in the occipital region, larger P300 and smaller N400 in the parietal-occipital region than congruent 3D-depth stimuli. The time-frequency analysis found that incongruent color stimuli induced a larger theta band (360-580 ms) activation in the fronto-central region than congruent color stimuli. Incongruent 3D-depth stimuli induced larger alpha and beta bands (240-350 ms) activation in the parietal region than congruent 3D-depth stimuli. Our results suggest that the human brain deals with violating general color or depth knowledge in different time courses. We speculate that the depth perception conflict was dominated by solving the problem with visual processing, whereas the color perception conflict was dominated by solving the problem with semantic violation.

Neural Research on Depth Perception and Stereoscopic Visual Fatigue in Virtual Reality

Virtual reality (VR) technology provides highly immersive depth perception experiences; nevertheless, stereoscopic visual fatigue (SVF) has become an important factor currently hindering the development of VR applications. However, there is scant research on the underlying neural mechanism of SVF, especially those induced by VR displays, which need further research. In this paper, a Go/NoGo paradigm based on disparity variations is proposed to induce SVF associated with depth perception, and the underlying neural mechanism of SVF in a VR environment was investigated. The effects of disparity variations as well as SVF on the temporal characteristics of visual evoked potentials (VEPs) were explored. Point-by-point permutation statistical with repeated measures ANOVA results revealed that the amplitudes and latencies of the posterior VEP component P2 were modulated by disparities, and posterior P2 amplitudes were modulated differently by SVF in different depth perception situations. Cortical source localization analysis was performed to explore the original cortex areas related to certain fatigue levels and disparities, and the results showed that posterior P2 generated from the precuneus could represent depth perception in binocular vision, and therefore could be performed to distinguish SVF induced by disparity variations. Our findings could help to extend an understanding of the neural mechanisms underlying depth perception and SVF as well as providing beneficial information for improving the visual experience in VR applications.

Investigate the Neuro Mechanisms of Stereoscopic Visual Fatigue

Stereoscopic visual fatigue (SVF) due to prolonged immersion in the virtual environment can lead to negative user experience, thus hindering the development of virtual reality (VR) industry. Previous studies have focused on investigating the evaluation indicators associated with SVF, while few studies have been conducted to reveal the underlying neural mechanism, especially in VR applications. In this paper, a modified Go/NoGo paradigm was adopted to induce SVF in VR environment with Go trials for maintaining participants' attention to experimental viewing tasks and NoGo trials for investigating the neural effects under SVF. Random dot stereograms (RDSs) with 11 disparities and 2 types of shapes (arrow and rectangle) were presented to evoke the depth-related visual evoked potentials (DVEPs) during 64-channel EEG recordings. EEG datasets collected from 15 participants in NoGo trials were selected to conduct individual processing and group analysis, in which the characteristics of the DVEPs components for various fatigue degrees were compared with one-way repeated-measurement ANOVA and independent components were clustered to explore the original cortex areas related to SVF. Point-by-point permutation statistics revealed that DVEPs sample points from 230ms to 280ms in most brain areas changed significantly with SVF. More specifically, we found that amplitudes of component P2 changed significantly when SVF increased. Additionally, independent component analysis (ICA) identified that component P2 which originated from posterior cingulate cortex and precuneus, was associated statistically with SVF. We believe that SVF is rather a conscious status concerning the changes of self-awareness or self-location awareness than the performance reduction of retinal image processing. Moreover, we suggest that indicators representing higher conscious state may be a better indicator for SVF evaluation in VR environments.

Unique Neural Activity Patterns Among Lower Order Cortices and Shared Patterns Among Higher Order Cortices During Processing of Similar Shapes With Different Stimulus Types

We investigated the neural mechanism of the processing of three-dimensional (3D) shapes defined by disparity and perspective. We measured blood oxygenation level-dependent signals as participants viewed and classified 3D images of convex-concave shapes. According to the cue (disparity or perspective) and element type (random dots or black and white dotted lines), three types of stimuli were used: random dot stereogram, black and white dotted lines with perspective, and black and white dotted lines with binocular disparity. The blood oxygenation level-dependent images were then classified by multivoxel pattern analysis. To identify areas selective to shape, we assessed convex-concave classification accuracy with classifiers trained and tested using signals evoked by the same stimulus type (same cue and element type). To identify cortical regions with similar neural activity patterns regardless of stimulus type, we assessed the convex-concave classification accuracy of transfer classification in which classifiers were trained and tested using different stimulus types (different cues or element types). Classification accuracy using the same stimulus type was high in the early visual areas and subregions of the intraparietal sulcus (IPS), whereas transfer classification accuracy was high in the dorsal subregions of the IPS. These results indicate that the early visual areas process the specific features of stimuli, whereas the IPS regions perform more generalized processing of 3D shapes, independent of a specific stimulus type.

Investigate the 3D Visual Fatigue Using Modified Depth-Related Visual Evoked Potential Paradigm

Prolonged viewing of 3D content may result in severe fatigue symptoms, giving negative user experience thus hindering the development of 3D industry. For 3D visual fatigue evaluation, previous studies focused on exploring the changes of frequency-domain features in EEG for various fatigue degrees. However, their time-domain features were scarcely investigated. In this study, a modified paradigm with a random disparities order is adopted to evoke the depth-related visual evoked potentials (DVEPs). Then the characteristics of the DVEPs components for various fatigue degrees are compared using one-way repeated-measurement ANOVA. Point-by-point permutation statistics revealed sample points from 100ms to 170ms - including P1 and N1 - in sensors Pz and P4 changed significantly with visual fatigue. More specifically, we find that the amplitudes of P1 and N1 change significantly when visual fatigue increases. Additionally, independent component analysis identify P1 and N1 which originate from posterior cingulate cortex are associated statistically with 3D visual fatigue. Our results indicate there is a significant correlation between 3D visual fatigue and P1 amplitude, as well as N1, of DVEPs on right parietal areas. We believe the characteristics (e.g., amplitude and latency) of identified components may be the indicators of 3D visual fatigue evaluation. Furthermore, we argue that 3D visual fatigue may be associated with the activities decrease of the attention and the processing capacity of disparity.

Effects of stereoscopic disparity on early ERP components during classification of three-dimensional objects

This study investigates the effects of stereo disparity on the perception of three-dimensional (3D) object shape. We tested the hypothesis that stereo input modulates the brain activity related to perceptual analyses of 3D shape configuration during image classification. High-density (256-channel) electroencephalogram (EEG) was used to record the temporal dynamics of visual shape processing under conditions of two-dimensional (2D) and 3D visual presentation. On each trial, observers made image classification judgements ('Same'/'Different') to two briefly presented, multi-part, novel objects. On different-object trials, stimuli could either share volumetric parts but not the global 3D shape configuration and have different parts but the same global 3D shape configuration or differ on both aspects. Analyses using mass univariate contrasts showed that the earliest sensitivity to 2D versus 3D viewing appeared as a negative deflection over posterior locations on the N1 component between 160 and 220 ms post-stimulus onset. Subsequently, event-related potential (ERP) modulations during the N2 time window between 240 and 370 ms were linked to image classification. N2 activity reflected two distinct components - an early N2 (240-290 ms) and a late N2 (290-370 ms) - that showed different patterns of responses to 2D and 3D input and differential sensitivity to 3D object structure. The results revealed that stereo input modulates the neural correlates of 3D object shape. We suggest that this reflects differential perceptual processing of object shape under conditions of stereo or mono input. These findings challenge current theories that attribute no functional role for stereo input during 3D shape perception.

Quality assessment of 3D visualizations with vertical disparity: An ERP approach

In an objective approach for the assessment of quality of experience the neural correlates of EEG data are studied when stereoscopic images are presented in three different conditions containing vertical disparity. These conditions are compared to a similar image in 2D both on the channel level by studying the ERP components and on the source level by the localization of the corresponding ERP component. Our findings posit that P1 component in the occipital cortex has significantly increased in amplitude for 3D condition without vertical disparity compared to the 2D condition. According to previous studies, this component increases when depth information are added to the stimulus which is in line with our findings. However the amplitude of this component has significantly decreased for 3D condition with maximum vertical disparity compared to the 3D condition without vertical disparity. We have concluded that the perception of stereoscopic depth by subjects have decreased in this case due to the distortion introduced by vertical disparity. The underlying sources corresponding to P1 component are localized. Except for the power differences, the source locations do not differ for different conditions.

Objective quality assessment of stereoscopic images with vertical disparity using EEG

OBJECTIVE

Neurophysiological correlates of vertical disparity in 3D images are studied in an objective approach using EEG technique. These disparities are known to negatively affect the quality of experience and to cause visual discomfort in stereoscopic visualizations.

APPROACH

We have presented four conditions to subjects: one in 2D and three conditions in 3D, one without vertical disparity and two with different vertical disparity levels. Event related potentials (ERPs) are measured for each condition and the differences between ERP components are studied. Analysis is also performed on the induced potentials in the time frequency domain.

MAIN RESULTS

Results show that there is a significant increase in the amplitude of P1 components in 3D conditions in comparison to 2D. These results are consistent with previous studies which have shown that P1 amplitude increases due to the depth perception in 3D compared to 2D. However the amplitude is significantly smaller for maximum vertical disparity (3D-3) in comparison to 3D with no vertical disparity. Our results therefore suggest that the vertical disparity in 3D-3 condition decreases the perception of depth compared to other 3D conditions and the amplitude of P1 component can be used as a discriminative feature.

SIGNIFICANCE

The results show that the P1 component increases in amplitude due to the depth perception in the 3D stimuli compared to the 2D stimulus. On the other hand the vertical disparity in the stereoscopic images is studied here. We suggest that the amplitude of P1 component is modulated with this parameter and decreases due to the decrease in the perception of depth.

Electrophysiological evidence of separate pathways for the perception of depth and 3D objects

Previous studies have investigated the neural mechanism of 3D perception, but the neural distinction between 3D-objects and depth processing remains unclear. In the present study, participants viewed three types of graphics (planar graphics, perspective drawings, and 3D objects) while event-related potentials (ERP) were recorded. The ERP results revealed the following: (1) 3D objects elicited a larger and delayed N1 component than the other two types of stimuli; (2) during the P2 time window, significant differences between 3D objects and the perspective drawings were found mainly over a group of electrode sites in the left lateral occipital region; and (3) during the N2 complex, differences between planar graphics and perspective drawings were found over a group of electrode sites in the right hemisphere, whereas differences between perspective drawings and 3D objects were observed at another group of electrode sites in the left hemisphere. These findings support the claim that depth processing and object identification might be processed by separate pathways and at different latencies.

The role of the right superior parietal lobule in processing visual context for the establishment of the egocentric reference frame

Visual cues contribute to the creation of an observer's egocentric reference frame, within which the locations and orientations of objects can be judged. However, these cues can also be misleading. In the rod-and-frame illusion, for example, a large tilted frame distorts the observer's sense of vertical, causing an enclosed rod to appear tilted in the opposite direction. To determine the brain region responsible for processing these spatial cues, we used TMS to suppress neural activity in the superior parietal lobule of healthy observers. Stimulation of the right hemisphere, but not the left, caused a significant reduction in rod-and-frame susceptibility. In contrast, a tilt illusion caused by a mechanism that does not involve a distortion of the observer's egocentric reference frame was unaffected. These results demonstrate that the right superior parietal lobule is actively involved in processing the contextual cues that contribute to our perception of egocentric space.

Modulation of human visual evoked potentials in 3-dimensional perception after stimuli produced with an integral imaging method

We investigated the neurophysiological correlates of stereoscopic 3-dimensional (3-D) depth perception by studying human visual evoked potentials (VEPs) with an integral imaging method characterized by horizontal but not vertical disparity. The VEPs were recorded in 10 healthy men under 4 conditions. In condition I, stimuli A (flat, 2-dimensional [2-D] image) and B (concave 3-D image) were presented at random. In condition II, stimuli A and C (convex 3-D image) were presented at random. In condition III, stimuli B and C were presented at random. In condition IV, stimuli A, B, and C were presented at random. The data for flat VEPs to stimulus A were combined in conditions I and II. The data for concave VEPs to stimulus B were combined in conditions I and III. The data for convex VEPs to stimulus C were combined in conditions II and III. When 2-way analysis of variance (ANOVA) for 2 factors, stimulus conditions (flat VEPs, concave VEPs, and convex VEPs) and electrode positions, was applied for VEP data, the N1 and N2 peak amplitudes differed significantly among the 3 stimulus conditions. In condition IV, the N1 peak amplitudes differed significantly among the 3 stimuli. Multiple comparisons followed by Bonferroni adjustment did not detect differences in the N1 peak amplitude between stimuli A and B, between stimuli A and C, or between stimuli B and C. We concluded that VEPs to concave or convex 3-D stimuli were significantly different from VEPs to flat 2-D stimuli. This is the first report showing modulation of human VEPs in 3-D perception with an integral imaging method.

P1 and P2 components of human visual evoked potentials are modulated by depth perception of 3-dimensional images

OBJECTIVE

To determine the cerebral activity correlated with depth perception of 3-dimensional (3D) images, by recording of human visual evoked potentials (VEPs).

METHODS

Two figures consisting of smaller and larger squares were presented alternately. VEPs were recorded in two conditions. In condition I, we used two figures which yielded flat 2-dimensional images. In condition II, we used two figures which yielded 3D images, which were concave and convex, respectively.

RESULTS

P1, P2, and N1/P2 amplitude were significantly greater in condition II than in condition I. The P1/N1 amplitude tended to be greater in condition II than in condition I. P1 and N1 were predominantly distributed over the right temporo-parieto-occipital regions. P2 and N2 were distributed over bilateral parieto-occipital regions.

CONCLUSIONS

The difference in P1 amplitude between two conditions can be explained by the difference between conditions, one of which yielded depth perception while the other did not, since previous studies showed that P1 and N1 are modulated by perception of images in depth. The role of P2 and the mechanism responsible for the increase in P2 amplitude during condition II remain unknown.

SIGNIFICANCE

We recorded VEPs and identified electrophysiological correlates of depth perception with 3D images produced by concave/convex figures.

Mapping the connectivity with structural equation modeling in an fMRI study of shape-from-motion task

In this fMRI study, we explore the connectivity among brain regions in a shape-from-motion task using the causal mapping analysis of structural equation modeling (SEM). An important distinction of our approach is that we have adapted SEM from its traditional role in confirmatory analysis to provide utility as an exploratory mapping technique. Our current approaches include (I) detecting brain regions that fit well in a hypothesized neural network model, and (II) identifying the best connectivity model at each brain region. We demonstrate that SEM effectively detects the dorsal and ventral visual pathways from the covariance structure in fMRI data, confirming previous neuroscience results. Further, our SEM mapping methodology found that the two pathways interact through specific cortical areas such as the superior lateral occipital cortex in the perception of shape from motion.

An fMRI study of the reverse perspective illusion

"Reverse perspective" is a powerful visual illusion similar to the hollow mask illusion, but more interesting in producing the perception of an illusory motion in a stationary picture. It is caused by conflict between motion parallax and pictorial depth cues in 3D "relief" paintings built with depth inversion. Here we report the measurement of brain activation using fMRI in response to a reverse perspective (RP) object, as well as a normal perspective, 3D-relief object ("shadow-box", SB) and a 2D painting of the same architectural scene. The stimuli were presented to 10 subjects in static and rotating conditions, subtraction of which revealed strong activation of area MT in all three cases. Contrasts between the RP, SB and 2D conditions showed the strongest activation for RP and almost no difference between SB and 2D. The similarity of brain activation between SB and 2D stimuli was interpreted as indicating that observers perceive the illusion of realistic 3D depth in 2D pictures as entirely normal and not qualitatively different from the 3D structure of the shadow-box stimulus. Contrasts between the RP stimulus and either the SB or the 2D stimulus revealed activation of Brodmann Areas 7, 19 and MT (and cerebellar cortex), suggesting the usage of brain regions involved in mental rotation and depth perception in response to the reverse perspective illusion.