Investigating the relationship between three-dimensional perception and presence in virtual reality-reconstructed architecture

Identifying and characterizing the factors that affect presence in virtual environments has been acknowledged as a critical step to improving Virtual Reality (VR) applications in the built environment domain. In the search to identify those factors, the research objective was to test whether three-dimensional perception affects presence in virtual environments. A controlled within-group experiment utilizing perception and presence questionnaires was conducted, followed by data analysis, to test the hypothesized unidirectional association between three-dimensional perception and presence in two different virtual environments (non-immersive and immersive). Results indicate no association in either of the systems studied, contrary to the assumption of many scholars in the field but in line with recent studies on the topic. Consequently, VR applications in architectural design may not necessarily need to incorporate advanced stereoscopic visualization techniques to deliver highly immersive experiences, which may be achieved by addressing factors other than depth realism. As findings suggest that the levels of presence experienced by users are not subject to the display mode of a 3D model (whether immersive or non-immersive display), it may still be possible for professionals involved in the review of 3D models (e.g., designers, contractors, clients) to experience high levels of presence through non-stereoscopic VR systems provided that other presence-promoting factors are included.

Calibrating Vision: Concepts and Questions

The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.

Learning to see in depth

Stereopsis provides us with a vivid impression of the depth and distance of objects in our 3- dimensional world. Stereopsis is important for a number of everyday visual tasks, including (but not limited to) reaching and grasping, fine visuo-motor control, and navigating in our world. This review briefly discusses the neural substrate for normal binocular vision and stereopsis and its development in primates; outlines some of the issues and limitations of stereopsis tests and examines some of the factors that limit the typical development of stereopsis and the causes and consequences of stereo-deficiency and stereo-blindness. Finally, we review several approaches to improving or recovering stereopsis in both neurotypical individuals and those with stereo-deficiency and stereo-blindness and outline some emerging strategies for improving stereopsis.

Recent understanding of binocular vision in the natural environment with clinical implications

Technological advances in recent decades have allowed us to measure both the information available to the visual system in the natural environment and the rich array of behaviors that the visual system supports. This review highlights the tasks undertaken by the binocular visual system in particular and how, for much of human activity, these tasks differ from those considered when an observer fixates a static target on the midline. The everyday motor and perceptual challenges involved in generating a stable, useful binocular percept of the environment are discussed, together with how these challenges are but minimally addressed by much of current clinical interpretation of binocular function. The implications for new technology, such as virtual reality, are also highlighted in terms of clinical and basic research application.

The retinal and perceived locus of fixation in the human visual system

Due to the dramatic difference in spatial resolution between the central fovea and the surrounding retinal regions, accurate fixation on important objects is critical for humans. It is known that the preferred retinal location (PRL) for fixation of healthy human observers rarely coincides with the retinal location with the highest cone density. It is not currently known, however, whether the PRL is consistent within an observer or is subject to fluctuations and, moreover, whether observers' subjective fixation location coincides with the PRL. We studied whether the PRL changes between days. We used an adaptive optics scanning laser ophthalmoscope to project a Maltese cross fixation target on an observer's retina and continuously imaged the exact retinal location of the target. We found that observers consistently use the same PRL across days, regardless of how much the PRL is displaced from the cone density peak location. We then showed observers small stimuli near the visual field location on which they fixated, and the observers judged whether or not the stimuli appeared in fixation. Observers' precision in this task approached that of fixation itself. Observers based their judgment on both the visual scene coordinates and the retinal location of the stimuli. We conclude that the PRL in a normally functioning visual system is fixed, and observers use it as a reference point in judging stimulus locations.

The relationship between reflex eye realignment and the percept of single vision in young children

Effective binocular vision is dependent on both motor and perceptual function. Young children undergo development of both components while interacting with their dynamic three-dimensional environment. When this development fails, eye misalignment and double vision may result. We compared the range of image disparities over which young children display reflex motor realignment of their eyes with the range over which they report a single versus double percept. In response to step changes in the disparity of a 2.2° wide stimulus, 5-year-olds generated an adult-like reflex vergence velocity tuning function peaking at 2° of disparity, with a mean latency of 210 ms. On average, they reported double vision for stimulus disparities of 3° and larger, compared to 1° in adult reports. Three-year-olds also generated reflex vergence tuning functions peaking at approximately 2° of disparity, but their percepts could not be assessed. These data suggest that, by age 5, reflex eye realignment responses and percepts driven by these brief stimuli are tightly coordinated in space and time to permit robust binocular function around the point of fixation. Importantly, the plastic neural processes maintaining this tight coordination during growth control the stability of visual information driving learning during childhood.

Development of stereo vision in young infants

In this study, infants' visual processing of depth-inducing stimuli was tested using a new method suitable for experimental settings. Stereograms of the Lang-Stereopad^® were presented in a timed preferential-looking paradigm to determine infants' preference for a stereogram as compared to a stimulus not inducing an impression of depth. A total of 80 infants were tested at 7 months of age; of these, a sub-sample of 41 infants were tested longitudinally at 4 and 7 months to characterize the developmental trajectory of their preference. Infants were simultaneously presented with a card showing a random-dot stereogram (800" disparity) and a similar looking dummy card without stereogram. In the total sample, 7-month-olds showed a clear preference for the stereogram regardless of sex. In the longitudinal sample, 7-month-olds but not 4-month-olds looked significantly longer to the stereogram as compared to the dummy card. On individual level, 56% of the 4-month-olds and 85% of the 7-month-olds predominantly looked at the stereogram. The findings yield evidence for a clear developmental progression and show that the test cards of the Lang-Stereopad^® prototype provide a viable instrument to determine the preference for depth-inducing stimuli in young infants when used in a controlled experimental setting.

Contributions of binocular and monocular cues to motion-in-depth perception

Intercepting and avoiding moving objects requires accurate motion-in-depth (MID) perception. Such motion can be estimated based on both binocular and monocular cues. Because previous studies largely characterized sensitivity to these cues individually, their relative contributions to MID perception remain unclear. Here we measured sensitivity to binocular, monocular, and combined cue MID stimuli using a motion coherence paradigm. We first confirmed prior reports of substantial variability in binocular MID cue sensitivity across the visual field. The stimuli were matched for eccentricity and speed, suggesting that this variability has a neural basis. Second, we determined that monocular MID cue sensitivity also varied considerably across the visual field. A major component of this variability was geometric: An MID stimulus produces the largest motion signals in the eye contralateral to its visual field location. This resulted in better monocular discrimination performance when the contralateral rather than ipsilateral eye was stimulated. Third, we found that monocular cue sensitivity generally exceeded, and was independent of, binocular cue sensitivity. Finally, contralateral monocular cue sensitivity was found to be a strong predictor of combined cue sensitivity. These results reveal distinct factors constraining the contributions of binocular and monocular cues to three-dimensional motion perception.

Binocular Summation for Reflexive Eye Movements: A Potential Diagnostic Tool for Stereodeficiencies

Purpose

Stereoscopic vision, by detecting interocular correlations, enhances depth perception. Stereodeficiencies often emerge during the first months of life, and left untreated can lead to severe loss of visual acuity in one eye and/or strabismus. Early treatment results in much better outcomes, yet diagnostic tests for infants are cumbersome and not widely available. We asked whether reflexive eye movements, which in principle can be recorded even in infants, can be used to identify stereodeficiencies.

Methods

Reflexive ocular following eye movements induced by fast drifting noise stimuli were recorded in 10 adult human participants (5 with normal stereoacuity, 5 stereodeficient). To manipulate interocular correlation, the stimuli shown to the two eyes were either identical, different, or had opposite contrast. Monocular presentations were also interleaved. The participants were asked to passively fixate the screen.

Results

In the participants with normal stereoacuity, the responses to binocular identical stimuli were significantly larger than those induced by binocular opposite stimuli. In the stereodeficient participants the responses were indistinguishable. Despite the small size of ocular following responses, 40 trials, corresponding to less than 2 minutes of testing, were sufficient to reliably differentiate normal from stereodeficient participants.

Conclusions

Ocular-following eye movements, because of their reliance on cortical neurons sensitive to interocular correlations, are affected by stereodeficiencies. Because these eye movements can be recorded noninvasively and with minimal participant cooperation, they can potentially be measured even in infants and might thus provide an useful screening tool for this currently underserved population.

Revisiting the functional significance of binocular cues for perceiving motion-in-depth

Binocular differencing of spatial cues required for perceiving depth relationships is associated with decreased sensitivity to the corresponding retinal image displacements. However, binocular summation of contrast signals increases sensitivity. Here, we investigated this divergence in sensitivity by making direct neural measurements of responses to suprathreshold motion in human adults and 5-month-old infants using steady-state visually evoked potentials. Interocular differences in retinal image motion generated suppressed response functions and correspondingly elevated perceptual thresholds compared to motion matched between the two eyes. This suppression was of equal strength for horizontal and vertical motion and therefore not specific to the perception of motion-in-depth. Suppression is strongly dependent on the presence of spatial references in the image and highly immature in infants. Suppression appears to be the manifestation of a succession of spatial and interocular opponency operations that occur at an intermediate processing stage either before or in parallel with the extraction of motion-in-depth.

Infants perceive two-dimensional shape from horizontal disparity

Previous studies observed that responsiveness to horizontal disparity as such emerges at approximately 2 months of age. Moreover, 3- to 4-month-old infants utilize stereoscopic information to perceive object variations in depth. The present study investigated infants' ability to respond to crossed horizontal disparity information that defines two-dimensional shape. Infants 4 and 5 months of age were habituated to either a cross or the outline of a square. During the posthabituation period, they were presented with both shapes. The stimuli were dynamic random dot stereograms shown on an autostereoscopic monitor. The participants 5 but not 4 months of age displayed significant novelty preferences for the unfamiliar shape during the posthabituation period. Five-month-old infants are hence sensitive to horizontal disparity information that specifies shape.

Human infants can generate vergence responses to retinal disparity by 5 to 10 weeks of age

Vergence is defined as a binocular eye movement during which the two eyes move in opposite directions to align to a target in depth. In adults, fine vergence control is driven primarily by interocular retinal image disparity. Although infants have not typically been shown to respond to disparity until 3 to 5 months postpartum, they have been shown to align their eyes from hours after birth. It remains unclear what drives these responses in young infants. In this experiment, 5- to 10-week-old human infants were presented with a dynamic random noise stimulus oscillating in disparity at 0.1 Hz over an amplitude of 2° for 30 s. Fourier transforms of the horizontal eye movements revealed significant disparity-driven responses at the frequency of the stimulus in over half of the tested infants. Because the stimulus updated dynamically, this experiment precluded the possibility of independent monocular fixations to a sustained target. These data demonstrate cortical binocular function in humans by five weeks, the youngest age tested here, which is as much as two months younger than previously believed.

Early human motor development: From variation to the ability to vary and adapt

This review summarizes early human motor development. From early fetal age motor behavior is based on spontaneous neural activity: activity of networks in the brainstem and spinal cord that is modulated by supraspinal activity. The supraspinal activity, first primarily brought about by the cortical subplate, later by the cortical plate, induces movement variation. Initially, movement variation especially serves exploration; its associated afferent information is primarily used to sculpt the developing nervous system, and less to adapt motor behavior. In the next phase, beginning at function-specific ages, movement variation starts to serve adaptation. In sucking and swallowing, this phase emerges shortly before term age. In speech, gross and fine motor development, it emerges from 3 to 4 months post-term onwards, i.e., when developmental focus in the primary sensory and motor cortices has shifted to the permanent cortical circuitries. With increasing age and increasing trial-and-error exploration, the infant improves its ability to use adaptive and efficicient forms of upright gross motor behavior, manual activities and vocalizations belonging to the native language.

The life-span trajectory of visual perception of 3D objects

Deriving a 3D structural representation of an object from its 2D input is one of the great challenges for the visual system and yet, this type of representation is critical for the successful recognition of and interaction with objects. Perhaps reflecting the importance of this computation, infants have some sensitivity to 3D structural information, and this sensitivity is, at least, partially preserved in the elderly population. To map precisely the life-span trajectory of this key visual computation, in a series of experiments, we compared the performance of observers from ages 4 to 86 years on displays of objects that either obey or violate possible 3D structure. The major findings indicate that the ability to derive fine-grained 3D object representations emerges after a prolonged developmental trajectory and is contingent on the explicit processing of depth information even in late childhood. In contrast, the sensitivity to object 3D structure remains stable even through late adulthood despite the overall reduction in perceptual competence. Together, these results uncover the developmental process of an important perceptual skill, revealing that the initial, coarse sensitivity to 3D information is refined, automatized and retained over the lifespan.

Development of Relative Disparity Sensitivity in Human Visual Cortex

Stereopsis is the primary cue underlying our ability to make fine depth judgments. In adults, depth discriminations are supported largely by relative rather than absolute binocular disparity, and depth is perceived primarily for horizontal rather than vertical disparities. Although human infants begin to exhibit disparity-specific responses between 3 and 5 months of age, it is not known how relative disparity mechanisms develop. Here we show that the specialization for relative disparity is highly immature in 4- to 6-month-old infants but is adult-like in 4- to 7-year-old children. Disparity-tuning functions for horizontal and vertical disparities were measured using the visual evoked potential. Infant relative disparity thresholds, unlike those of adults, were equal for vertical and horizontal disparities. Their horizontal disparity thresholds were a factor of ∼10 higher than adults, but their vertical disparity thresholds differed by a factor of only ∼4. Horizontal relative disparity thresholds for 4- to 7-year-old children were comparable with those of adults at ∼0.5 arcmin. To test whether infant immaturity was due to spatial limitations or insensitivity to interocular correlation, highly suprathreshold horizontal and vertical disparities were presented in alternate regions of the display, and the interocular correlation of the interdigitated regions was varied from 0% to 100%. This manipulation regulated the availability of coarse-scale relative disparity cues. Adult and infant responses both increased with increasing interocular correlation by similar magnitudes, but adult responses increased much more for horizontal disparities, further evidence for qualitatively immature stereopsis based on relative disparity at 4-6 months of age.SIGNIFICANCE STATEMENT Stereopsis, our ability to sense depth from horizontal image disparity, is among the finest spatial discriminations made by the primate visual system. Fine stereoscopic depth discriminations depend critically on comparisons of disparity relationships in the image that are supported by relative disparity cues rather than the estimation of single, absolute disparities. Very young human and macaque infants are sensitive to absolute disparity, but no previous study has specifically studied the development of relative disparity sensitivity, a hallmark feature of adult stereopsis. Here, using high-density EEG recordings, we show that 4- to 6-month-old infants display both quantitative and qualitative response immaturities for relative disparity information. Relative disparity responses are adult-like no later than 4-7 years of age.