Unmixing binocular signals

Binocular viewing geometry shapes the neural representation of the dynamic three-dimensional environment

Sensory signals give rise to patterns of neural activity, which the brain uses to infer properties of the environment. For the visual system, considerable work has focused on the representation of frontoparallel stimulus features and binocular disparities. However, inferring the properties of the physical environment from retinal stimulation is a distinct and more challenging computational problem-this is what the brain must actually accomplish to support perception and action. Here we develop a computational model that incorporates projective geometry, mapping the three-dimensional (3D) environment onto the two retinae. We demonstrate that this mapping fundamentally shapes the tuning of cortical neurons and corresponding aspects of perception. For 3D motion, the model explains the strikingly non-canonical tuning present in existing electrophysiological data and distinctive patterns of perceptual errors evident in human behavior. Decoding the world from cortical activity is strongly affected by the geometry that links the environment to the sensory epithelium.

Eye-specific pattern-motion signals support the perception of three-dimensional motion

An object moving through three-dimensional (3D) space typically yields different patterns of velocities in each eye. For an interocular velocity difference cue to be used, some instances of real 3D motion in the environment (e.g., when a moving object is partially occluded) would require an interocular velocity difference computation that operates on motion signals that are not only monocular (or eye specific) but also depend on each eye's two-dimensional (2D) direction being estimated over regions larger than the size of V1 receptive fields (i.e., global pattern motion). We investigated this possibility using 3D motion aftereffects (MAEs) with stimuli comprising many small, drifting Gabor elements. Conventional frontoparallel (2D) MAEs were local—highly sensitive to the test elements being in the same locations as the adaptor (Experiment 1). In contrast, 3D MAEs were robust to the test elements being in different retinal locations than the adaptor, indicating that 3D motion processing involves relatively global spatial pooling of motion signals (Experiment 2). The 3D MAEs were strong even when the local elements were in unmatched locations across the two eyes during adaptation, as well as when the adapting stimulus elements were randomly oriented, and specified global motion via the intersection of constraints (Experiment 3). These results bolster the notion of eye-specific computation of 2D pattern motion (involving global pooling of local, eye-specific motion signals) for the purpose of computing 3D motion, and highlight the idea that classically “late” computations such as pattern motion can be done in a manner that retains information about the eye of origin.

Neural representation for object recognition in inferotemporal cortex

We suggest that population representation of objects in inferotemporal cortex lie on a continuum between a purely structural, parts-based description and a purely holistic description. The intrinsic dimensionality of object representation is estimated to be around 100, perhaps with lower dimensionalities for object representations more toward the holistic end of the spectrum. Cognitive knowledge in the form of semantic information and task information feed back to inferotemporal cortex from perirhinal and prefrontal cortex respectively, providing high-level multimodal-based expectations that assist in the interpretation of object stimuli. Integration of object information across eye movements may also contribute to object recognition through a process of active vision.

Lustrous material appearances: Internal and external constraints on triggering conditions for binocular lustre

Lustrous surface appearances can be elicited by simple image configurations with no texture or specular highlights, as most prominently illustrated by Helmholtz' demonstration of stereoscopic lustre. Three types of explanatory framework have been proposed for stereoscopic lustre, which attribute the phenomenon to a binocular luminance conflict, an internalised physical regularity (Helmholtz), or to a disentangling of "essential" and "accidental" attributes in surface representations (Hering). In order to investigate these frameworks, we used haploscopically fused half-images of centre-surround configurations in which the luminances of the test patch were dynamically modulated. Experiment 1 shows that stereoscopic lustre is not specifically tied to situations of a luminance conflict between the eyes. Experiment 2 identifies a novel aspect in the binocular temporal dynamics that provides a physical basis for lustrous appearances, namely the occurrence of a temporal luminance counter-modulation between the eyes. This feature sheds some light on the internal principles underlying a disentangling of "accidental" and "essential" surface attributes. Experiment 3 reveals an asymmetry between a light and a dark reference level for the counter-modulations. This finding again suggests an interpretation in terms of an internalised physical regularity with respect to the dynamics of perceiving illuminated surfaces.

Multiplexing in the primate motion pathway

This article begins by reviewing recent work on 3D motion processing in the primate visual system. Some of these results suggest that 3D motion signals may be processed in the same circuitry already known to compute 2D motion signals. Such "multiplexing" has implications for the study of visual cortical circuits and neural signals. A more explicit appreciation of multiplexing--and the computations required for demultiplexing--may enrich the study of the visual system by emphasizing the importance of a structured and balanced "encoding/decoding" framework. In addition to providing a fresh perspective on how successive stages of visual processing might be approached, multiplexing also raises caveats about the value of "neural correlates" for understanding neural computation.

Binocular retinal image differences influence eye-position signals for perceived visual direction

Correctly perceiving the direction of a visible object with respect to one's self (egocentric visual direction) requires that information about the location of the image on the retina (oculocentric visual direction) be combined with signals about the position of the eyes in the head. The Wells-Hering laws that govern the perception of visual direction and modern restatements of these laws assume implicitly that retinal and eye-position information are independent of one another. By measuring observers' manual pointing responses to targets in different horizontal locations, we show that retinal and eye-position information are not treated independently in the brain. In particular, decreasing the relative visibility of one eye's retinal image reduces the strength of the eye-position signal associated with that eye. The results can be accounted for by interactions between eye-specific retinal and eye-position signals at a common neural location.