The motor side of depth vision

What's special about horizontal disparity

Horizontal disparity has been recognized as the primary signal driving stereoscopic depth since the invention of the stereoscope in the 1830s. It has a unique status in our understanding of binocular vision. The direction of offset of the eyes gives the disparities of corresponding image point locations across the two retinas a strong horizontal bias. Beyond the retina, other factors give shape to the effective disparity direction used by visual mechanisms. The influence of orientation is examined here. I argue that horizontal disparity is an inflection point along a continuum of effective directions, and its role in stereo vision can be reinterpreted. The pointwise geometric justification for its special status neglects the oriented structural elements of spatial vision, its physiological support is equivocal, and psychophysical support of its special status may partially reflect biased stimulus sampling. The literature shows that horizontal disparity plays no particular role in the processing of one-dimensional stimuli, a reflection of the stereo aperture problem. The resulting depth is non-veridical, even non-transitive. Although one-dimensional components contribute to the stereo depth of visual objects generally, two-dimensional stimuli appear not to inherit the aperture problem. However, a look at the two-dimensional stimuli that predominate in experimental studies shows regularities in orientation that give a new perspective on horizontal disparity.

Hess Lancaster Screen Test with Eye Tracker: An Objective Method for the Measurement of Binocular Gaze Direction

Background: To objectively measure with an eye tracker both eye movement conjugacy and gaze direction in different positions when performing the Hess Lancaster Screen Test (HLST) in a sample of control subjects without binocular dysfunction and compare the results with those of the traditional subjective HLST performance. Methods: The sample was selected avoiding subjects who suffered from suppression of one of the two eyes, visual acuity less than 20/25 on the Snellen chart in each eye, strabismus, or any symptom of binocular dysfunction that could alter the results. While performing the HLST, the examiner wrote down each of the points on a template in a traditional way while the eye tracker (Tobii Pro Fusion, Tobii AB, Danderyd, Sweden), placed in front of the subject, took objective measurements of the position of both eyes at each point. Of the 29 subjects recruited in this study, 13 subjects between 18 and 27 years old underwent the complete optometric examination and the HLST wearing anaglyph glasses; meanwhile, 16 people were excluded because of binocular or accommodative dysfunctions or because they didn’t give reliable eye-tracking results. Additionally, a specific program called Etracker Parse Video (University of Zaragoza, Zaragoza, Spain) was developed to analyse the prismatic deviation between both eyes at each evaluated point. Results: Similar horizontal prismatic deviations of visual axes were obtained in the different gaze positions with the Maddox rod, the manually annotated HLST, and the eye-tracker measurement. Variations were found in the magnitude of the deviation between methods but not in the direction. On the other hand, vertical deviations were more difficult for the examiner to detect and quantify, especially those with small magnitudes; more exact values were obtained when measuring objectively with the eye tracker. Conclusion: The HLST is very useful and allows the amount of heterophoria or heterotropia to be recorded in the patient’s medical record in all the main gaze positions. This test is complementary; by itself, it is not diagnostic and does not replace a complete examination of binocular vision. The eye tracker is an objective method with which we can evaluate the HLST in patients with no binocular problems, obtaining more accurate results than when it is performed in the traditional manner.

Stereopsis without correspondence

Stereopsis has traditionally been considered a complex visual ability, restricted to large-brained animals. The discovery in the 1980s that insects, too, have stereopsis, therefore, challenged theories of stereopsis. How can such simple brains see in three dimensions? A likely answer is that insect stereopsis has evolved to produce simple behaviour, such as orienting towards the closer of two objects or triggering a strike when prey comes within range. Scientific thinking about stereopsis has been unduly anthropomorphic, for example assuming that stereopsis must require binocular fusion or a solution of the stereo correspondence problem. In fact, useful behaviour can be produced with very basic stereoscopic algorithms which make no attempt to achieve fusion or correspondence, or to produce even a coarse map of depth across the visual field. This may explain why some aspects of insect stereopsis seem poorly designed from an engineering point of view: for example, paying no attention to whether interocular contrast or velocities match. Such algorithms demonstrably work well enough in practice for their species, and may prove useful in particular autonomous applications. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Effector-dependent stochastic reference frame transformations alter decision-making

Psychophysical, motor control, and modeling studies have revealed that sensorimotor reference frame transformations (RFTs) add variability to transformed signals. For perceptual decision-making, this phenomenon could decrease the fidelity of a decision signal's representation or alternatively improve its processing through stochastic facilitation. We investigated these two hypotheses under various sensorimotor RFT constraints. Participants performed a time-limited, forced-choice motion discrimination task under eight combinations of head roll and/or stimulus rotation while responding either with a saccade or button press. This paradigm, together with the use of a decision model, allowed us to parameterize and correlate perceptual decision behavior with eye-, head-, and shoulder-centered sensory and motor reference frames. Misalignments between sensory and motor reference frames produced systematic changes in reaction time and response accuracy. For some conditions, these changes were consistent with a degradation of motion evidence commensurate with a decrease in stimulus strength in our model framework. Differences in participant performance were explained by a continuum of eye–head–shoulder representations of accumulated motion evidence, with an eye-centered bias during saccades and a shoulder-centered bias during button presses. In addition, we observed evidence for stochastic facilitation during head-rolled conditions (i.e., head roll resulted in faster, more accurate decisions in oblique motion for a given stimulus–response misalignment). We show that perceptual decision-making and stochastic RFTs are inseparable within the present context. We show that by simply rolling one's head, perceptual decision-making is altered in a way that is predicted by stochastic RFTs.

Eye-head-hand coordination during visually guided reaches in head-unrestrained macaques

Nonhuman primates have been used extensively to study eye-head coordination and eye-hand coordination, but the combination-eye-head-hand coordination-has not been studied. Our goal was to determine whether reaching influences eye-head coordination (and vice versa) in rhesus macaques. Eye, head, and hand motion were recorded in two animals with search coil and touch screen technology, respectively. Animals were seated in a customized "chair" that allowed unencumbered head motion and reaching in depth. In the reach condition, animals were trained to touch a central LED at waist level while maintaining central gaze and were then rewarded if they touched a target appearing at 1 of 15 locations in a 40° × 20° (visual angle) array. In other variants, initial hand or gaze position was varied in the horizontal plane. In similar control tasks, animals were rewarded for gaze accuracy in the absence of reach. In the Reach task, animals made eye-head gaze shifts toward the target followed by reaches that were accompanied by prolonged head motion toward the target. This resulted in significantly higher head velocities and amplitudes (and lower eye-in-head ranges) compared with the gaze control condition. Gaze shifts had shorter latencies and higher velocities and were more precise, despite the lack of gaze reward. Initial hand position did not influence gaze, but initial gaze position influenced reach latency. These results suggest that eye-head coordination is optimized for visually guided reach, first by quickly and accurately placing gaze at the target to guide reach transport and then by centering the eyes in the head, likely to improve depth vision as the hand approaches the target.NEW & NOTEWORTHY Eye-head and eye-hand coordination have been studied in nonhuman primates but not the combination of all three effectors. Here we examined the timing and kinematics of eye-head-hand coordination in rhesus macaques during a simple reach-to-touch task. Our most novel finding was that (compared with hand-restrained gaze shifts) reaching produced prolonged, increased head rotation toward the target, tending to center the binocular field of view on the target/hand.

Autonomous Development of Active Binocular and Motion Vision Through Active Efficient Coding

We present a model for the autonomous and simultaneous learning of active binocular and motion vision. The model is based on the Active Efficient Coding (AEC) framework, a recent generalization of classic efficient coding theories to active perception. The model learns how to efficiently encode the incoming visual signals generated by an object moving in 3-D through sparse coding. Simultaneously, it learns how to produce eye movements that further improve the efficiency of the sensory coding. This learning is driven by an intrinsic motivation to maximize the system's coding efficiency. We test our approach on the humanoid robot iCub using simulations. The model demonstrates self-calibration of accurate object fixation and tracking of moving objects. Our results show that the model keeps improving until it hits physical constraints such as camera or motor resolution, or limits on its internal coding capacity. Furthermore, we show that the emerging sensory tuning properties are in line with results on disparity, motion, and motion-in-depth tuning in the visual cortex of mammals. The model suggests that vergence and tracking eye movements can be viewed as fundamentally having the same objective of maximizing the coding efficiency of the visual system and that they can be learned and calibrated jointly through AEC.

An alignment maximization method for the kinematics of the eye and eye-head fixations

The orientation of human eyes is uniquely defined with respect to their gaze direction, known as Donders' law. Further, the manner in which the eyes follow Donders' law varies as a function of the situation. When the head is stationary, the Donders' surfaces are flat planes but they tilt when eye fixation distance changes. These planes also shift and rotate when head orientation changes with respect to the direction of gravito-inertial acceleration. When the head is free to rotate, the Donders' surfaces are twisted. In this paper, we present a systematic method to analyze the kinematics of the eye under different gaze situations utilizing the measurement of alignment between various coordinate frames. Kinematic equations are presented for various eye movements ranging from simple head-fixed monocular shifts of eye gaze to complex eye-head shifts of gaze. At each stage, we show that simulated eye orientations that derived from our equations are able to capture the variations of Donders' surfaces and they are comparable with experimental results in the literature. The final equations we propose provide the unified kinematics of head-upright far gaze, head-upright binocular fixation, head static tilted monocular gaze and head-free monocular gaze.

Functional anatomy of human extraocular muscles during fusional divergence

We employed magnetic resonance imaging to quantify human extraocular muscle contractility during centered target fusion and fusional divergence repeated with each eye viewing monocularly at 20 cm through 8Δ and at 400 cm through 4Δ base in prism. Contractility, indicated by posterior partial volume (PPV) change, was analyzed in transverse rectus and in medial and lateral superior oblique (SO) muscle compartments and by cross-sectional area change in the inferior oblique (IO). At 20 cm, 3.1 ± 0.5° (SE) diverging eye abduction in 10 subjects was associated with 4.2 ± 1.5% whole lateral rectus (LR) PPV increase ( P < 0.05) and 1.7 ± 1.1% overall medial rectus (MR) PPV decrease attributable to 3.1 ± 1.8% reduction in the superior compartment ( P < 0.025), without change in its inferior compartment or in muscles of the aligned eye. At 400 cm, 2.2 ± 0.5° diverging eye abduction in nine subjects was associated with 6.1 ± 1.3% whole LR PPV increase ( P < 10^-5) but no change in MR, with compartmentally similar relaxation in the LR and MR of the aligned eye. Unlike convergence, there were no IO or SO contractile changes for divergence to either target nor any change in rectus pulley positions. Results confirm and extend to proximal divergence the unique role of the superior MR compartment, yet no MR role for far divergence. Corelaxation of aligned eye LR and MR combined with failure of MR relaxation during divergence is consistent with the limited behavioral range of divergence. NEW & NOTEWORTHY Magnetic resonance imaging shows that the lateral rectus muscle must overcome continued contraction by its opponent the medial rectus when humans diverge their visual axes to achieve single, binocular vision. While the upper but not lower compartment of the medial rectus assists by relaxing for near targets, it does not do so when targets are far away. This behavior violates Sherrington's law of reciprocal action of antagonists and conventional assumptions about the ocular motor system.

A New Dyslexia Reading Method and Visual Correction Position Method

Pediatricians and educators may interact daily with several dyslexic patients or students. One dyslexic author accidently developed a personal, effective, corrective reading method. Its effectiveness was evaluated in 3 schools. One school utilized 8 demonstration special education students. Over 3 months, one student grew one third year, 3 grew 1 year, and 4 grew 2 years. In another school, 6 sixth-, seventh-, and eighth-grade classroom teachers followed 45 treated dyslexic students. They all excelled and progressed beyond their classroom peers in 4 months. Using cyclovergence upper gaze, dyslexic reading problems disappeared at one of the Positional Reading Arc positions of 30°, 60°, 90°, 120°, or 150° for 10 dyslexics. Positional Reading Arc on 112 students of the second through eighth grades showed words read per minute, reading errors, and comprehension improved. Dyslexia was visually corrected by use of a new reading method and Positional Reading Arc positions.

Effect of depth information on multiple-object tracking in three dimensions: A probabilistic perspective

Many daily situations require us to track multiple objects and people. This ability has traditionally been investigated in observers tracking objects in a plane. This simplification of reality does not address how observers track objects when targets move in three dimensions. Here, we study how observers track multiple objects in 2D and 3D while manipulating the average speed of the objects and the average distance between them. We show that performance declines as speed increases and distance decreases and that overall tracking accuracy is always higher in 3D than in 2D. The effects of distance and dimensionality interact to produce a more than additive improvement in performance during tracking in 3D compared to 2D. We propose an ideal observer model that uses the object dynamics and noisy observations to track the objects. This model provides a good fit to the data and explains the key findings of our experiment as originating from improved inference of object identity by adding the depth dimension.

Many daily life situations require us to track objects that are in motion. In the laboratory, this multiple object tracking problem is classically studied with objects moving on a two-dimensional screen, but in the real world objects typically move in three dimensions. Here we show that, despite the complexity of seeing in depth, observers track multiple objects better when they move in 3D than 2D. A probabilistic inference model explains this by showing that the association of noisy visual signals to the objects that caused them is less ambiguous when depth cues are available. This highlights the role that depth cues play in our everyday ability to track objects.

A dataset of stereoscopic images and ground-truth disparity mimicking human fixations in peripersonal space

Binocular stereopsis is the ability of a visual system, belonging to a live being or a machine, to interpret the different visual information deriving from two eyes/cameras for depth perception. From this perspective, the ground-truth information about three-dimensional visual space, which is hardly available, is an ideal tool both for evaluating human performance and for benchmarking machine vision algorithms. In the present work, we implemented a rendering methodology in which the camera pose mimics realistic eye pose for a fixating observer, thus including convergent eye geometry and cyclotorsion. The virtual environment we developed relies on highly accurate 3D virtual models, and its full controllability allows us to obtain the stereoscopic pairs together with the ground-truth depth and camera pose information. We thus created a stereoscopic dataset: GENUA PESTO-GENoa hUman Active fixation database: PEripersonal space STereoscopic images and grOund truth disparity. The dataset aims to provide a unified framework useful for a number of problems relevant to human and computer vision, from scene exploration and eye movement studies to 3D scene reconstruction.

The Active Side of Stereopsis: Fixation Strategy and Adaptation to Natural Environments

Depth perception in near viewing strongly relies on the interpretation of binocular retinal disparity to obtain stereopsis. Statistical regularities of retinal disparities have been claimed to greatly impact on the neural mechanisms that underlie binocular vision, both to facilitate perceptual decisions and to reduce computational load. In this paper, we designed a novel and unconventional approach in order to assess the role of fixation strategy in conditioning the statistics of retinal disparity. We integrated accurate realistic three-dimensional models of natural scenes with binocular eye movement recording, to obtain accurate ground-truth statistics of retinal disparity experienced by a subject in near viewing. Our results evidence how the organization of human binocular visual system is finely adapted to the disparity statistics characterizing actual fixations, thus revealing a novel role of the active fixation strategy over the binocular visual functionality. This suggests an ecological explanation for the intrinsic preference of stereopsis for a close central object surrounded by a far background, as an early binocular aspect of the figure-ground segregation process.

A three dimensional view of stereopsis in dentistry

Stereopsis and its role in dental practice has been a topic of debate in recent editions of this Journal. These discussions are particularly timely as they come at a point when virtual reality simulators are becoming increasingly popular in the education of tomorrow's dentists. The aim of this article is to discuss the lack of robust empirical evidence to ascertain the relationship (if any) between stereopsis and dentistry and to build a case for the need for further research to build a strong evidence base on the topic.

Three-dimensional ocular kinematics underlying binocular single vision

We have analyzed the binocular coordination of the eyes during far-to-near refixation saccades based on the evaluation of distance ratios and angular directions of the projected target images relative to the eyes' rotation centers. By defining the geometric point of binocular single vision, called Helmholtz point, we found that disparities during fixations of targets at near distances were limited in the subject's three-dimensional visual field to the vertical and forward directions. These disparities collapsed to simple vertical disparities in the projective binocular image plane. Subjects were able to perfectly fuse the vertically disparate target images with respect to the projected Helmholtz point of single binocular vision, independent of the particular location relative to the horizontal plane of regard. Target image fusion was achieved by binocular torsion combined with corrective modulations of the differential half-vergence angles of the eyes in the horizontal plane. Our findings support the notion that oculomotor control combines vergence in the horizontal plane of regard with active torsion in the frontal plane to achieve fusion of the dichoptic binocular target images.

The distinctive vertical heterophoria of dyslexics

In this study, we looked for the presence of vertical heterophoria (VH) in 42 dyslexic children (22 males and 20 females) aged 118.5±12.9 months who were compared with a control group of 22 nondyslexic children (eleven males and eleven females) aged 112±9.8 months. Dyslexics presented a low-level (always <1 prism diopter) VH combined with torsion. This oculomotor feature clearly separates the dyslexic group from the normal readers group. It is independent of the type of dyslexia. The essential feature of this VH is a lability that appears during specific stimulation of sensory receptors involved in postural regulation. This lability is demonstrated using a vertical Maddox test conducted under very specific conditions in which postural sensors are successively stimulated in a predetermined order. A quantitative variation in this VH may be seen during the Bielchowsky Head Tilt Test, which reveals hypertonia of the lower or upper oblique muscles. Vertical orthophoria can be achieved by placing low-power prisms asymmetrically within the direction of action of the superior or inferior oblique muscles. The selection of power and axis is not only guided by elements of the eye examination but also from observation of postural muscle tone. All these elements suggest that the VH could be of postural origin and somehow related to the vertical action of the oblique muscles. VH and torsion are not harmful per se. There is no statistical relationship between their level and the various parameters used to assess the reading skills of dyslexic children. VH and torsion could be a clinical marker of global proprioceptive dysfunction responsible for high-level multisensory disturbances secondary to poor spatial localization of visual and auditory information. This dysfunction might also explain the motor disorders concomitant to dyslexia.

A kinematic model for 3-D head-free gaze-shifts

Rotations of the line of sight are mainly implemented by coordinated motion of the eyes and head. Here, we propose a model for the kinematics of three-dimensional (3-D) head-unrestrained gaze-shifts. The model was designed to account for major principles in the known behavior, such as gaze accuracy, spatiotemporal coordination of saccades with vestibulo-ocular reflex (VOR), relative eye and head contributions, the non-commutativity of rotations, and Listing's and Fick constraints for the eyes and head, respectively. The internal design of the model was inspired by known and hypothesized elements of gaze control physiology. Inputs included retinocentric location of the visual target and internal representations of initial 3-D eye and head orientation, whereas outputs were 3-D displacements of eye relative to the head and head relative to shoulder. Internal transformations decomposed the 2-D gaze command into 3-D eye and head commands with the use of three coordinated circuits: (1) a saccade generator, (2) a head rotation generator, (3) a VOR predictor. Simulations illustrate that the model can implement: (1) the correct 3-D reference frame transformations to generate accurate gaze shifts (despite variability in other parameters), (2) the experimentally verified constraints on static eye and head orientations during fixation, and (3) the experimentally observed 3-D trajectories of eye and head motion during gaze-shifts. We then use this model to simulate how 2-D eye-head coordination strategies interact with 3-D constraints to influence 3-D orientations of the eye-in-space, and the implications of this for spatial vision.

The kinematics of far-near re-fixation saccades

We have analyzed the three-dimensional spatiotemporal characteristics of saccadic refixations between far and near targets in three behaviorally trained rhesus monkeys. The kinematics underlying these rapid eye movements can be accurately described by rotations of the eyes in four different planes, namely, first disconjugate rotations in the horizontal plane of regard converging the eyes toward the near target, followed by rotations in each eye's vertical direction plane, and finally, disconjugate rotations in a common frontoparallel plane. This compounded rotation of the eye was underlying an initially fast-rising variable torsion that typically overshot the final torsion, which the eyes attained at the time of target acquisition. The torsion consisted of a coarse, widely varying component of opposite polarity in the two eyes, which contained a more robust, much smaller modulation that sharply increased toward the end of saccades. The reorientation of the eyes in torsion depended on each eye's azimuth, elevation, and target distance. We conclude that refixation saccades are generated by motor commands that control ocular torsion in concert with the saccade generator, which operates in Donders-Listing kinematics underlying Listing's law.

Magnetic resonance imaging demonstrates compartmental muscle mechanisms of human vertical fusional vergence

Vertical fusional vergence (VFV) normally compensates for slight vertical heterophorias. We employed magnetic resonance imaging to clarify extraocular muscle contributions to VFV induced by monocular two-prism diopter (1.15°) base-up prism in 14 normal adults. Fusion during prism viewing requires monocular infraduction. Scans were repeated without prism, and with prism shifted contralaterally. Contractility indicated by morphometric indexes was separately analyzed in medial and lateral vertical rectus and superior oblique (SO) putative compartments, and superior and inferior horizontal rectus extraocular muscle putative compartments, but in the whole inferior oblique (IO). Images confirmed appropriate VFV that was implemented by the inferior rectus (IR) medial compartment contracting ipsilateral and relaxing contralateral to prism. There was no significant contractility in the IR lateral compartment. The superior but not inferior lateral rectus (LR) compartment contracted significantly in the prism viewing eye, but not contralateral to prism. The IO contracted ipsilateral but not contralateral to the prism. In the infraducting eye, the SO medial compartment relaxed significantly, while the lateral compartment was unchanged; contralateral to prism, the SO lateral compartment contracted, while the medial compartment was unchanged. There was no contractility in the superior or medial rectus muscles in either eye. There was no globe retraction. We conclude that the vertical component of VFV is primarily implemented by IR medial compartment contraction. Since appropriate vertical rotation is not directly implemented, or is opposed, by associated differential LR and SO compartmental activity, and IO contraction, these actions probably implement a torsional component of VFV.

Stereo vision and strabismus

Binocular stereopsis, or stereo vision, is the ability to derive information about how far away objects are, based solely on the relative positions of the object in the two eyes. It depends on both sensory and motor abilities. In this review, I briefly outline some of the neuronal mechanisms supporting stereo vision, and discuss how these are disrupted in strabismus. I explain, in some detail, current methods of assessing stereo vision and their pros and cons. Finally, I review the evidence supporting the clinical importance of such measurements.

Effects of Light, Rotation, and Texture on Errors in the Estimated Length of a Three-Dimensional Object in a Two-Dimensional Display

This study investigated the effects of light type (spot, directional, and point), forward-rotated stimulus angle (20°, 45°, and 70°), and surface texture (wood, metal, transparent glass, dark brown leather, and translucent plastic) on errors in the estimated vertical diameter of a three-dimensional object. Twenty female and 12 male Taiwanese college students, ages 18–22 years, participated in the current study. Participants were asked to judge the vertical diameter of an elliptical hole within a square plate that was displayed on an LCD monitor. Analysis indicated that the errors in estimated vertical diameter for rotated angles of 20° and 45° were greater than were those for 70°. Additionally, the error of estimated vertical diameter for a rotated angle of 45° was less than was that for 20°. The surface texture also affected errors in estimated vertical diameter: transparent glass was associated with more errors than were wood and metal textures. However, light type had no effect on errors in estimated vertical diameter. These results have implications for the graphics-based design of interfaces.

Adaptive Optimal Control Without Weight Transport

Many neural control systems are at least roughly optimized, but how is optimal control learned? There are algorithms for this purpose, but in their current forms, they are not suited for biological neural networks because they rely on a type of communication that is not available in the brain, namely, weight transport—transmitting the strengths, or “weights,” of individual synapses to other synapses and neurons. Here we show how optimal control can be learned without weight transport. Our method involves a set of simple mechanisms that can compensate for the absence of weight transport in the brain and so may be useful for neural computation generally.

The nonlinearity of passive extraocular muscles

Passive extraocular muscles (EOMs), like most biological tissues, are hyperelastic, that is, their stiffness increases as they are stretched. It has always been assumed, and in a few occasions argued, that this is their only nonlinearity and that it can be ignored in central gaze. However, using novel measurement techniques in anesthetized paralyzed monkeys, we have recently demonstrated that EOMs are characterized by another prominent nonlinearity: the forces induced by sequences of stretches do not sum. Thus, superposition, a central tenet of linear and quasi-linear models, does not hold in passive EOMs. Here, we outline the implications of this finding, especially in light of the common assumption that it is easier for the brain to control a linear than a nonlinear plant. We argue against this common belief: the specific nonlinearity of passive EOMs may actually make it easier for the brain to control the plant than if muscles were linear.

Stereo correspondence is optimized for large viewing distances

Stereo '3D' vision depends on correctly matching up the differing images of objects seen by our two eyes. But vertical disparity between the retinal images changes with binocular eye posture, reflecting for example the different convergence angles required for different viewing distances. Thus, stereo correspondence must either dynamically adapt to take account of changes in viewing distance, or be hard-wired to perform best at one particular viewing distance. Here, using psychophysical experiments, we show for the first time that human stereo correspondence does not adapt to changes in physical viewing distance. We examine performance on a stereo correspondence task at a short viewing distance (30 cm) and show that performance is improved when we simulate the disparity pattern for viewing infinity, even though these disparities are impossible at the physical viewing distance. We estimate the vertical extent of the retinally fixed 'search zones' as < 0.6° at 14° eccentricity, suggesting that most V1 neurons must be tuned to near-zero vertical disparity. We also show that performance on our stereo task at 14° eccentricity is affected by the pattern of vertical disparity beyond 20° eccentricity, even though this is irrelevant to the task. Performance is best when vertical disparities within and beyond 20° eccentricity both indicate the same convergence angle (even if not the physical angle), than when the pattern of vertical disparity across the visual field is globally inconsistent with any single convergence angle. This novel effect of the periphery may indicate cooperative interactions between disparity-selective neurons activated by the same eye postures.

Visuomotor Velocity Transformations for Smooth Pursuit Eye Movements

Smooth pursuit eye movements are driven by retinal motion signals. These retinal motion signals are converted into motor commands that obey Listing's law (i.e., no accumulation of ocular torsion). The fact that smooth pursuit follows Listing's law is often taken as evidence that no explicit reference frame transformation between the retinal velocity input and the head-centered motor command is required. Such eye-position-dependent reference frame transformations between eye- and head-centered coordinates have been well-described for saccades to static targets. Here we suggest that such an eye (and head)-position-dependent reference frame transformation is also required for target motion (i.e., velocity) driving smooth pursuit eye movements. Therefore we tested smooth pursuit initiation under different three-dimensional eye positions and compared human performance to model simulations. We specifically tested if the ocular rotation axis changed with vertical eye position, if the misalignment of the spatial and retinal axes during oblique fixations was taken into account, and if ocular torsion (due to head roll) was compensated for. If no eye-position-dependent velocity transformation was used, the pursuit initiation should follow the retinal direction, independently of eye position; in contrast, a correct visuomotor velocity transformation would result in spatially correct pursuit initiation. Overall subjects accounted for all three components of the visuomotor velocity transformation, but we did observe differences in the compensatory gains between individual subjects. We concluded that the brain does perform a visuomotor velocity transformation but that this transformation was prone to noise and inaccuracies of the internal model.

Latitude and longitude vertical disparities

The literature on vertical disparity is complicated by the fact that several different definitions of the term "vertical disparity" are in common use, often without a clear statement about which is intended or a widespread appreciation of the properties of the different definitions. Here, we examine two definitions of retinal vertical disparity: elevation-latitude and elevation-longitude disparities. Near the fixation point, these definitions become equivalent, but in general, they have quite different dependences on object distance and binocular eye posture, which have not previously been spelt out. We present analytical approximations for each type of vertical disparity, valid for more general conditions than previous derivations in the literature: we do not restrict ourselves to objects near the fixation point or near the plane of regard, and we allow for non-zero torsion, cyclovergence, and vertical misalignments of the eyes. We use these expressions to derive estimates of the latitude and longitude vertical disparities expected at each point in the visual field, averaged over all natural viewing. Finally, we present analytical expressions showing how binocular eye position-gaze direction, convergence, torsion, cyclovergence, and vertical misalignment-can be derived from the vertical disparity field and its derivatives at the fovea.

Tolerance of stereopsis to conjunctive cyclorotation

Stereopsis is largely unperturbed by various types of eye, head, and target movements. Here, we used a simple setup to investigate the limits of a previously untested type of stimulus motion on stereoscopic depth perception. Clockwise and counterclockwise rotations of an autostereogram were used to describe the limits of stereopsis to roll-tilt. The result showed intact depth perception with stimulus rotation up to approximately 12 degrees, regardless of rotation direction and viewing distance, indicating a tolerant mechanism for stereoscopic processing by the human neural system.

Binocular energy responses to natural images

The binocular energy model provides a good description of the first stages of cortical binocular processing. Three important determinants of the responses of neurons under this model are the disparity of a stimulus, its spatial variation in disparity and its second-order luminance statistics. The influence of the latter two factors on the disparity tuning of the energy model were investigated. While each can have a significant effect on the energy response, neither presents a significant challenge when one considers the range of variation expected in natural images. The response of the energy model to natural binocular images was also investigated. The strongest responses were found for model neurons tuned to small disparities. This trend was more evident for vertical than for horizontal disparity, and flattened rapidly as image eccentricity increased. These results are predicted on the basis of simple geometrical considerations, and are reflected in both physiological and psychophysical measures of the disparity tuning of the visual system.

Depth cues, rather than perceived depth, govern vergence

We studied the influence of perceived surface orientation on vergence accompanying a saccade while viewing an ambiguous stimulus. We used the slant rivalry stimulus, in which perspective foreshortening and disparity specified opposite surface orientations. This rivalrous configuration induces alternations of perceived surface orientation, while the slant cues remain constant. Subjects were able to voluntarily control their perceptual state while viewing the ambiguous stimulus. They were asked to make a saccade across the perceived slanted surface. Our data show that vergence responses closely approximated the vergence response predicted by the disparity cue, irrespective of voluntarily controlled perceived orientation. However, comparing the data obtained while viewing the ambiguous stimulus with data from an unambiguous stimulus condition (when disparity and perspective specified similar surface orientations) revealed an effect of perspective cues on vergence. Collectively our results show that depth cues rather than perceived depth govern vergence.

Illusory depth induced by binocular torsional misalignment

This study reports a new depth illusion in which a static flat pattern appears stratified stereoscopically when viewed binocularly with an elevated gaze. Three psychophysical experiments measured perceived relative depth and fixational cyclodisparity (a rotation of one eye's view relative to the other eye's view about the line of sight) when flat patterns drawn with solid or dashed curved lines were fixated at various levels of gaze elevation. Experiments 1 and 2 showed that the patterns drawn with solid lines produced illusory depth only at large gaze elevations (downward and upward). Experiment 3 showed that the magnitude of the illusory depth was correlated with that of fixational cyclodisparity. These results suggest that the illusory depth originates in the binocular torsional misalignment generated by gaze elevation.

Multi-area visuotopic map complexes in macaque striate and extra-striate cortex

We propose that a simple, closed-form mathematical expression-the Wedge-Dipole mapping-provides a concise approximation to the full-field, two-dimensional topographic structure of macaque V1, V2, and V3. A single map function, which we term a map complex, acts as a simultaneous descriptor of all three areas. Quantitative estimation of the Wedge-Dipole parameters is provided via 2DG data of central-field V1 topography and a publicly available data set of full-field macaque V1 and V2 topography. Good quantitative agreement is obtained between the data and the model presented here. The increasing importance of fMRI-based brain imaging motivates the development of more sophisticated two-dimensional models of cortical visuotopy, in contrast to the one-dimensional approximations that have been in common use. One reason is that topography has traditionally supplied an important aspect of "ground truth," or validation, for brain imaging, suggesting that further development of high-resolution fMRI will be facilitated by this data analysis. In addition, several important insights into the nature of cortical topography follow from this work. The presence of anisotropy in cortical magnification factor is shown to follow mathematically from the shared boundary conditions at the V1-V2 and V2-V3 borders, and therefore may not causally follow from the existence of columnar systems in these areas, as is widely assumed. An application of the Wedge-Dipole model to localizing aspects of visual processing to specific cortical areas-extending previous work in correlating V1 cortical magnification factor to retinal anatomy or visual psychophysics data-is briefly discussed.

Temporal dynamics of stereo correspondence bi-stability

Periodic stereoscopic stimuli offer multiple viable solutions to the stereo correspondence problem. When viewing such stimuli for prolonged periods of time, observers continually switch their perceptual state between alternative correspondence solutions. We examine the temporal dynamics of this correspondence bi-stability. Participants were presented with an ambiguous stereogram comprised of regularly spaced dots. This stimulus was perceived as a fronto-parallel plane situated either behind or in front of fixation, depending on the achieved correspondence solution. The stimulus was presented continuously for one minute, with participants instructed to report the sign of the perceived depth at the sound of an auditory prompt presented, on average, every 2 s. Inter-ocular contrast and available disparities were varied so as to manipulate preferred correspondence. We find that participants are initially biased to perceive the stimulus as having an uncrossed disparity. Furthermore, we find that following an initial period of change, perceptual preference and perceptual stability (measured as the probability of an observer's percept changing between consecutive responses) remain constant over the presentation period. Finally, we find that manipulations of matching preference affect both the transient preference for, and stability of, one percept over another. Our results suggest two distinct phases of biasing in the correspondence matching process, one early, the other sustained.

Neural bases of stereopsis across visual field of the alert macaque monkey

Left and right retinal images of an object seen by the 2 eyes can occupy slightly disparate horizontal and/or vertical locations. The role of horizontal disparity (HD) in stereoscopic vision is well established, but the functional contribution of vertical disparity (VD) remains unclear. Various psychophysical studies have shown that HD and VD are used differently by the visual system depending on their location in the visual field, whether near the center of gaze or more peripheral. We show this horizontal/vertical distinction at the cellular level in monkey primary visual cortex (area V1). The range of VD encoding is reduced in central but not in the peripheral representation of the visual field. Moreover, neurons respond selectively to particular combinations of both types of disparities depending on the coded orientation as predicted by the disparity energy model. The preferred orientations of neurons near the fovea present a vertical bias that is well suited for stereopsis based on HD selectivity alone. In the periphery, instead, preferred orientations are radially biased, which allows a peripheral detector to convey the same depth signal based on either HD or VD. Such an organization has functional implications in both the perceptual and oculomotor domains.

The coordination of binocular eye movements: vertical and torsional alignment

Precise binocular alignment of the visual axes is of utmost importance for good vision. The fact that so few of us ever experience diplopia is evidence of how well the oculomotor system performs this function in the face of changes due to development, disease and injury. The capacity of the oculomotor system to adapt to visual stimuli that mimic alignment deficits has been extensively explored in laboratory experiments. While the present paper reviews many of those studies, the primary focus is on issues involved in maintaining good vertical and torsional alignment in everyday viewing situations where the parsing of muscle forces may vary for the same horizontal and vertical eye positions due to changes in horizontal vergence and head posture.

Current concepts of mechanical and neural factors in ocular motility

PURPOSE OF REVIEW

The oculomotor periphery was classically regarded as a simple mechanism executing complex behaviors specified explicitly by neural commands. A competing view has emerged that many important aspects of ocular motility are properties of the extraocular muscles and their associated connective tissue pulleys. This review considers current concepts regarding aspects of ocular motility that are mechanically determined versus those that are specified explicitly as innervation.

RECENT FINDINGS

While it was established several years ago that the rectus extraocular muscles have connective tissue pulleys, recent functional imaging and histology has suggested that the rectus pulley array constitutes an inner mechanism, analogous to a gimbal, that is rotated torsionally around the orbital axis by an outer mechanism driven by the oblique extraocular muscles. This arrangement may account mechanically for several commutative aspects of ocular motor control, including Listing's Law, yet permits implementation of non-commutative motility. Recent human behavioral studies, as well as neurophysiology in monkeys, are consistent with implementation of Listing's Law in the oculomotor periphery, rather than centrally.

SUMMARY

Varied evidence now strongly supports the conclusion that Listing's Law and other important ocular kinematics are mechanically determined. This finding implies more limited possibilities for neural adaptation to some ocular motor pathologies, but indicates possibilities for surgical treatments.

Three-dimensional kinematics at the level of the oculomotor plant

Motor systems often require that superfluous degrees of freedom be constrained. For the oculomotor system, a redundancy in the degrees of freedom occurs during visually guided eye movements and is solved by implementing Listing's law and the half-angle rule, kinematic constraints that limit the range of eye positions and angular velocities used by the eyes. These constraints have been attributed either to neurally generated commands or to the physical mechanics of the eye and its surrounding muscles and tissues (i.e., the ocular plant). To directly test whether the ocular plant implements the half-angle rule, critical to the maintenance of Listing's law, we microstimulated the abducens nerve with the eye at different initial vertical eye positions. We report that the electrically evoked eye velocity exhibits the same eye position dependence as seen in visually guided smooth-pursuit eye movements. These results support an important role for the ocular plant in providing a solution to the degrees-of-freedom problem during eye movements.

Transsaccadic integration of visual features in a line intersection task

Transsaccadic integration (TSI) refers to the perceptual integration of visual information collected across separate gaze fixations. Current theories of TSI disagree on whether it relies solely on visual algorithms or also uses extra-retinal signals. We designed a task in which subjects had to rely on internal oculomotor signals to synthesize remembered stimulus features presented within separate fixations. Using a mouse-controlled pointer, subjects estimated the intersection point of two successively presented bars, in the dark, under two conditions: Saccade task (bars viewed in separate fixations) and Fixation task (bars viewed in one fixation). Small, but systematic biases were observed in both intersection tasks, including position-dependent vertical undershoots and order-dependent horizontal biases. However, the magnitude of these errors was statistically indistinguishable in the Saccade and Fixation tasks. Moreover, part of the errors in the Saccade task were dependent on saccade metrics, showing that egocentric oculomotor signals were used to fuse remembered location and orientation features across saccades. We hypothesize that these extra-retinal signals are normally used to reduce the computational load of calculating visual correspondence between fixations. We further hypothesize that TSI may be implemented within dynamically updated recurrent feedback loops that interconnect a common eye-centered map in occipital cortex with both the "dorsal" and "ventral" streams of visual analysis.

Understanding the cortical specialization for horizontal disparity

Because the eyes are displaced horizontally, binocular vision is inherently anisotropic. Recent experimental work has uncovered evidence of this anisotropy in primary visual cortex (V1): neurons respond over a wider range of horizontal than vertical disparity, regardless of their orientation tuning. This probably reflects the horizontally elongated distribution of two-dimensional disparity experienced by the visual system, but it conflicts with all existing models of disparity selectivity, in which the relative response range to vertical and horizontal disparities is determined by the preferred orientation. Potentially, this discrepancy could require us to abandon the widely held view that processing in V1 neurons is initially linear. Here, we show that these new experimental data can be reconciled with an initial linear stage; we present two physiologically plausible ways of extending existing models to achieve this. First, we allow neurons to receive input from multiple binocular subunits with different position disparities (previous models have assumed all subunits have identical position and phase disparity). Then we incorporate a form of divisive normalization, which has successfully explained many response properties of V1 neurons but has not previously been incorporated into a model of disparity selectivity. We show that either of these mechanisms decouples disparity tuning from orientation tuning and discuss how the models could be tested experimentally. This represents the first explanation of how the cortical specialization for horizontal disparity may be achieved.

Control of eye orientation: where does the brain's role end and the muscle's begin?

Our understanding of how the brain controls eye movements has benefited enormously from the comparison of neuronal activity with eye movements and the quantification of these relationships with mathematical models. Although these early studies focused on horizontal and vertical eye movements, recent behavioural and modelling studies have illustrated the importance, but also the complexity, of extending previous conclusions to the problems of controlling eye and head orientation in three dimensions (3-D). An important facet in understanding 3-D eye orientation and movement has been the discovery of mobile, soft-tissue sheaths or 'pulleys' in the orbit which might influence the pulling direction of extraocular muscles. Appropriately placed pulleys could generate the eye-position-dependent tilt of the ocular rotation axes which are characteristic for eye movements which follow Listing's law. Based on such pulley models of the oculomotor plant it has recently been proposed that a simple two-dimensional (2-D) neural controller would be sufficient to generate correct 3-D eye orientation and movement. In contrast to this apparent simplification in oculomotor control, multiple behavioural observations suggest that the visuo-motor transformations, as well as the premotor circuitry for saccades, pursuit eye movements and the vestibulo-ocular reflexes, must include a neural controller which operates in 3-D, even when considering an eye plant with pulleys. This review summarizes the most recent work and ideas on this controversy. In addition, by proposing directly testable hypotheses, we point out that, in analogy to the previously successful steps towards elucidating the neural control of horizontal eye movements, we need a quantitative characterization first of motoneuron and next of premotor neuron properties in 3-D before we can succeed in gaining further insight into the neural control of 3-D motor behaviours.

Changes in ocular torsion position produced by a single visual line rotating around the line of sight––visual “entrainment” of ocular torsion

A large- or full-field visual stimulus slowly rotating around the naso-occipital axis of an observer causes both eyes to tort, and many of the factors controlling this optokinetic torsional response have been identified. The present study reports that a single line rotating about the line of sight can cause both eyes to tort in the same direction as the stimulus but with a low gain. We have used the term 'entrainment' to describe this torsional response. This paper reports some of the factors associated with entrainment. Video measures of 3-d eye position were recorded while the subject made settings of a simple visual line to subjective visual horizontal (SVH) and vertical (SVV) using the standard method-of-adjustment paradigm. The visual line was composed of 11 light-emitting diodes; the line subtended a visual angle of 19 degrees, and moved at a constant speed of 4.8 degrees /s. Settings were made in an otherwise darkened room, and also in the light. Subjects were required to maintain fixation of the central LED while making settings from starting positions 10 or 20 degrees either side of gravitational horizontal or vertical. We show that entrainment of ocular torsion by the single moving visual line is low in gain but a reliable and repeatable effect and that (1) there are considerable individual differences between subjects but within-subject consistency, (2) all subjects show larger and more consistent torsional entrainment for lines moving to SVH than lines moving to SVV, (3) the strongest entrainment generally occurs within about 10 degrees of the target position, and (4) entrainment is also present in the light, though with slightly reduced gain.

Neural control of three-dimensional eye and head movements

Although the eyes and head can potentially rotate about any three-dimensional axis during orienting gaze shifts, behavioral recordings have shown that certain lawful strategies--such as Listing's law and Donders' law--determine which axis is used for a particular sensory input. Here, we review recent advances in understanding the neuromuscular mechanisms for these laws, the neural mechanisms that control three-dimensional head posture, and the neural mechanisms that coordinate three-dimensional eye orientation with head motion. Finally, we consider how the brain copes with the perceptual consequences of these motor acts.

Three-dimensional ocular kinematics during eccentric rotations: evidence for functional rather than mechanical constraints

Previous studies have reported that the translational vestibuloocular reflex (TVOR) follows a three-dimensional (3D) kinematic behavior that is more similar to visually guided eye movements, like pursuit, rather than the rotational VOR (RVOR). Accordingly, TVOR rotation axes tilted with eye position toward an eye-fixed reference frame rather than staying relatively fixed in the head like in the RVOR. This difference arises because, contrary to the RVOR where peripheral image stability is functionally important, the TVOR like pursuit and saccades cares to stabilize images on the fovea. During most natural head and body movements, both VORs are simultaneously activated. In the present study, we have investigated in rhesus monkeys the 3D kinematics of the combined VOR during yaw rotation about eccentric axes. The experiments were motivated by and quantitatively compared with the predictions of two distinct hypotheses. According to the first (fixed-rule) hypothesis, an eye-position-dependent torsion is computed downstream of a site for RVOR/TVOR convergence, and the combined VOR axis would tilt through an angle that is proportional to gaze angle and independent of the relative RVOR/TVOR contributions to the total eye movement. This hypothesis would be consistent with the recently postulated mechanical constraints imposed by extraocular muscle pulleys. According to the second (image-stabilization) hypothesis, an eye-position-dependent torsion is computed separately for the RVOR and the TVOR components, implying a processing that takes place upstream of a site for RVOR/TVOR convergence. The latter hypothesis is based on the functional requirement that the 3D kinematics of the combined VOR should be governed by the need to keep images stable on the fovea with slip on the peripheral retina being dependent on the different functional goals of the two VORs. In contrast to the fixed-rule hypothesis, the data demonstrated a variable eye-position-dependent torsion for the combined VOR that was different for synergistic versus antagonistic RVOR/TVOR interactions. Furthermore, not only were the eye-velocity tilt slopes of the combined VOR as much as 10 times larger than what would be expected based on extraocular muscle pulley location, but also eye velocity during antagonistic RVOR/TVOR combinations often tilted opposite to gaze. These results are qualitatively and quantitatively consistent with the image-stabilization hypothesis, suggesting that the eye-position-dependent torsion is computed separately for the RVOR and the TVOR and that the 3D kinematics of the combined VOR are dependent on functional rather than mechanical constraints.

Three-dimensional eye-head coordination is implemented downstream from the superior colliculus

How the brain transforms two-dimensional visual signals into multi-dimensional motor commands, and subsequently how it constrains the redundant degrees of freedom, are fundamental problems in sensorimotor control. During fixations between gaze shifts, the redundant torsional degree of freedom is determined by various neural constraints. For example, the eye- and head-in-space are constrained by Donders' law, whereas the eye-in-head obeys Listing's law. However, where and how the brain implements these laws is not yet known. In this study, we show that eye and head movements, elicited by unilateral microstimulations of the superior colliculus (SC) in head-free monkeys, obey the same Donders' strategies observed in normal behavior (i.e., Listing's law for final eye positions and the Fick strategy for the head). Moreover, these evoked movements showed a pattern of three-dimensional eye-head coordination, consistent with normal behavior, where the eye is driven purposely out of Listing's plane during the saccade portion of the gaze shift in opposition to a subsequent torsional vestibuloocular reflex slow phase, such that the final net torsion at the end of each head-free gaze shift is zero. The required amount of saccade-related torsion was highly variable, depending on the initial position of the eye and head prior to a gaze shift and the size of the gaze shift, pointing to a neural basis of torsional control. Because these variable, context-appropriate torsional saccades were correctly elicited by fixed SC commands during head-free stimulations, this shows that the SC only encodes the horizontal and vertical components of gaze, leaving the complexity of torsional organization to downstream control systems. Thus we conclude that Listing's and Donders' laws of the eyes and head, and their three-dimensional coordination mechanisms, must be implemented after the SC.

Magnetic resonance imaging of human extraocular muscles in convergence

Extraocular muscle (EOM) paths during asymmetrical convergence were evaluated by tri-planar, contrast-enhanced magnetic resonance imaging of the orbits of eight young adults during binocular fixation of a target aligned to one eye at 800 and 15 cm distance. Cross sections and paths of EOMs were determined from area centroids. In convergence, the aligned eye rotated and translated negligibly, while its inferior oblique (IO) muscle exhibited significant contractile thickening. There were no significant contractile changes in the cross sections of aligned eye rectus or superior oblique (SO) muscles in convergence. The converging eye rotated nasally 22.4 degrees but translated negligibly. The converging eye medial (MR) and lateral rectus (LR) muscles exhibited large contractile cross-section changes, and the IO showed significant contractile thickening, while the vertical rectus muscles and the SO did not. Anterior paths of three aligned eye rectus EOMs could be determined in convergence and shifted consistent with a 1.9 degrees extorsion of the rectus pulley array. Such extorsional reconfiguration of the rectus pulleys would move the pulleys in coordination with globe extorsion and avoid imparting torsional action to these EOMs. Extorsional rectus pulley shift in convergence is inconsistent with the reconfiguration predicted to explain the temporal tilting of Listing's planes, instead suggesting that this temporal tilting is due to variations in oblique EOM innervation. Absence of globe translation in convergence argues against overall EOM co-contraction. The reconfiguration of EOM geometry in convergence has important implications for single-unit studies of neural control.