The perception of surface curvature from optical motion

Perceiving depth from texture and disparity cues: Evidence for a non-probabilistic account of cue integration

Bayesian inference theories have been extensively used to model how the brain derives three-dimensional (3D) information from ambiguous visual input. In particular, the maximum likelihood estimation (MLE) model combines estimates from multiple depth cues according to their relative reliability to produce the most probable 3D interpretation. Here, we tested an alternative theory of cue integration, termed the intrinsic constraint (IC) theory, which postulates that the visual system derives the most stable, not most probable, interpretation of the visual input amid variations in viewing conditions. The vector sum model provides a normative approach for achieving this goal where individual cue estimates are components of a multidimensional vector whose norm determines the combined estimate. Individual cue estimates are not accurate but related to distal 3D properties through a deterministic mapping. In three experiments, we show that the IC theory can more adeptly account for 3D cue integration than MLE models. In Experiment 1, we show systematic biases in the perception of depth from texture and depth from binocular disparity. Critically, we demonstrate that the vector sum model predicts an increase in perceived depth when these cues are combined. In Experiment 2, we illustrate the IC theory radical reinterpretation of the just noticeable difference (JND) and test the related vector sum model prediction of the classic finding of smaller JNDs for combined-cue versus single-cue stimuli. In Experiment 3, we confirm the vector sum prediction that biases found in cue integration experiments cannot be attributed to flatness cues, as the MLE model predicts.

The case against probabilistic inference: a new deterministic theory of 3D visual processing

How the brain derives 3D information from inherently ambiguous visual input remains the fundamental question of human vision. The past two decades of research have addressed this question as a problem of probabilistic inference, the dominant model being maximum-likelihood estimation (MLE). This model assumes that independent depth-cue modules derive noisy but statistically accurate estimates of 3D scene parameters that are combined through a weighted average. Cue weights are adjusted based on the system representation of each module's output variability. Here I demonstrate that the MLE model fails to account for important psychophysical findings and, importantly, misinterprets the just noticeable difference, a hallmark measure of stimulus discriminability, to be an estimate of perceptual uncertainty. I propose a new theory, termed Intrinsic Constraint, which postulates that the visual system does not derive the most probable interpretation of the visual input, but rather, the most stable interpretation amid variations in viewing conditions. This goal is achieved with the Vector Sum model, which represents individual cue estimates as components of a multi-dimensional vector whose norm determines the combined output. This model accounts for the psychophysical findings cited in support of MLE, while predicting existing and new findings that contradict the MLE model. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Form and Function in Information for Visual Perception

Visual perception involves spatially and temporally coordinated variations in diverse physical systems: environmental surfaces and symbols, optical images, electro-chemical activity in neural networks, muscles, and bodily movements—each with a distinctly different material structure and energy. The fundamental problem in the theory of perception is to characterize the information that enables both perceptual awareness and real-time dynamic coordination of these diverse physical systems. Gibson's psychophysical and ecological conception of this problem differed from that of mainstream science both then and now. The present article aims to incorporate Gibson's ideas within a general conception of information for visual perception. We emphasize the essential role of spatiotemporal form, in contrast with symbolic information. We consider contemporary understanding of surface structure, optical images, and optic flow. Finally, we consider recent evidence about capacity limitations on the rate of visual perception and implications for the ecology of vision.

The many facets of shape

Shape is an interesting property of objects because it is used in ordinary discourse in ways that seem to have little connection to how it is typically defined in mathematics. The present article describes how the concept of shape can be grounded within Euclidean and non-Euclidean geometry and also to human perception. It considers the formal methods that have been proposed for measuring the differences among shapes and how the performance of those methods compares with shape difference thresholds of human observers. It discusses how different types of shape change can be perceptually categorized. It also evaluates the specific data structures that have been used to represent shape in models of both human and machine vision, and it reviews the psychophysical evidence about the extent to which those models are consistent with human perception. Based on this review of the literature, we argue that shape is not one thing but rather a collection of many object attributes, some of which are more perceptually salient than others. Because the relative importance of these attributes can be context dependent, there is no obvious single definition of shape that is universally applicable in all situations.

Age Effects on Visuo-Haptic Length Discrimination: Evidence for Optimal Integration of Senses in Senior Adults

Demographic changes in most developed societies have fostered research on functional aging. While cognitive changes have been characterized elaborately, understanding of perceptual aging lacks behind. We investigated age effects on the mechanisms of how multiple sources of sensory information are merged into a common percept. We studied visuo-haptic integration in a length discrimination task. A total of 24 young (20-25 years) and 27 senior (69-77 years) adults compared standard stimuli to appropriate sets of comparison stimuli. Standard stimuli were explored under visual, haptic, or visuo-haptic conditions. The task procedure allowed introducing an intersensory conflict by anamorphic lenses. Comparison stimuli were exclusively explored haptically. We derived psychometric functions for each condition, determining points of subjective equality and discrimination thresholds. We notably evaluated visuo-haptic perception by different models of multisensory processing, i.e., the Maximum-Likelihood-Estimate model of optimal cue integration, a suboptimal integration model, and a cue switching model. Our results support robust visuo-haptic integration across the adult lifespan. We found suboptimal weighted averaging of sensory sources in young adults, however, senior adults exploited differential sensory reliabilities more efficiently to optimize thresholds. Indeed, evaluation of the MLE model indicates that young adults underweighted visual cues by more than 30%; in contrast, visual weights of senior adults deviated only by about 3% from predictions. We suggest that close to optimal multisensory integration might contribute to successful compensation for age-related sensory losses and provides a critical resource. Differentiation between multisensory integration during healthy aging and age-related pathological challenges on the sensory systems awaits further exploration.

Local Solid Shape

Local solid shape applies to the surface curvature of small surface patches—essentially regions of approximately constant curvatures—of volumetric objects that are smooth volumetric regions in Euclidean 3-space. This should be distinguished from local shape in pictorial space. The difference is categorical. Although local solid shape has naturally been explored in haptics, results in vision are not forthcoming. We describe a simple experiment in which observers judge shape quality and magnitude of cinematographic presentations. Without prior training, observers readily use continuous shape index and Casorati curvature scales with reasonable resolution.

If you watch it move, you'll recognize it in 3D: Transfer of depth cues between encoding and retrieval

Viewing objects with stereoscopic displays provides additional depth cues through binocular disparity supporting object recognition. So far, it was unknown whether this results from the representation of specific stereoscopic information in memory or a more general representation of an object's depth structure. Therefore, we investigated whether continuous object rotation acting as depth cue during encoding results in a memory representation that can subsequently be accessed by stereoscopic information during retrieval. In Experiment 1, we found such transfer effects from continuous object rotation during encoding to stereoscopic presentations during retrieval. In Experiments 2a and 2b, we found that the continuity of object rotation is important because only continuous rotation and/or stereoscopic depth but not multiple static snapshots presented without stereoscopic information caused the extraction of an object's depth structure into memory. We conclude that an object's depth structure and not specific depth cues are represented in memory.

What is binocular disparity?

What are the geometric primitives of binocular disparity? The Venetian blind effect and other converging lines of evidence indicate that stereoscopic depth perception derives from disparities of higher-order structure in images of surfaces. Image structure entails spatial variations of intensity, texture, and motion, jointly structured by observed surfaces. The spatial structure of binocular disparity corresponds to the spatial structure of surfaces. Independent spatial coordinates are not necessary for stereoscopic vision. Stereopsis is highly sensitive to structural disparities associated with local surface shape. Disparate positions on retinal anatomy are neither necessary nor sufficient for stereopsis.

The influence of intersensory discrepancy on visuo-haptic integration is similar in 6-year-old children and adults

When participants are given the opportunity to simultaneously feel an object and see it through a magnifying or reducing lens, adults estimate object size to be in-between visual and haptic size. Studies with young children, however, seem to demonstrate that their estimates are dominated by a single sense. In the present study, we examined whether this age difference observed in previous studies, can be accounted for by the large discrepancy between felt and seen size in the stimuli used in those studies. In addition, we studied the processes involved in combining the visual and haptic inputs. Adults and 6-year-old children judged objects that were presented to vision, haptics or simultaneously to both senses. The seen object length was reduced or magnified by different lenses. In the condition inducing large intersensory discrepancies, children's judgments in visuo-haptic conditions were almost dominated by vision, whereas adults weighted vision just by ~40%. Neither the adults' nor the children's discrimination thresholds were predicted by models of visuo-haptic integration. With smaller discrepancies, the children's visual weight approximated that of the adults and both the children's and adults' discrimination thresholds were well predicted by an integration model, which assumes that both visual and haptic inputs contribute to each single judgment. We conclude that children integrate seemingly corresponding multisensory information in similar ways as adults do, but focus on a single sense, when information from different senses is strongly discrepant.

Visual and haptic integration in the estimation of softness of deformable objects

Softness perception intrinsically relies on haptic information. However, through everyday experiences we learn correspondences between felt softness and the visual effects of exploratory movements that are executed to feel softness. Here, we studied how visual and haptic information is integrated to assess the softness of deformable objects. Participants discriminated between the softness of two softer or two harder objects using only-visual, only-haptic or both visual and haptic information. We assessed the reliabilities of the softness judgments using the method of constant stimuli. In visuo-haptic trials, discrepancies between the two senses' information allowed us to measure the contribution of the individual senses to the judgments. Visual information (finger movement and object deformation) was simulated using computer graphics; input in visual trials was taken from previous visuo-haptic trials. Participants were able to infer softness from vision alone, and vision considerably contributed to bisensory judgments (∼35%). The visual contribution was higher than predicted from models of optimal integration (senses are weighted according to their reliabilities). Bisensory judgments were less reliable than predicted from optimal integration. We conclude that the visuo-haptic integration of softness information is biased toward vision, rather than being optimal, and might even be guided by a fixed weighting scheme.

Fechner, information, and shape perception

How do retinal images lead to perceived environmental objects? Vision involves a series of spatial and material transformations--from environmental objects to retinal images, to neurophysiological patterns, and finally to perceptual experience and action. A rationale for understanding functional relations among these physically different systems occurred to Gustav Fechner: Differences in sensation correspond to differences in physical stimulation. The concept of information is similar: Relationships in one system may correspond to, and thus represent, those in another. Criteria for identifying and evaluating information include (a) resolution, or the precision of correspondence; (b) uncertainty about which input (output) produced a given output (input); and (c) invariance, or the preservation of correspondence under transformations of input and output. We apply this framework to psychophysical evidence to identify visual information for perceiving surfaces. The elementary spatial structure shared by objects and images is the second-order differential structure of local surface shape. Experiments have shown that human vision is directly sensitive to this higher-order spatial information from interimage disparities (stereopsis and motion parallax), boundary contours, texture, shading, and combined variables. Psychophysical evidence contradicts other common ideas about retinal information for spatial vision and object perception.

Does this computational theory solve the right problem? Marr, Gibson, and the goal of vision

David Marr's book Vision attempted to formulate athoroughgoing formal theory of perception. Marr borrowed much of the "computational" level from James Gibson: a proper understanding of the goal of vision, the natural constraints, and the available information are prerequisite to describing the processes and mechanisms by which the goal is achieved. Yet, as a research program leading to a computational model of human vision, Marr's program did not succeed. This article asks why, using the perception of 3D shape as a morality tale. Marr presumed that the goal of vision is to recover a general-purpose Euclidean description of the world, which can be deployed for any task or action. On this formulation, vision is underdetermined by information, which in turn necessitates auxiliary assumptions to solve the problem. But Marr's assumptions did not actually reflect natural constraints, and consequently the solutions were not robust. We now know that humans do not in fact recover Euclidean structure--rather, they reliably perceive qualitative shape (hills, dales, courses, ridges), which is specified by the second-order differential structure of images. By recasting the goals of vision in terms of our perceptual competencies, and doing the hard work of analyzing the information available under ecological constraints, we can reformulate the problem so that perception is determined by information and prior knowledge is unnecessary.

The selectivity of neurons in the macaque fundus of the superior temporal area for three-dimensional structure from motion

Motion is a potent cue for the perception of three-dimensional (3D) shape in primates, but little is known about its underlying neural mechanisms. Guided by recent functional magnetic resonance imaging results, we tested neurons in the fundus of the superior temporal sulcus (FST) area of two macaque monkeys (Macaca mulatta, one male) using motion-defined surface patches with various 3D shapes such as slanted planes, saddles, or cylinders. The majority of the FST neurons (>80%) were selective for stimuli depicting specific shapes, and all the surfaces tested were represented among the selective FST neurons. Importantly, this selectivity tolerated changes in speed, position, size, or between binocular and monocular presentations. This tolerance demonstrates that the 3D structure-from-motion (3D-SFM) selectivity of FST neurons is a higher-order selectivity, which cannot be reduced to a lower-order speed selectivity. The 3D-SFM selectivity of FST neurons was unaffected by removal of the opposed-motion cue that supplemented the speed gradient cue in the standard stimuli. When tested with the same standard stimuli, fewer neurons in the middle temporal/visual 5 (MT/V5) area were selective than FST neurons. In addition, selective MT/V5 neurons represented fewer types of surfaces and were less tolerant of stimulus changes than FST neurons. Overall, these results indicate that FST neurons code motion-defined 3D shape fragments, underscoring the central role of FST in processing 3D-SFM.

Large perspective changes yield perception of metric shape that allows accurate feedforward reaches-to-grasp and it persists after the optic flow has stopped!

Lee et al. (Percept Psychophys 70:1032-1046, 2008a) investigated whether visual perception of metric shape could be calibrated when used to guide feedforward reaches-to-grasp. It could not. Seated participants viewed target objects (elliptical cylinders) in normal lighting using stereo vision and free head movements that allowed small (approximately 10 degrees) perspective changes. The authors concluded that poor perception of metric shape was the reason reaches-to-grasp should be visually guided online. However, Bingham and Lind (Percept Psychophys 70:524-540, 2008) showed that large perspective changes (> or =45 degrees) yield good perception of metric shape. So, now we repeated the Lee et al.'s study with the addition of information from large perspective changes. The results were accurate feedforward reaches-to-grasp reflecting accurate perception of both metric shape and metric size. Large perspective changes occur when one locomotes into a workspace in which reaches-to-grasp are subsequently performed. Does the resulting perception of metric shape persist after the large perspective changes have ceased? Experiments 2 and 3 tested reaches-to-grasp with delays (Exp. 2, 5-s delay; Exp. 3, approximately 16-s delay) and multiple objects to be grasped after a single viewing. Perception of metric shape and metric size persisted yielding accurate reaches-to-grasp. We advocate the study of nested actions using a dynamic approach to perception/action.

Haptic shape perception from force and position signals varies with exploratory movement direction and the exploring finger

We investigated how exploratory movement influences signal integration in active touch. Participants judged the amplitude of a bump specified by redundant signals: When a finger slides across a bump, the finger's position follows the bump's geometry (position signal); simultaneously, it is exposed to patterns of forces depending on the gradient of the bump (force signal). We varied amplitudes specified by force signals independently of amplitudes specified by position signals. Amplitude judgment was a weighted linear function of the amplitudes specified by both signals, under different exploratory conditions. The force signal's contribution to the judgment was higher when the participants explored with the index finger, as opposed to the thumb, and when they explored along a tangential axis, as opposed to a radial one (pivot congruent with shoulder joint). Furthermore, for tangential, as compared with radial, axis exploration, amplitude judgments were larger (and more accurate), and amplitude discrimination was better. We attribute these exploration-induced differences to biases in estimating bump amplitude from force signals. Given the choice, the participants preferred tangential explorations with the index finger-a behavior that resulted in good discrimination performance. A role for an active explorer, as well as biases that depend on exploration, should be taken into account when signal integration models are extended to active touch.

Material properties determine how force and position signals combine in haptic shape perception

When integrating estimates from redundant sensory signals, humans seem to weight these estimates according to their reliabilities. In the present study, human observers used active touch to judge the curvature of a shape. The curvature was specified by positional and force signals: When a finger slides across a surface, the finger's position follows the surface geometry (position signal). At the same time, it is exposed to patterns of forces depending on the gradient of the surface (force signal; Robles-de-la-Torre, G., & Hayward, V. (2001). Force can overcome object geometry in the perception of shape through active touch. Nature, 412, 445-448). We show that variations in the surface's material properties (compliance, friction) influence the sensorily available position and force signals, as well as the noise associated with these signals. Along with this, material properties affect the weights given to the position and force signals for curvature judgements. Our findings are consistent with the notion of an observer who weights signal estimates according to their reliabilities. That is, signal weights shifted with the signal noise, which in the present case resulted from active exploration.

Visual discrimination of local surface structure: slant, tilt, and curvedness

In four experiments, observers were required to discriminate interval or ordinal differences in slant, tilt, or curvedness between designated probe points on randomly shaped curved surfaces defined by shading, texture, and binocular disparity. The results reveal that discrimination thresholds for judgments of slant or tilt typically range between 4 degrees and 10 degrees; that judgments of one component are unaffected by simultaneous variations in the other; and that the individual thresholds for either the slant or tilt components of orientation are approximately equal to those obtained for judgments of the total orientation difference between two probed regions. Performance was much worse, however, for judgments of curvedness, and these judgments were significantly impaired when there were simultaneous variations in the shape index parameter of curvature.

Visual perception of surface wrinkles

To study the relationship between visual perception of magnitude of wrinkles and geometrical parameters of surfaces, four potentially relevant parameters of the surface profile were considered: the variance (sigma2), the cutting frequency (Fc), the effective disparity curvature (Dce) of the wrinkled surface over the eyeball distance of the observer, and the frequency component of the disparity curvature (Dcf). Analysis of garment seams with varying amount of pucker showed that, while the logarithm for each of these four parameters has a strong linear relationship with the visually perceived magnitude of wrinkles, following the Fechner Law, the effective disparity curvature (Dce) and the frequency component of the disparity curvature (Dcf) with visual perception appeared stronger. This modeling may be an objective method for measuring magnitude of surface wrinkles.

Integration of force and position cues for shape perception through active touch

This article systematically explores cue integration within active touch. Our research builds upon a recently made distinction between position and force cues for haptic shape perception: when sliding a finger across a bumpy surface, the finger follows the surface geometry (position cue). At the same time, the finger is exposed to forces related to the slope of the surface (force cue). Experiment 1 independently varied force and position cues to the curvature of 3D arches. Perceived curvature could be well described as a weighted average of the two cues. Experiment 2 found more weight of the position cue for more convex high arches and higher weight of the force cue for less convex shallow arches--probably mediated through a change in relative cue reliability. Both findings are in good agreement with the maximum-likelihood estimation (MLE) model for cue integration and, thus, carry this model over to the domain of active haptic perception.

Do humans integrate routes into a cognitive map? Map- versus landmark-based navigation of novel shortcuts

Do humans integrate experience on specific routes into metric survey knowledge of the environment, or do they depend on a simpler strategy of landmark navigation? The authors tested this question using a novel shortcut paradigm during walking in a virtual environment. The authors find that participants could not take successful shortcuts in a desert world but could do so with dispersed landmarks in a forest. On catch trials, participants were drawn toward the displaced landmarks whether the landmarks were clustered near the target location or along the shortcut route. However, when landmarks appeared unreliable, participants fell back on coarse survey knowledge. Like honeybees (F. C. Dyer, 1991), humans do not appear to derive accurate cognitive maps from path integration to guide navigation but, instead, depend on landmarks when they are available.

Spatial integration in structure from motion

In three experiments we investigated whether the perception of 3D structure from the optic-flow involves a process of spatial integration. The observer's task was to judge the 3D orientation of local velocity field patches. In two conditions, the patches were presented either in isolation, or as part of a global optic-flow. In Experiment 1, the global optic-flow was a linear velocity field. In Experiment 2, the patches were embedded in a randomly perturbed linear velocity field. In Experiment 3, the local patches belonged to a smoothly curved surface. The results of these three experiments lead to two main conclusions: (1) a process linking spatially separated patches into global entities does affect the perception of local surface orientation induced by the optic-flow, and (2) linearity or smoothness of the global velocity field are not necessary conditions for spatial integration.

Evidence for patchwork approximation of shape primitives

Investigators have proposed that qualitative shapes are the primitive information of spatial vision: They preserve an approximately one-to-one mapping between surfaces, images, and perception. Given their importance, we examined how the visual system recovers these primitives from sparse disparity fields that do not provide sufficient information for their recovery. We hypothesized that the visual system interpolates sparse disparities with planes, resulting in a patchwork approximation of the implicitly defined shapes. We presented observers with stereo displays simulating planar or smooth curved surfaces having different curvatures. The observers' task was to detect whether dots deviated from these surfaces or to discriminate planar from curved or planar from scrambled surfaces. Consistent with our hypothesis, increasing curvature had detrimental effects on observers' performance (Experiments 1-3). Importantly, this patchwork approximation leads to the recovery of the proposed shape primitives, since observers were more accurate at discriminating planar-from-curved than planar-from-scrambled surfaces with matched disparity range (Experiment 4).

Perspective distortion of trajectory forms and perceptual constancy in visual event identification

Previous studies have shown that people can use the information in trajectory forms to recognize visual events. A trajectory form is composed of the path of motion and the change in speed along that path. In past studies, however, only sensitivity to trajectory forms viewed from a single perspective was examined. The optical components change when an event is viewed from different perspectives, and the projected form of the trajectory is transformed. Does event recognition exhibit constancy despite these changes? In Experiment 1, participants were familiarized with five different trajectory forms viewed from a single perspective. Then the participants had to identify the same events viewed from different perspectives: from the side, at an angle, and entirely in depth. The participants exhibited perceptual constancy. Experiment 2 revealed, however, that both the change in optical components and the perspective transformations affected recognition.

The visual perception of 3-D shape from multiple cues: are observers capable of perceiving metric structure?

Three experiments are reported in which observers judged the three-dimensional (3-D) structures of virtual or real objects defined by various combinations of texture, motion, and binocular disparity under a wide variety of conditions. The tasks employed in these studies involved adjusting the depth of an object to match its width, adjusting the planes of a dihedral angle so that they appeared orthogonal, and adjusting the shape of an object so that it appeared to match another at a different viewing distance. The results obtained on all of these tasks revealed large constant errors and large individual differences among observers. There were also systematic failures of constancy over changes in viewing distance, orientation, or response task. When considered in conjunction with other, similar reports in the literature, these findings provide strong evidence that human observers do not have accurate perceptions of 3-D metric structure.

Short-term temporal recruitment in structure from motion

Temporal integration was investigated in the minimal conditions necessary to perform a structure-from-motion (SFM) task. Observers were asked to discriminate three-dimensional (3D) surface orientations in conditions in which the stimulus displays simulated velocity fields providing, in each frame transition, either sufficient (3 moving dots) or insufficient information (1 or 2 moving dots) to perform the task. When only two moving dots were shown in each frame transition of the stimulus displays (Experiment 1), we found that performance decreased as dot-lifetime increased. A facilitation effect of the overall display duration was also found. The negative effect of dot-lifetime on performance contrasts with what found in Experiment 2 with three dots in each frame transition, where performance improved with increasing dot-lifetime up to 170 ms, and then reached a plateau. Finally, for an optimal dot-lifetime of 150 ms, we found that performance was still above chance when each frame transition specified the motion of only one dot (Experiment 3). These results indicate that temporal recruitment alone can support the recovery of 3D information from sparse motion signals, thus providing a strong indication for the importance of temporal integration in the perceptual analysis of the optic flow. Our results reveal, moreover, that temporal integration in SFM has different characteristics, depending on whether, in each frame transition, the stimulus displays provide either sufficient (3 or more moving dots) or insufficient information (1 or 2 moving dots) to specify the higher-order properties of the optic flow necessary for 3D surface recovery.

Trajectory forms as a source of information about events

The ability to use trajectory forms as visual information about events was tested. A trajectory form is defined as the variation in velocity along a path of motion. In Experiment 1, we tested the ability to detect trajectory form differences between simulations of a freely swinging pendulum and a hand-moved pendulum. The trajectory form of the freely swinging pendulum was symmetric around the mid-point, whereas the hand moved was not. In Experiment 2, we isolated trajectory form information by varying the amplitudes of events while holding their periods constant. Straight path versions of the harmonic events from Experiment 1 were tested. In Experiment 3, we tested sensitivity to symmetrical peakening or flattening of trajectory forms. Participants detected small differences in all three experiments. In Experiment 4, we tested the ability to identify specific events based only on differences in trajectory forms. Participants were able to identify four different events. We investigated properties of trajectory forms that might potentially be detected and used as information, and we found that the curvature yielded good results.

Coherence of early motion signals

Oscillation thresholds were evaluated for detecting motion and discriminating relative motion. Three horizontally aligned Gaussian blobs oscillated horizontally, with the center in-phase or out-of-phase with the two flankers. Motion thresholds were well below those for static bisection, and involved small contrast changes (<0.25%). Remarkably, acuity was better for discriminating phase relations than for detecting rigid motion, averaging 8.7 and 11.0 arcsec, respectively, for 100 arcmin between blobs. Phase discrimination acuities were robust over separations of 20-320 arcmin and temporal frequencies of 1.5-6 Hz. Motion phase relations must be coherent among spatially separate retinal signals, carrying information about intrinsic image structure.

Experience-dependent visual cue integration based on consistencies between visual and haptic percepts

We study the hypothesis that observers can use haptic percepts as a standard against which the relative reliabilities of visual cues can be judged, and that these reliabilities determine how observers combine depth information provided by these cues. Using a novel visuo-haptic virtual reality environment, subjects viewed and grasped virtual objects. In Experiment 1, subjects were trained under motion relevant conditions, during which haptic and visual motion cues were consistent whereas haptic and visual texture cues were uncorrelated, and texture relevant conditions, during which haptic and texture cues were consistent whereas haptic and motion cues were uncorrelated. Subjects relied more on the motion cue after motion relevant training than after texture relevant training, and more on the texture cue after texture relevant training than after motion relevant training. Experiment 2 studied whether or not subjects could adapt their visual cue combination strategies in a context-dependent manner based on context-dependent consistencies between haptic and visual cues. Subjects successfully learned two cue combination strategies in parallel, and correctly applied each strategy in its appropriate context. Experiment 3, which was similar to Experiment 1 except that it used a more naturalistic experimental task, yielded the same pattern of results as Experiment 1 indicating that the findings do not depend on the precise nature of the experimental task. Overall, the results suggest that observers can involuntarily compare visual and haptic percepts in order to evaluate the relative reliabilities of visual cues, and that these reliabilities determine how cues are combined during three-dimensional visual perception.

Experience-dependent integration of texture and motion cues to depth

Previous investigators have shown that observers' visual cue combination strategies are remarkably flexible in the sense that these strategies adapt on the basis of the estimated reliabilities of the visual cues. However, these researchers have not addressed how observers' acquire these estimated reliabilities. This article studies observers' abilities to learn cue combination strategies. Subjects made depth judgments about simulated cylinders whose shapes were indicated by motion and texture cues. Because the two cues could indicate different shapes, it was possible to design tasks in which one cue provided useful information for making depth judgments, whereas the other cue was irrelevant. The results of experiment 1 suggest that observers' cue combination strategies are adaptable as a function of training; subjects adjusted their cue combination rules to use a cue more heavily when the cue was informative on a task versus when the cue was irrelevant. Experiment 2 demonstrated that experience-dependent adaptation of cue combination rules is context-sensitive. On trials with presentations of short cylinders, one cue was informative, whereas on trials with presentations of tall cylinders, the other cue was informative. The results suggest that observers can learn multiple cue combination rules, and can learn to apply each rule in the appropriate context. Experiment 3 demonstrated a possible limitation on the context-sensitivity of adaptation of cue combination rules. One cue was informative on trials with presentations of cylinders at a left oblique orientation, whereas the other cue was informative on trials with presentations of cylinders at a right oblique orientation. The results indicate that observers did not learn to use different cue combination rules in different contexts under these circumstances. These results are consistent with the hypothesis that observers' visual systems are biased to learn to perceive in the same way views of bilaterally symmetric objects that differ solely by a symmetry transformation. Taken in conjunction with the results of Experiment 2, this means that the visual learning mechanism underlying cue combination adaptation is biased such that some sets of statistics are more easily learned than others.

Human cortical regions involved in extracting depth from motion

We used functional magnetic resonance imaging (fMRI) to investigate brain regions involved in extracting three-dimensional structure from motion. A factorial design included two-dimensional and three-dimensional structures undergoing rigid and nonrigid motions. As predicted from monkey data, the human homolog of MT/V5 was significantly more active when subjects viewed three-dimensional (as opposed to two-dimensional) displays, irrespective of their rigidity. Human MT/V5+ (hMT/V5+) is part of a network with right hemisphere dominance involved in extracting depth from motion, including a lateral occipital region, five sites along the intraparietal sulcus (IPS), and two ventral occipital regions. Control experiments confirmed that this pattern of activation is most strongly correlated with perceived three-dimensional structure, in as much as it arises from motion and cannot be attributed to numerous two-dimensional image properties or to saliency.

The visual perception of surface orientation from optical motion

Observers viewed monocular animations of rotating dihedral angles and were required to indicate their perceived structures by adjusting the magnitude and orientation of a stereoscopic dihedral angle. The motion displays were created by directly manipulating various aspects of the image velocity field, including the mean translation, the horizontal and vertical velocity gradients, and the manner in which these gradients changed over time. The adjusted orientation of each planar facet was decomposed into components of slant and tilt. Although the tilt component was estimated with a high degree of accuracy, the judgments of slant exhibited large systematic errors. The magnitude of perceived slant was determined primarily by the magnitude of the velocity gradient scaled by its direction. The results also indicate that higher order temporal derivatives of the moving elements had little effect on observers' judgments.

Optimal integration of texture and motion cues to depth

We report the results of a depth-matching experiment in which subjects were asked to adjust the height of an ellipse until it matched the depth of a simulated cylinder defined by texture and motion cues. In one-third of the trials the shape of the cylinder was primarily given by motion information, in another one-third of the trials it was given by texture information, and on the remaining trials it was given by both sources of information. Two optimal cue combination models are described where optimality is defined in terms of Bayesian statistics. The parameter values of the models are set based on subjects' responses on trials when either the motion cue or the texture cue was informative. These models provide predictions of subjects' responses on trials when both cues were informative. The results indicate that one of the optimal models provides a good fit to the subjects' data, and the second model provides an exceptional fit. Because the predictions of the optimal models closely match the experimental data, we conclude that observers' cue-combination strategies are indeed optimal, at least under the conditions studied here.

Modeling the combination of motion, stereo, and vergence angle cues to visual depth

Three models of visual cue combination were simulated: a weak fusion model, a modified weak model, and a strong model. Their relative strengths and weaknesses are evaluated on the basis of their performances on the tasks of judging the depth and shape of an ellipse. The models differ in the amount of interaction that they permit among the cues of stereo, motion, and vergence angle. Results suggest that the constrained nonlinear interaction of the modified weak model allows better performance than either the linear interaction of the weak model or the unconstrained nonlinear interaction of the strong model. Further examination of the modified weak model revealed that its weighting of motion and stereo cues was dependent on the task, the viewing distance, and, to a lesser degree, the noise model. Although the dependencies were sensible from a computational viewpoint, they were sometimes inconsistent with psychophysical experimental data. In a second set of experiments, the modified weak model was given contradictory motion and stereo information. One cue was informative in the sense that it indicated an ellipse, while the other cue indicated a flat surface. The modified weak model rapidly reweighted its use of stereo and motion cues as a function of each cue's informativeness. Overall, the simulation results suggest that relative to the weak and strong models, the modified weak fusion model is a good candidate model of the combination of motion, stereo, and vergence angle cues, although the results also highlight areas in which this model needs modification or further elaboration.